National Academies Press: OpenBook

Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions (2022)

Chapter: 6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions

« Previous: 5 Barriers and Opportunities in NASA Space Missions
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×

6

Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal Investigator–Led Missions

Throughout this report the committee has attempted to delineate the end-to-end mission proposal process; the extent of demographic diversity within the National Aeronautics and Space Administration (NASA) Science Mission Directorate (SMD)-related fields, workforces and proposer pools; as well as the variety of factors underlying these realities. In the course of examining these areas, several issues emerged that warrant further attention in an effort to articulate clear paths forward for NASA SMD, as it relates to realizing its stated goal to increase diversity, equity, inclusion, and accessibility (DEIA) in the leadership of competed space missions. A growing body of literature, as well as an increasing number of examples from other institutions and organizations have put forth strategies and practices that NASA SMD can either adopt or adapt to make progress in a number of areas that can aggregately lead to the desired outcomes for principal investigator (PI)-led missions.

In the charge to the committee, other federal agencies, such as the National Science Foundation (NSF) and the National Institutes of Health (NIH), were specifically cited as references to consider in providing guidance for overcoming barriers to diversity and inclusion in the context of funding large, PI-led, multi-institutional projects. In looking across the landscape of federal agencies, the committee found that other organizations, such as the Department of Energy (DOE) and the Department of Defense (DoD), utilize vehicles similar to the NASA announcement of opportunity (AO) when procuring large-scale goods and services, albeit not particularly well defined or understood as can be the case with new technologies. In this chapter, we examine the policies, processes, and education/development programs at representative federal science agencies and organizations. The committee also examines the extant literature, and in the committee’s judgment, present the practices that are currently considered the best to foster DEIA in the following areas: (1) proposal review, (2) data collection, (3) accountability, and (4) investment in science, technology, engineering, and mathematics (STEM) pathways.

DEFINITIONS OF BEST PRACTICES

The term “best practice” is widely used in a number of sectors such as business, industry, education, research, and public policy; but often used without an accompanying definition for what this designation actually means in the respective context. In a recent report, Best Practices for Diversity and Inclusion in STEM Education and Research: A Guide for and by Federal Agencies, a “best practice” is defined as “a procedure that has been shown by research and experience to produce optimal results and that is established or proposed as a standard suitable for widespread adoption” (NSTC 2021). Additionally, the report outlines three levels to distinguish among best

Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×

practices based on the extent of the evidence-base supporting the effectiveness of the approach: evidence-based practices, promising practices, and emerging practices; as well as definitions of each term in order to standardize their use and improve overall reporting of best practices by federal agencies.

Evidence-based practices are defined as those that are grounded in research and evaluation and have met some established test of validity in improving a specific outcome. Promising practices are those that have been successfully implemented, but sufficient evidence that substantiates all the parameters associated with success of the practice has not been collected or generated. Additionally, practices that are known to be “evidence-based” in one context, would be defined as “promising” when being applied in a different context. Finally, emerging practices are those that are new, innovative, or exploratory in nature, and while they may be based on some level of evidence, it is not sufficient for the practices to be considered “promising” or “evidence-based.” In making determinations about relevant policies, processes, and programs at federal agencies for addressing DEIA, and for consistency with designations already in place across federal agencies, the committee adopted these definitions in the context of this report. Additionally, it is important to note that the committee did not identify examples of evidence-based, promising, and emerging practices in every area discussed.

FOSTERING DEIA IN PROPOSAL REVIEW FOR SCIENCE AND ENGINEERING RESEARCH ACTIVITIES

In Chapter 2 of this report, we described the comprehensive process through which proposal review for competed space missions occurs as governed by section 1872.4 of the NASA Federal Acquisition Regulation (FAR) Supplement. The needs in terms of what is required to put together a winning proposal (i.e., compelling science investigation, technically feasible, and programmatically credible) vary across mission scope and size, and becoming a PI or co-investigator (Co-I) on a winning proposal is typically a long training and development process for the investigators as well as the proposing team as a whole. The resources required for such proposal development in terms of dollars and time are not negligible. Neither are investments for the awarded projects negligible, ranging from tens of millions of dollars—over $1 billion awarded funds to winning proposals. Thus, the need for such extensive resources creates a challenging environment for both proposers and reviewers. Yet, despite efforts on the part of NASA to ensure that this stage of the proposal process is carried out responsibly and with fairness and objectivity, the influence that implicit and explicit bias can have on the results of proposal review are significant (see Chapters 2 and 5).

To this end, NASA is not the only agency facing this challenge, and in fact both NSF and NIH have piloted experiments to identify and reduce the impact of any possible bias in grant proposal evaluation in response to findings of an uneven playing field between proposers (EOP and OPM 2016). In the case of NIH, substantial funding disparities were found to exist between African American/Black and White applicants, even when controlling for demographics, education, training, employer characteristics, NIH experience, and research productivity (Ginther et al. 2011). African American/Black scientists were significantly less likely to receive Research Project Grant (R01) funding compared to White scientists. Some important factors that were found to be associated with the observed funding gap included assignment of poorer scores and decisions to discuss an application or not—factors that are at the hands of reviewers (Hoppe et al. 2019; Erosheva et al. 2020). Clearly R01 grants and large acquisitions such as NASA space missions selected through a competitive process that is governed by FAR are not equivalent, and so this case is only meant to be an illustration of the impact that possible bias can have in the context of federal research funding, and possible measures to take in assessing and addressing it.

Promising Practices for Assessing the Proposal Review Process

To understand whether bias in proposal review contributes to the observed funding disparities, the NIH has employed multiple promising approaches. These include testing the utility of anonymizing grant applications before review; conducting more research to gain a richer understanding of scientific, technical, and demographic issues that might be at play; using text-mining software to examine applications and reviewer critiques for evidence of potential bias; reexamining the evaluation process used to review grants for indicators of bias on a continual basis;

Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×

and providing opportunities for early-stage researchers to serve on review panels (Valantine and Collins 2015; Nakamura et al. 2021). Additionally, the NIH Office of Extramural Research is engaging in several analyses of applicant flow data to examine differences between groups (e.g., according to race/ethnicity, disability status, etc.) in their matriculation through the proposal process from application to selections.

NSF’s Merit Review process is a mechanism by which the agency ensures that the competitive peer review of proposals submitted for support identifies proposals that are of high-quality and advance scientific knowledge, and is carried out with biases and conflicts minimized. Additionally, this process ensures that proposers are provided constructive and useful feedback. As part of proposal review, reviewers receive a short orientation that covers topics such as writing analytic reviews, the broader impacts criterion, and training on ways to reduce the impact of unconscious bias including anchoring bias, confirmation bias, halo effect, and language bias (NSF 2020). Additionally, an external Committee of Visitors periodically examines the integrity and efficiency of the merit review process, and the resulting reports along with NSF responses are made publicly available. With these strategies in place, a 2019 Survey of proposers and reviewers indicated that 74% of survey respondents agreed their proposal was treated fairly in the review process, despite the fact that 73% of their proposals could not be funded. Moreover, the impact of these strategies on the participation of different groups in the agency’s award portfolio is documented and openly reported on an annual basis.

Promising Practices for Proposal Evaluation

A substantial amount of research evidence demonstrates that diversity and inclusion positively impact innovation and the quality and outputs of science (Cohen et al. 2002; Page 2007; Bell et al. 2011; Hofstra et al. 2020). In fact, much of the legislation around broadening participation in STEM during the Obama administration was motivated by the acknowledgment that enriching the STEM enterprise and the innovations and solutions it produces, is inextricably linked to realizing the potential contributions of people from all backgrounds (EOP 2016).

In 1997, NSF established a specific broader impacts criterion as part proposal review (NSF 1997), which evaluates how the proposed work will advance scientific knowledge, and also contribute to the achievement of societally relevant outcomes. Such outcomes include, but are not limited to full participation of women, persons with disabilities, and underrepresented minorities in STEM; improved STEM education and educator development at any level; increased public scientific literacy and public engagement with science and technology; improved well-being of individuals in society; development of a diverse, globally competitive STEM workforce; increased partnerships between academia, industry, and others; improved national security; increased economic competitiveness of the United States; use of science and technology to inform public policy; and enhanced infrastructure for research and education (NSF 2008). The committee could not find any documented evidence of direct impact as a result of requiring this criterion, because NSF’s merit review process considers intellectual merit and broader impacts together when evaluating the impact of its federal investments. However, some indirect impacts have been captured through the outcomes of funded projects: economic impact via the development of new billion-dollar industries1 and societal impact via the establishment of a community of practice for developing sustainable and scalable institutional capacity and engagement in broader impacts activities.2

Other agencies have also implemented diversity and inclusion criteria as part of proposal evaluation. In 2017, the National Institute of General Medical Sciences (NIGMS) made diversity at all education and career levels a part of the scored review criteria (Gammie 2021). In funding its Cooperative Institutes (CIs), the National Oceanic and Atmospheric Administration (NOAA) has also established evaluation criteria that assesses proposal plans to recruit and train a more diverse and inclusive workforce, as well as to involve partnerships with other universities or research institutions, including Minority Serving Institutions (MSIs) and NOAA Educational Partnership

___________________

1 Efforts through the Computer Science and Telecommunications Board at the National Academies, which is funded by NSF, at https://www.nationalacademies.org/our-work/depicting-innovation-in-information-technology.

2 Efforts through the National Alliance for Broader Impacts (NABI) Center for Advancing Research Impact in Society (ARIS), at https://researchinsociety.org.

Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×

Program (EPP)/MSI Cooperative Science Centers, in ways that describe how the engaged partners will meaningfully contribute to the proposed activities of the CI (Koch 2021).

The committee acknowledges that NASA provides unique opportunities to PIs to run incredibly large budget projects (tens of millions of dollars to over $1 billion), and recognizes that no other federal agency offers comparable science and engineering (S&E) investments in their acquisitions for similar purposes. Nevertheless, NSF’s Materials Research Science and Engineering Centers (MRSECs) were identified in the study’s statement of task as the closest in nature to NASA’s PI-led missions in terms of the proposal review process. It should be noted that other NSF programs like the Physics Frontier Center and the Science and Technology Centers programs follow a similar format and at higher budget levels. Looking specifically at MRSECs for illustrative purposes, they comprise two to three topically distinct core research units that operate as interdisciplinary research groups (IRGs), each IRG employs interdisciplinary research to address unique, fundamental materials research problems that are intellectually challenging and important to society, and each are supported through 6-year awards ranging from ~$1.6 million to ~$4 million per year (Jones 2021). Proposal review for MRSECs is comprised of three stages: (1) pre-proposal (similar to step 1 of two-step mission proposals), (2) full proposal (similar to step 2 of mission proposals during which down selection occurs), and (3) site visits (similar to the final stage of selection for some mission proposals). Additionally, during the proposal review process MRSECs are partially evaluated based on their Diversity Strategic Plan, which must (1) articulate plans to broaden participation at all levels; (2) demonstrate a meaningful inclusion of participants from underrepresented groups: faculty, post-docs, graduate and undergraduate students, and staff; and (3) include metrics for assessment and desired, attainable outcomes. Moreover, implementation and efficacy of the Diversity Strategic Plan are addressed during site visits and at the time of annual reviews.

Emerging Practices for Proposal Evaluation

At the time that the work of this committee began, considering diversity on mission proposal teams was encouraged, but no specific evaluation criteria for diversity existed in the context of proposal review. In September 2021, NASA released a request for information (RFI) on “Adding Inclusion, Diversity, Equity, and Accessibility Requirements to NASA Announcements of Opportunity” (NASA 2021h). This RFI announced NASA’s desire to add explicit language about DEIA to the standard AO template, all future SMD AOs and the Stand-Alone Missions of Opportunity (SALMON) AO (see Annex 6.A for more details on this AO type), and amend a SALMON AO that was open at the time of the release of the RFI (NASA 2021g). As part of this new policy, proposals are required to

  • “Include a description of the processes used to assemble the proposed team and how these processes align with NASA’s IDEA values,” and
  • “Describe the processes that will be employed to enable and monitor (i) the creation and maintenance of an inclusive and equitable environment throughout the project lifecycle, (ii) the maintenance of a diverse team, (iii) and the continued access to equitable opportunities for contributions from team members toward mission success.”

More importantly, these DEIA plans would now be assessed as part of the evaluation of Scientific Implementation Merit and Feasibility of the Proposed Investigation (also known as Form B; see Chapter 2 and Annex 2.B, Table 2.B.2); however, no separate score or grade would be provided for this component.

It remains to be seen what outcomes such policies will have on the pool of PIs for competed space missions, but the committee acknowledges that this is an important step taken by NASA and commends the agency for this effort. Furthermore, if NASA is able to pivot successfully with the implementation of the new DEIA requirements for mission proposals, then the agency is positioned to be an exemplar for other organizations who utilize similar procurement processes, but have yet to adopt such policies. We provide additional considerations for utilizing diversity criterion in Chapter 7.

Finding: Recently, NASA has taken steps to require and evaluate DEIA plans as part of its AOs for competed missions, but compared to NIH and NSF, fewer approaches have been employed by the agency to identify and reduce any possible bias in the current mission proposal process.

Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×

DATA COLLECTION AND PROMOTING DEIA IN S&E FIELDS

Federal agencies like NASA play a significant role in supporting S&E in the United States, and in 2020, total federal research and development (R&D) funding was estimated to be to $155.6 billion (NSF 2021c). Therefore, such agencies have an obligation to be wise stewards of investing taxpayers’ dollars, and to be effectively transparent about any knowledge ascertained in S&E fields whenever possible, thus providing evidence of an appropriate return on the public’s investment. As mentioned in the previous section, some federal agencies have established systematic processes for tracking and assessing impacts on investments. In the case of NASA, there is evidence that within the Office of STEM Engagement (OSTEM) a comprehensive performance and assessment strategy is in place for monitoring and reporting accomplishments, particularly progress toward pre-established goals, and for collecting and analyzing data to assess how well a particular initiative is working and why. But as mentioned in Chapter 3 of this report, there is currently no adequate mechanism in place for collecting accurate demographic data on PIs and team members for competed missions, on grant proposers, as well as on the workforce across the four space mission-related science divisions of NASA SMD. Simultaneous to this study, another National Academies consensus study report committee was established and tasked with defining what data NASA needs to collect to assess the health and vitality of SMD’s research communities. It is this committee’s understanding that the Committee on Building a Foundation for Assessing the Health and Vitality of the Science Mission Directorate’s Research Communities will specifically identify best practices for data collection in keeping with its charge. For this reason, this report will only briefly highlight strategies that have been employed at the federal level to maximize the benefit to society resulting from research funding.

Evidence-Based Practices for Data Collection

NSF collects a variety of data about proposals, awards, proposers, and reviewers, and annually publishes a report, the Merit Review Digest,3 that contains summary statistics of some of these data. Table 6.1, for example, shows the kind of information collected from PIs of proposals in 2019 in order to assess diversity of participation in NSF’s research activities, and a full list of the comprehensive data included in the annual report is described in Annex 6.B. It is important to reiterate that while demographic information about proposers is based on self-reported data, and not all proposers choose to disclose this information, 71-78% of proposers for NSF funding elected to do so. Thus, this data collection strategy provides important information across virtually every dimension of the proposal review/award process, but more importantly, demonstrates transparency and visibility into investments for all stakeholders.

Promising Practices for Data Collection

Although PI-led missions as we know them today began in earnest in the 1990s, information on PIs and team composition began to be collected for the first time by NASA SMD more than a decade later. The contents of this database have been enhanced with the development and implementation of NASA’s Solicitation and Proposal Integrated Review and Evaluation System (NSPIRES), but the fact that the provision of demographic information is voluntary is frequently cited by the agency as a major barrier for collecting more comprehensive data on proposers. Other federal agencies (e.g., NIH and NSF) have also acknowledged this as a challenge, along with maintaining up to date demographic information on proposers as well as on the composition of the agency’s respective workforces. Additionally, these agencies have implemented strategies to work around these challenges, such as obtaining demographic information by way of leveraging data generated by other organizations. In the past, NSF for example, has worked with the Higher Education Research Institute at the University of California, Los Angeles, to obtain demographic data on students and faculty in STEM fields to use in its National Science and Engineering Indicators biennial reports. Similarly, NIH has leveraged data from both the National Center for Education Statistics (NCES) and the American Association of Medical Colleges (AAMC) to track the distribution

___________________

3 National Science Board, “NSF Merit Review Reports,” https://www.nsf.gov/nsb/publications/pubmeritreview.jsp.

Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×

TABLE 6.1 Participation in NSF’s Research Activities by Members of Different Racial and Ethnic Groups

Hispanic Non-Hispanic Unknown Total
American Indian or Native Alaskan 27 40 a b
Asian 20 7,533 590 8,143
Black/African American 23 626 21 670
Native Hawaiian or Pacific Islander a 29 a b
White 795 15,023 986 16,804
Multi-racial 54 308 21 383
Unknown 334 1,040 6,135 7,509
TOTAL 1,253 24,599 7,761 33,613

NOTES: This table reflects the number of research proposals submitted by various racial and ethnic groups in fiscal year (FY) 2019. Demographic information about principal investigators of research proposals acted on in FY2019 is based on self-reported data.

a Number less than 10.

b Row sum not available because a cell includes a number less than 10.

SOURCE: Adopted from NSF’s Merit Review Process: FY 2019 Digest.

Image
FIGURE 6.1 Snapshot of the biological/biomedical workforce in 2017-2018, disaggregated by (binary) gender, highest degree earned, and status of representation in STEM. SOURCE: Marie Bernard, National Institutes of Health (produced from data from NCES and AAMC).
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×

of scientists by gender, race and ethnicity through the pathway from undergraduate matriculation to the highest levels of academic medicine leadership (see Figure 6.1). In the physical sciences, we expect that such distributions will look different for women of all races and underrepresented men, and so this figure is simply meant to illustrate the data collection practices currently in place at NIH. Additionally, NIH tracks funding rates, as well as gender and career stage of all its PIs of Program Project Grant (P01) proposals across all institutes and centers.

Finding: NASA’s Solicitation and Proposal Integrated Review and Evaluation System (NSPIRES) has enhanced the quality of its database of information on proposers, including mission PIs. Yet, the agency does not currently have much of the needed infrastructure, including adequate staffing, standardized data collection practices, monitoring and analytic systems, annual reporting capability, and external partnerships that other agencies like NSF and NIH have leveraged to overcome the challenge of tracking participation in S&E research activities via demographic information acquired on a voluntary basis.

Conclusion 6-1: Both the lack of a transparent, systematic process for assessing proposal reviews and the inadequate infrastructure for collecting demographic data on proposers severely limit NASA SMD’s ability to accurately determine the relationship between the current mission proposal process and diversity in the pool of PIs. Furthermore, these insufficiencies result in limited ability to identify barriers in the mission proposal process and consequently the specific kinds of interventions that are needed to eliminate them and make progress toward the stated goal of effectively increasing DEIA in the leadership of competed missions.

ACCOUNTABILITY AND PROMOTING DEIA IN S&E FIELDS

The NSF merit review process described earlier is overseen by the National Science Board (NSB), which establishes the policies of NSF within the framework of applicable national policies set forth by the president and Congress. In this capacity, NSB identifies issues that are critical to NSF’s future, reviews NSF’s strategic budget directions and the annual budget submission to the Office of Management and Budget, and approves new major programs and awards. Other agencies including NASA, NIH, DOE, and DoD also have oversight mechanisms in place, including but not limited to a “Committee of Visitors” for reviewing the management of different federal programs. DOE, for example, utilizes evaluation and management to monitor and assess its workforce training programs, and as part of the evaluation, a Committee of Visitors is used to provide the assessments of federal program management and the status of the programs relative to comparable programs supported by other agencies. Similarly, DoD’s STEM portfolio activities and outcomes are guided by the DoD STEM Strategic Plan and overseen by an established STEM Advisory Council (SAC), but also managed and evaluated in accordance to DoDI 1025.11 (DoD 2020), a DoD-specific directive established in 2020. It is in the view of this committee that longstanding national advisory capabilities, comprised of nationally recognized experts in diverse disciplines, are essential for federal agencies to continue to help the nation leverage the talent of diverse communities to tackle today’s highly complex challenges in S&E.

Evidence-Based Practices for Accountability Through an Advisory Committee

In addition to NSB, the Committee on Equal Opportunities in Science and Engineering (CEOSE) is another congressionally mandated advisory committee that advises NSF on policies and programs concerning the implementation of the provisions of the Science and Engineering Equal Opportunities Act and other policies and activities to encourage full participation of women, minorities, and persons with disabilities in S&E fields. This committee holds meetings three times per year, and most recently gave focused attention to issues such as increasing accountability for broadening participation in S&E fields, addressing sexual harassment, promoting community engagement in science, understanding intersectionality, measuring broadening participation efforts, and promoting visibility and leadership in diversity, equity, and inclusion (DEI) initiatives and programs. Additionally, under the congressional mandate this committee is tasked with submitting a biennial report to the NSF Director and to Congress concerning:

Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
  • The state of participation of underrepresented groups in S&E fields;
  • An assessment of NSF’s policies and funding opportunities to broaden participation;
  • Summary of all CEOSE activities during the 2-year reporting period and future plans for the next 2 years; and
  • Recommendations to NSF for improving participation levels of underrepresented groups in S&E fields.

Most notably, recommendations from this committee have prompted (1) new and bolder agency-wide commitments to broadening participation, such as through the NSF Inclusion across the Nation of Communities of Learners of Underrepresented Discoverers in Engineering and Science (INCLUDES) initiative; (2) operationalized accountability for programs focused on broadening participation and for the agency itself; and (3) increased support for place-based implementation research projects that are grounded in and engage local communities, as well as increased attention to broadening the participation of diverse community members across its research and education portfolios through community-driven projects.4

Promising Practices for Accountability Through Independent Advisory Bodies

Since 1977, the NASA Advisory Council (NAC) has served as the independent advisory committee through which NASA obtains advice, guidance, and recommendations on major program and policy issues before the agency. Currently, the NAC includes five committees that advise on matters related to aeronautics, human exploration and operations, science, STEM engagement, and technology, innovation, engineering, as well as a regulatory and policy committee. Upon further examination of the mission and scope of the NAC, the committee was able to obtain charter information for every subcommittee, except the STEM Engagement Committee, which appears to be the most recent committee established as part of the NAC in 2015. In 2022, DEIA is one of the NASA priorities to which the NAC will align their work. It remains to be seen how DEIA will be prioritized by the NAC, and thus what role it will play in providing guidance to NASA as it relates to SMD realizing its stated goals of broadening participation in PI-led missions in space science.

Finding: The NAC provides expert guidance, advice, and recommendations to NASA concerning a number of program and policy issues in S&E fields. However, there is a lack of evidence to substantiate that this committee is being appropriately leveraged by NASA to promote meaningful change in the area of broadening participation as is carried out by NSF’s congressionally mandated CEOSE and NSB, NSF’s oversight board.

Conclusion 6-2: The lack of a NAC committee focused on DEIA, and a respective committee chair serving on the NAC itself, misses a unique opportunity to both help set the tone at the top of the Agency and ensure ongoing and critical focus on shaping NASA SMD’s broadening participation efforts in this area.

Alongside the efforts of standing advisory bodies, many federal agencies have enacted additional policies and procedures for promoting DEIA in S&E fields. In 2015, the Office of Science and Technology Policy (OSTP) and the Office of Personnel Management (OPM) established an interagency policy group, which is comprised of the Executive Office of the President and the relevant federal agencies. This working group was charged with cataloguing effective government-wide policies, practices, and strategies that have been used to mitigate the impact of implicit, institutional, and explicit bias in an effort to diversify their own STEM workforce (OSTP-OPM, 2016). Some identified policies and practices include evaluating recruitment, hiring, and promotion processes, conducting compliance reviews, and proactive use of DEI grants.

___________________

4 See the Committee on Equal Opportunities in Science and Engineering (CEOSE) biennial reports to Congress for 2011-2012, 2015-2016, and 2017-2018, available at https://www.nsf.gov/od/oia/activities/ceose.

Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×

Evidence-Based Practices for Accountability Through Evaluating Recruitment, Hiring, and Promotion Processes

Continual examination of recruitment, hiring, and promotion guidelines, as well as engagement strategies for current employees, is a common best practice used by federal agencies to promote DEIA in the workplace and foster equitable, safe, and respectful environments. NIH, for example, has done extensive work in this area. This includes, but is not limited to conducting periodic workplace climate surveys (NIH 2019a); analyzing recruitment, retention, and relocation incentives data to examine the racial and ethnic composition of employees receiving these incentives; reviewing the recipients of the NIH Directors Awards to determine whether racial and ethnic differences exist between groups in the nomination and selections rates for these honorary awards; and establishing expert committees to develop NIH-wide strategic planning for addressing any existing or potential barriers to DEIA in order to improve the culture of diversity and inclusion within the agency. Most recently, the agency has developed and published an Ending Structural Racism webpage5 that offers a wealth of information, resources, and educational materials on understanding and combating systemic racism and having tough conversations about race in the work environment. NIH also regularly collects workforce demographics data on employees and publishes data on hires, separations, promotions, and awards in its biennial workforce demographic reports. A virtual dashboard6 reflecting the latest demographic composition of the NIH total workforce has also been developed and these data are made available for all NIH employees, the entire biomedical research community, and the public.

Promising Practices for Accountability Through Evaluating Recruitment, Hiring, and Promotion Processes

NASA has embraced some strategies to promote DEIA within the agency’s internal workplace, which includes affirmative employment, conflict management training for employees, establishing an anti-harassment program, and a diversity and inclusion framework (NASA 2015b). As part of the diversity and inclusion framework approach, the agency has deployed comprehensive diversity and inclusion surveys to all employees for a more in-depth measurement of its diversity and inclusion efforts and to discover potential areas of bias or perceptions of exclusion that might exist in the workplace. NASA first deployed this survey in 2010 and repeated it in 2014. Since 2014, NASA has relied upon the New Inclusion Quotient of the annual Office of Personnel Management Federal Employee Viewpoint Survey to assess diversity and inclusion in the workplace.7

Evidence-Based Practices for Accountability Through Compliance Reviews

Federal agency compliance reviews are often used to identify and examine any forms of bias (i.e., implicit, institutional, or explicit), discrimination, and/or harassment at the time of review of grantees and to provide specific recommendations for institutional leadership to reduce their impact. Many organizations and institutions who are recipients of federal dollars are obligated to comply with several regulations prohibiting discrimination of any kind. Of the list of regulations, the most germane to the scope of this report are Title VI of the Civil Rights Act of 1964,8 which prohibits discrimination on the basis of race, color and national origin in federally assisted programs, and Title IX of the Education Amendments Act of 1972,9 which prohibits sex discrimination in federally assisted educational programs. Upon further examination of the operational role of compliance reviews in the context of federal funding, we found limited, available data associated with the Title VI of the Civil Rights Act of 1964; therefore, we focus our discussion on Title IX compliance reviews.

___________________

5 See the NIH “Ending Structural Racism” website at http://nih.gov/ending-structural-racism.

6 See the NIH “Workforce Profile Dashboard” website at https://www.edi.nih.gov/people/resources/advancing-racial-equity/nih-workforce-profile-fy21q02.

7 See the “Federal Employee Viewpoint Survey” at https://www.opm.gov/fevs.

8 Title VI, Civil Rights Act of 1964, 42 U.S.C. § 2000d et seq: https://www.justice.gov/crt/fcs/TitleVI.

9 Title IX of the Education Amendments of 1972, 20 U.S.C. § 1681 et seq: https://www.justice.gov/crt/title-ix-education-amendments-1972.

Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×

NIH is one of many federal agencies who have policies in place for addressing compliance with all applicable terms and conditions, policies, and requirements for institutions and individuals supported on federally funded awards, and in accordance with federal civil rights protections, which includes Title IX.10 Specifically at NIH, preaward assessments are conducted to verify that every applicant organization is eligible for funding on the basis of their status within multiple federal-wide systems used for management and oversight of federal funding recipients, and depending on specific circumstances, NIH reports instances of non-compliance into these systems. Post-award, recipient institutions are obligated to promptly report any administrative action taken, such as imposing leave or terminating employment, that affects the ability of senior/key personnel to continue on an NIH grant award, or otherwise constitutes a change in their status. A detailed description of NIH’s standard procedures for handling allegations of harassment involving award recipients is publicly available.11 Additionally, specific resources on preventing and addressing harassment and inappropriate conduct have been developed and are publicly available to the entire biomedical research community, including NIH employees. To document the impacts of these efforts, NIH regularly collects complaints-related data, and publishes these data as part of their annual Notification and Federal Employee Anti-discrimination and Retaliation (No FEAR) Act data reports submitted to Congress.

Promising Practices for Accountability Through Compliance Reviews

In the case of NSF, the Office of Equity and Civil Rights (OECR) is charged with conducting Title IX compliance reviews, and selects sites for review in coordination with other agencies to avoid duplicative efforts (NSF 2019). To date, 11 reviews have been conducted and while there is no established process in place at NSF to specifically determine the effectiveness of Title IX reviews, the committee learned that current efforts in the area of compliance do include designing metrics for measuring the impact of the new term and condition (T&C) (NSF 2021a) established to address sexual harassment, other forms of harassment, or sexual assault in the context of NSF-funded activities and within the agency.

The committee also heard testimony from a former Associate Administrator of the Office of Diversity and Equal Opportunity (ODEO) at NASA about the nature of its Title IX compliance reviews, and learned that the agency conducts two reviews per year. To date, more than 20 reviews have been conducted, and identified areas of concern are documenting during these reviews, and a final report outlining corrective actions and recommendations are subsequently provided to institutions by the ODEO. However, similar to the NSF, no established metrics or processes have been established to determine the effectiveness of this form of accountability to date.

Emerging Practices for Accountability Through DEI Grants

Proactive use of DEI grants afford programmatic support for developing and employing policies, practices, training materials, and recruitment and retention strategies designed to mitigate any bias within federally funded institutions of higher education. For example, beginning in 2017, the NSF Directorate for Computer and Information Science and Engineering (CISE) specifically enhanced support for PIs to learn about barriers to broadening participation in computing (BPC), with the ultimate goal of requiring PIs to include meaningful BPC plans in future proposal submissions. Similarly, in 2020 NIH put out a request for applications related to faculty institutional recruitment, as a mechanism for sustainable transformation at NIH-funded extramural institutions (RFA-RM-20-22). Specifically, this grant affords support for coordinating, collecting and analyzing data on institutional culture, diversity and inclusion, with the ultimate goal of recruiting and retaining a diverse group of early-career faculty who are competitive for an advertised research tenure-track or equivalent faculty position and who have demonstrated strong commitment to promoting diversity and inclusive excellence. The committee found no evidence of proactive use of DEI grants by NASA to date.

___________________

10 NIH Grant Policy Statement: https://grants.nih.gov/grants/policy/nihgps/HTML5/section_4/4.1.2_civil_rights_protections.htm#Educatio; NIH Actions and Oversight to Address Harassment: https://grants.nih.gov/grants/policy/harassment/actions-oversight.htm.

11 NIH Process for Handling Allegations of Harassment on an NIH-Funded Project at a Recipient Institution: https://grants.nih.gov/grants/policy/harassment/actions-oversight/allegation-process.htm.

Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×

Finding: With consideration for the suite of evidence-based, promising, and emerging government-wide practices that have been implemented, NASA has some analogous, albeit less developed, accountability mechanisms in place for promoting DEIA compared to other agencies like NSF and NIH. But, competed missions do not show robust evidence of positive outcomes across Divisions that advance these DEIA efforts. Additionally, there appear to be no established processes and metrics for evaluating the effectiveness of the current practices in place and for measuring progress toward stated goals.

Conclusion 6-3: The promise of NASA’s existing best practices to address DEIA, both within the agency and PI-led mission opportunities, is limited by the lack of defined metrics to evaluate effectiveness and track progress resulting from these interventions.

PROMOTING DEIA THROUGH INVESTMENTS IN STEM PATHWAYS

As first described in Chapter 4, individual pathways from education to occupation within S&E fields are often complex, and may include multiple entry and exit points along trajectories over the course an individual’s academic and working lives (Cannady et al. 2014). Interestingly, about 20% of the U.S. STEM workforce majored in a field other than STEM (Landivar 2013). Thus, in this report, we emphasize “STEM pathways” rather than a “STEM pipeline” to better reflect the non-linear nature of paths into STEM fields (NASEM 2016; NSTC 2021). It is in the view of this committee that building and sustaining education-to-career pathways that invite and prepare the next generation of diverse STEM professionals is also critical for the future of competed space missions. Therefore, NASA SMD has an important role to play in bolstering STEM pathways. In the final section of this chapter, we discuss best practices for advancing DEIA along STEM pathways based on our review of the research literature and a range of STEM-related programs, research, and activities in the federal landscape. It is important to note that significantly more initiatives have been implemented than can be covered in this report, and so the policies and programs highlighted below are representative examples of the respective best practice, but do not reflect an exhaustive list. Additionally, rather than prescribe specific interventions along STEM pathways that NASA SMD should develop and/or support, the committee felt that it would be more beneficial to outline key considerations for the Agency as it makes future investment decisions in light of its appropriate role in this area. As an organizing framework for this section, our discussion focuses on six evidence-based guiding principles for STEM pathway investments that emerged through the committee’s review: (1) promote the development of STEM identities, (2) establish flexible and relevant STEM education-to-career pathways, (3) intentionally recruit from historically underrepresented groups, (4) provide access to diverse mentorship, (5) foster career-life balance, and (6) promote systemic change around DEI.

Guiding Principle 1: Promote the Development of STEM Identities

Research on student persistence in STEM trajectories reveal that the development of a STEM identity is a leading important factor, especially for students from historically underrepresented groups (Calabrese et al. 2013; Vincent-Ruz and Schunn 2018; White et al. 2019). In fact, programs that afford opportunities for the development of student’s science efficacy, motivation, and identity are found to be the most successful at promoting persistence (Estrada et al. 2016). Additionally, STEM identities are shaped by opportunities to meaningfully engage in the practice of these disciplines and subsequently transfer this knowledge into action on issues of interest (Tan et al. 2013). Therefore, authentic and culturally relevant STEM engagement and research experiences are also critical to promoting the development of STEM identities. The development of a STEM identity is also positively associated with the likelihood of pursuing a STEM career (Stets et al. 2016; Cheng et al. 2018). To this end, a number of federal investments in STEM education have revolved around attracting a broad and diverse set of students and sparking interest in STEM, as well as providing opportunities for K-16 students to engage in valuable and authentic STEM research.

Both NASA’s OSTEM and the Science Activation Program (SciAct) within NASA SMD have deployed a number of initiatives to inspire and motivate learners through exposure to NASA’s various assets (e.g., state-of-the-art

Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×

facilities; cutting-edge technology, missions and associated data and images). Specifically, NASA’s Next Gen STEM efforts are mission-driven activities and have yielded a suite of evidence-based products. These facilitate opportunities for students to learn about new science discoveries and to acquire new skills in STEM. Additionally, the NASA Sounding Rocket RockOn! and RockSat-C Space Grant programs support experiences for middle school, high school and university students to design and build small science instruments for suborbital space flights in a 1-week workshop setting. Participants also gain experience in analyzing preliminary data from these flights, and can also take advantage of longer-term opportunities to deepen their skills in developing a scientific payload for flight throughout a school year. DoD offers a suite of activities for K-20 students, including but not limited to STEM camps whereby students connect with various STEM professionals around the country through STEM-focused lessons and talks, project-based competitions, student research opportunities, and paid apprenticeships and internships at DoD laboratories. Lastly, the annual Geostationary Operational Environmental Satellites (GOES-16/17) Science Fair, hosted by NOAA, affords opportunities for students in grade 6 through grades 13/14 (community college or university) to investigate weather and natural hazards using NOAA’s satellite data. Participants also submit a project individually or as part of a small team to the science fair describing their findings.

Guiding Principle 2: Establish Flexible and Relevant STEM Education-to-Career Pathways

Sustaining STEM identities once they have been developed is also critically important. Research shows that STEM interest and identity developed at one point in time could potentially be lost as individuals move toward careers due to a variety of factors (e.g., lack of mentorship, limited access to authentic research experiences, and mismatches between racial, ethnic, and gender identities and the perceived meanings of science and a scientist) (see Chapter 4 of this report) (Carlone 2004; Gazley et al. 2014; Remich et al. 2016). Supporting and cultivating STEM identities is also especially crucial for the retention of historically underrepresented groups in STEM. In the life sciences, self-efficacy in science research, greater intentions to pursue a STEM PhD, and higher scholarly productivity were all found to be associated with intentional efforts to sustain the scientific identities of students from groups underrepresented in STEM (Aikens et al. 2017). Opportunities that support gains in the ability to think and work like a scientist were also found to positively influence persistence in STEM, as well as serve as indicators of student growth as scientists and possible predictors of students’ pursuit of science research–related careers (Estrada et al. 2011; Aikens et al. 2017).

Over the last two decades, some government-funded interventions to broaden participation in STEM have evolved to focus on establishing flexible and relevant STEM pathways that connect education and workforce opportunities, and accommodate students’ diverse interests and backgrounds. For example, DOE supports tailored training opportunities specific to the fields and disciplines that are not adequately covered in university curriculum (e.g., high performance computing, accelerator science and technology, nuclear physics, nuclear chemistry, and isotope R&D). Additionally, the DOE Office of Science (SC) is starting a new initiative in FY2022, Reaching a New Energy Sciences Workforce (RENEW), that will leverage SC’s world-unique national laboratories, user facilities, and other research infrastructures to provide undergraduate and graduate training opportunities for students and academic institutions not currently well represented in the U.S. science and technology ecosystem. DoD’s Manufacturing Engineering Education Program expands opportunities for youth all the way up to adults already in the workforce to acquire deep skills in cutting-edge manufacturing techniques such as additive manufacturing and robotics, as well as shop floor skills like welding and machining. More broadly, DoD provides tuition support, stipends, and internships for undergraduate, master’s, and doctoral students through its SMART Scholarships, which culminate in guaranteed employment at the agency.

The NASA Community College Aerospace Scholars (NCAS) Program consists of a 5-week online course led by engineers at different NASA centers and a 4-day engineering design workshop at a NASA center. Through this program, community college STEM students are also encouraged to participate in other competitive NASA projects (e.g., robotics competition) and internships, and to pursue a 4-year degree or career in a STEM field. Overall, the program aims to help students make the connection between a STEM degree and career opportunities at NASA. NOAA’s José E. Serrano Educational Partnership Program with Minority Serving Institutions (EPP/MSI) Graduate Fellowship Program is a new program that provides professional development and 1 year of experiential training

Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×

at a NOAA facility to students currently pursuing a graduate degree in a NOAA mission-related field specifically at an EPP/MSI Cooperative Science Center. Furthermore, the long-term goal of this program is to build out a workforce pathway for future employment at NOAA.

Guiding Principle 3: Intentionally Recruit from Historically Underrepresented Groups

A critical driver to ensuring that the provision of flexible and relevant STEM pathways actually yield an inclusive STEM workforce is intentionality—a calculated and coordinated method of engagement used by institutions, agencies, organizations and the private sector to effectively meet the needs of a designated population (NASEM 2019a). As mentioned in Chapter 1 of this report, the inevitable changing demographics in the United States confer numerous opportunities for building a future, diverse STEM talent pool. Accordingly, a number of federal agencies have rolled out strategies to facilitate targeted recruitment of individuals from historically underrepresented groups at all levels including but not limited to actively engaging with MSIs that intentionally educate and serve the professional needs of these populations (NSTC 2021).

The Historically Black Colleges and Universities/Minority Institutions (HBCU/MI) Summer Research Program is a research experience for undergraduate and graduate students at DoD-supported HBCU/MI aimed at bridging classroom and real-world experiences in STEM disciplines, with a goal of increasing the number of individuals from historically underrepresented groups who choose careers in S&E fields. NSF has committed investments toward broadening participation in STEM, especially through its Inclusion across the Nation of Communities of Learners of Underrepresented Discoverers in Engineering and Science (NSF INCLUDES) initiative, which catalyzes the building of collaborative infrastructure for proactively seeking and effectively developing STEM talent at scale within populations that are historically underrepresented in STEM. NASA has recently joined the national network of partners for NSF INCLUDES and will specifically direct its efforts toward diversifying the engineering workforce,12 a field that is indispensable to NASA’s capacity for space exploration and discovery.

Within the past 5 years, NSF has also demonstrated more explicit commitments to building the capacity and research infrastructure at MSIs through the establishment of programs such as the Historically Black Colleges and Universities Excellence in Research (HBCU-EiR) program, the Tribal College & University (TCU) Enterprise Advancement Centers, and the Improving Undergraduate STEM Education: Hispanic-Serving Institutions (HSIs) program. NIGMS has implemented a suite of programs that collectively reflect targeted and intentional recruitment and development of diverse talent along a continuum of academic milestones (see Box 6.1).

Cultivating mutually beneficial partnerships and collaborations with MSIs is another facilitator of the recruitment and retention of individuals from historically underrepresented groups in STEM, and the NSF Partnerships for Research and Education and Materials (PREM) Program is one example of this approach. Specifically, PREM partnerships between MSIs and Division of Materials Research (DMR)-supported centers/facilities support recruitment, retention, degree attainment and pathways to careers for groups most underrepresented in materials science. Additionally, these partnerships are aimed at enhancing research productivity, programming, and infrastructure at partner institutions. Likewise, the NSF Louis Stokes Alliances for Minority Participation (LSAMP) program supports the recruitment and retention of students from underrepresented populations in STEM, primarily African Americans, Hispanics, American Indians, Native Hawaiians, and Pacific Islanders, through funding to alliances of institutions of higher education for the design, implementation, and study of STEM pathway-strengthening initiatives, such as bridge-to-doctorate programs and STEM Pathways projects. Similarly, NOAA’s Educational Partnership Program with Minority Serving Institutions has demonstrated increases in retention and achievement of supported scholars from historically underrepresented groups in STEM. There are also key facilitators of the success of these partnerships including collaborative planning and program design with partner institutions, having the partnerships led by and the impacts mostly realized by MSIs, and a commitment by NOAA leadership to support the program. The NASA HBCU/MSI Technology Infusion Road Tour facilitates opportunities for MSIs to collaborate with NASA and its partners (e.g., Boeing and Lockheed Martin) in the context of a multi-day forum

___________________

12 NASA and NSF INCLUDES Building MSI-Led Coalitions to Strengthen Broadening Participation in Engineering: https://www.nasa.gov/stem/murep/includes.html.

Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×

hosted on an MSI campus. Through this effort, MSIs are better positioned to acquire funding through more lucrative federal contracts, which in turn, enhances research capacity and competitiveness at these institutions (NASEM 2019a; NSTC 2021).

Guiding Principle 4: Provide Access to Diverse Mentorship

Developing and retaining promising personnel through appropriate mentorship—including sponsorship, peer mentorship, and tiered mentorship—is also essential for advancing DEIA in STEM (NASEM 2019a; NSTC 2021). Effective mentorship affords individuals from underrepresented groups the potential to see themselves through the eyes of an influential role model (NASEM 2019b), and is closely tied to their development of a sense of belonging in STEM contexts (Byars-Winston et al. 2015). Furthermore, mentorship has been established as a high-impact and effective practice for promoting long-term student success in STEM fields (e.g., increased likelihood to pursue doctorates in STEM fields) (Kuh 2008; Hubbard and Stage 2010; NASEM, 2019a). Alternatively, lack of mentorship (formal or informal) or an inadequate support system can exacerbate feelings of isolation and invisibility in individuals from underrepresented groups, and as a result, negatively impact their persistence and success in STEM (Graham et al. 2013; NASEM 2019b).

Given the impacts of mentoring in producing a more diverse workforce and innovative research outcomes, a number of federal initiatives that leverage the benefits of mentoring in order to diversify respective STEM workforces have been employed. The NIH Distinguished Scholars Program, for example, aims to build a more inclusive community of NIH PIs through mentorship and other professional development activities that foster research and career success to investigators from diverse backgrounds. Similarly, the NIH National Research Mentoring Network (NRMN) is a nationwide consortium designed to enhance mentoring and career development of individuals from diverse backgrounds, including those from groups underrepresented in the biomedical research workforce. The NSF Research and Mentoring for Postbaccalaureates in Biological Sciences (RaMP) supports full-time research, mentoring, and training for baccalaureate graduates who have had few or no research or training opportunities during college, as a way to remedy the loss of diverse talent at junctions between the attainment of a STEM undergraduate degree and entry into a STEM career pathway. Additionally, the NSF Research Experience and Mentoring (REM) program provides authentic research experiences in STEM fields and ongoing mentorship and professional development to high school and undergraduate students, K-12 teachers, faculty, and veterans.

Guiding Principle 5: Foster Career-Life Balance

As discussed earlier in this chapter, workplace climate can either be a facilitator or a significant barrier to realizing DEIA in STEM, as well as the well-being of all types of workers (Cech and O’Connor 2018; López et al. 2018; Cech and Blair-Loy 2019). Research shows that female employees are more impacted by work-life imbalance compared to male employees, but workplace flexibility bias—a sense that one will face career consequences for making work schedule adjustments for family or personal reasons—can negatively impact the well-being of all types of workers (Ong et al. 2011; Kachchaf et al. 2015; Cech and O’Connor 2018; López et al. 2018; Cech and Blair-Loy 2019).

Marriage and childbirth are key factors associated with the departure of women from the STEM workforce between PhD attainment and achieving tenure (NSF 2011). Consequently, some federal policies and practices have been implemented to minimize institutional barriers that can undermine performance and promotion due to parental status and other family responsibilities. For example, the NIH Research Supplements to Promote Reentry and Reintegration into Health-Related Research Careers support full- or part-time research by women or men returning to the scientific workforce after an interruption of at least 6 months for family responsibilities or other qualifying circumstances. The supplements also afford time for re-entering scientists to bring existing research skills and knowledge up-to-date, so that by the end of the supplement period, the scientist will be prepared to apply for other types of competitive independent research funding for continual support throughout their careers. NSF has also adopted similar policies and its Career-Life Balance Initiative was established to bolster the development of the STEM workforce through increasing the placement, advancement, and retention of women in STEM disciplines,

Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×

particularly women seeking tenure in academia. Relatedly, Career-Life Balance supplements specifically support the expansion of eligible NSF awards to accommodate additional project personnel to sustain research while a researcher (e.g., graduate student, postdoctoral fellow, early-career scientist) supported by NSF on the award is on a leave of absence for dependent care responsibilities or other direct family considerations.

Guiding Principle 6: Promote Systemic Change Around DEI

Wide-spread adoption of evidence-based practices is needed to achieve lasting progress on DEI in STEM fields, as well as systemic change within contexts where these practices are operationalized. To this end, some federal agencies have devoted resources toward preparing and empowering leaders and champions of diversity in order to transform the cultures of STEM fields. For example, in the geosciences, which has experienced little to no progress on increasing diversity in over 40 years (Bernard and Cooperdock 2018), targeted efforts have been put in place to build capacity for broadening participation. Specifically, as part of the NSF Geosciences Opportunity for Leadership in Diversity (GOLD) Program, geoscientists with interests in broadening participation receive professional development and training from experts in the areas of broadening participation, behavioral change, social psychology, institutional change management, leadership development and pedagogies in professional development, which effectively prepare them to be change agents and diversity leaders in the geosciences. The NSF ADVANCE: Organizational Change for Gender Equity in STEM Academic Professions supports systemic change and greater implementation of evidence-based strategies that facilitate equity and inclusion, while mitigating inequities in STEM academic professions. Additionally, realizing gender equity for STEM faculty in institutions of higher education is the preeminent program goal.

Similarly, the NIH Research on Interventions that Promote the Careers of Individuals in the Biomedical Research Enterprise Program strives to encourage the use of evidence-informed practices to enhance interest, motivation, persistence, and preparedness for careers in the biomedical research workforce. Moreover, addressing the impacts of structural racism, discrimination, and harassment on biomedical career progression are additional stated goals to change the culture of the biomedical research enterprise and ensure that future generations of biomedical researchers emerge from a diverse pool of talented individuals. Additionally, NIH’s Training Modules for Creating Safe, Inclusive, and Supportive Research Environments are specifically designed to catalyze cultural change in scientific, clinical, and research training environments, such that they are free from harassment, intimidation and discrimination, and thus are optimized for productive learning and research. These training modules are geared toward a wide range of scientific disciplines and career stages, reflecting the goals of broader dissemination and impact. Finally, NIH established the NIH Equity Committee (NEC) in 2017 as part of its efforts to facilitate institutional transformation and culture change at both NIH and in the extramural research community. This committee is comprised of internal NIH staff and systematically reviews and assesses annual metrics on (1) the pool of tenured and tenure-track investigators in the biomedical workforce as it relates to equity in salaries and hiring and promotion policies and procedures; (2) equity in review practices (Board of Scientific Counselors and ad hoc reviewers, promotion, and tenure committees); (3) efforts to redress inequities; and (4) efforts to promote DEI.

Indeed, federal agencies have a non-trivial role in supporting diverse and inclusive education-career STEM pathways as demonstrated by the number of federal investments targeted at multiple points along these pathways that have been employed. Figure 6.2 illustrates where along education-career pathways select federal agencies have made sustained, intentional investments at any funding level toward advancing DEIA in STEM fields. This snapshot also highlights where purposeful investments are still needed at the federal level more broadly, and more specifically where there are opportunities for NASA to bolster its investments in order to leverage the potential of PI-led missions in space sciences. As described in Chapter 4 of this report, the first major drop in the participation of women (aggregate) in physics occurs during the high school to college transition, the first major drop in the participation of underrepresented groups (aggregate) in physics and astronomy occurs during the post-secondary stage, and attrition within both of these subgroups continues into the workforce stages. Thus, investments targeting multiple junctures along STEM pathways that could synergistically improve recruitment and retention in the Earth and space sciences may be needed.

Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Image
FIGURE 6.2 Snapshot of select federal investments to advance DEIA along education-to-career pathways into STEM fields. For simplicity, each example is organized according to only one corresponding goal (far left), but it is important to note that more than one of these goals may be prioritized in a given intervention. The committee also acknowledges that education-to-career STEM pathways are not universally linear and so the respective placement of the different stages of education-to-career pathways (top) is meant to only facilitate the distribution of investments at the intersection of goals and target populations.
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×

Finding: Current NASA investments in STEM education and STEM-related training programs and activities in the United States appear to be concentrated during the K-graduate years of STEM pathways, with very little intentional interventions employed during the post-PhD stage.

Conclusion 6-4: NASA’s direct influence on the growth and development of a more diverse and inclusive Earth and space sciences workforce is potentially minimal due to limited investments in the post-PhD career stage. The lack of consistent and intentional investments post-PhD, may in part contribute to the underrepresentation of women of all races and men of color in the pool of leaders for competed space missions.

Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×

ANNEX 6.A

Details of SALMON and Program Element Appendix

SALMON are one solicitation mechanism used by NASA for announcements of opportunity (AOs). For example, the NASA AO, entitled “Third Stand Alone Missions of Opportunity Notice (SALMON-3),” provides a solicitation and procurement base for opportunities for modest investigations (missions of opportunities [MOs]) requiring spaceflight that advance the high priority science, exploration, and technology objectives of NASA’s Science Mission Directorate (SMD), Human Exploration and Operations Mission Directorate (HEOMD), and Space Technology Mission Directorate (STMD). The SALMON-3 AO does not, in and of itself, solicit proposals. The actual solicitation is enabled by a Program Element Appendix (PEA) that is appended to the SALMON-3 AO. The AO provides the standard requirements for all SALMON-3 solicitations and specific requirements that may only apply to particular types of MOs. The PEA will call out the SALMON-3 specific requirements that apply and any additional program requirements for the specific solicitation and proposal opportunity. Program specific requirements spelled out in the PEA include the scope of the solicitation, the available funding, the proposal due date, and other program specific requirements as well as deviations or exceptions from SALMON-3 standard requirements (e.g., they may be tailored for two-step evaluations). The mission proposal opportunities are not on a fixed schedule, but a planning document that extends out a number years is available, such as https://soma.larc.nasa.gov/StandardAO/doc_files/Planning%20List%20for%20SMD%20Solicitations%2020210308-for-publication.docx.

Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×

ANNEX 6.B

Data Collection and Annual Reporting on Federal Awards at NSF

The following list, from the table of contents for the National Science Foundation (NSF 2020), provides a comprehensive list of the data included in NSF’s Merit Review Process Annual Report:

  1. Proposals and Awards
    1. Data on Research Grants

      A1. Research Proposal, Award, and Funding Rate Trends

      A2. Research Grant Size and Duration

      A3. Diversity of Participation

      A4. Number of Investigators per Research Project

      A5. Number of Research Grants per PI

      A6. Number of People Supported on Research Grants

      A7. Average Number of Months of Budgeted Salary Support for Single-PI & Multi-PI Research Grants

      A8. Principal Investigator Funding Rates

      A9. Early and Later Career PIs

    2. Competitive Proposals, Awards, and Proposal Funding Rates
    3. Diversity of Participation
    4. Types of Awards
    5. Awards by Sector and Type of Institution
    6. Time to Decision (Proposal Dwell Time)
    7. Mechanisms to Encourage Transformative Research

      G1. Small Grants for Exploratory Research (SGER), EArly-concept Grants for Exploratory Research (EAGER) and Grants for Rapid Response Research (RAPID)

      G2. Research Advanced by Interdisciplinary Science and Engineering (RAISE)

  2. The NSF Merit Review Process
    1. Merit Review Criteria
    2. Description of the Merit Review Process
    3. Program Officer Recommendations
    4. Review Information for Proposers and the Reconsideration Process
    5. Methods of External Review
    6. Data on Reviewers
    7. Reviewer Proposal Ratings and the Impact of Budget Constraints
    8. Program Officer Characteristics
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 92
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 93
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 94
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 95
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 96
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 97
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 98
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 99
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 100
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 101
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 102
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 103
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 104
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 105
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 106
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 107
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 108
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 109
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 110
Suggested Citation:"6 Examining Best Practices at the Federal Level for Advancing Diversity, Equity, Inclusion, and Accessibility in Principal InvestigatorLed Missions." National Academies of Sciences, Engineering, and Medicine. 2022. Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions. Washington, DC: The National Academies Press. doi: 10.17226/26385.
×
Page 111
Next: 7 Paths Forward: Recommendations »
Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions Get This Book
×
Buy Paperback | $35.00 Buy Ebook | $28.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Fostering diverse and inclusive teams that are highly skilled, innovative, and productive is critical for maintaining U.S. leadership in space exploration. In recent years, NASA has taken steps to advance diversity, equity, inclusion, and accessibility (DEIA) in their workforce by releasing its equity action plan, emphasizing how diverse and inclusive teams help maximize scientific returns, and requiring DEIA plans as part of announcements of opportunities. To further its efforts to advance DEIA, the Agency requested the National Academies undertake a study to evaluate ways NASA can address the lack of diversity in space mission leadership.

Advancing Diversity, Equity, Inclusion, and Accessibility in the Leadership of Competed Space Missions outlines near and long-term actions NASA can take to make opportunities for leadership and involvement in competed space missions more accessible, inclusive, and equitable. Report recommendations range from changes to the mission proposal process to investments in STEM education and career pathways. This report makes 15 recommendations for advancing DEIA within NASA's Science Mission Directorate divisions that support competed space mission programs. However, many of the report's recommendations could also be applied broadly to research at NASA and other federal agencies and institutions, leading to a more diverse research workforce.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!