Summary
The U.S. space science community includes thousands of scientists across multiple disciplines that influence and are influenced by the many engineers, technicians, and support personnel that are part of the space research enterprise. More than one-third of NASA’s budget is devoted to space science, and the agency currently operates more than 50 space missions in the fields of astrophysics, Earth science, solar and space physics, planetary science, and the biological and physical sciences. The strength of NASA science lies with its people, both those who work directly for the agency and the thousands of researchers and professionals who are funded by NASA grants and contracts.
In 2020, NASA’s Science Mission Directorate (SMD) charged the National Academies of Sciences, Engineering, and Medicine to examine the foundation or base for healthy and vital research communities. The charge required (1) determining indicators of health and vitality and (2) identifying the data needed to assess them and define the data that NASA needs to collect to enable each decadal survey to analyze its community. Drawing on extant resources, the committee converged on six attributes of community health and vitality:
- Clarity of science for efficient research and public support, using clear scientific questions guiding research and funding;
- A representative workforce, demonstrating diversity (FSU 2017; Gomez and Bernet 2019), inclusion and equity and competence across a broad range of characteristics;
- Sustaining a diverse science enterprise through outreach, acceptance and development, which is ensured by ongoing outreach to the larger community at all stages and the acceptance and development of its members;
- Adequate funding, in providing support for the long cycle efforts required to maintain and grow a healthy research community;
- Resilience to emerging challenges, leveraging the ability to adapt to changes in the human, technological, and political environment; and
- Community standards of conduct, reinforcing recognized standards of conduct and equity in processes.
The committee concluded that to meet the larger outcome that SMD set forth, the research communities have to have clear scientific goals; a workforce illustrating diversity of backgrounds and skills; an environment open to the society writ large; funding for advancing the research agenda; capacity to adjust to change; and shared standards for comportment.
Each of the six attributes requires measures of their presence or absence, level of intensity, and sustainability. Thus, the second part of the charge focuses on the evidence, or the data associated with the indicators. Information to assess scientific clarity comes primarily from the decadal studies and other science reviews already in place by SMD. Examining the breadth and depth of diversity within the workforce requires data on the size and composition of the workforce along many axes. Measures of openness to the public at large, to agency employees, and to elected officials come from studies of the culture and climate. These studies also often provide data on adjustment to change and shared standards of conduct. Budget documents, compiled by NASA and the researchers receiving its support, provide information on funding.
Determining the health and vitality of research communities requires more than the collection of raw data, with each element treated separately. Needed are analyses that enable the building of models illuminating processes and trends. The committee determined that SMD and its research communities compile extensive information that relates in various ways to the selected indicators. But too frequently the data are not linked together, often because they do not emerge from collaborations. Sometimes the information is presented without context, leaving the reader unclear on how it might be used to advance community health and vitality.
As SMD contemplates the building of a data-centric foundation, more will be needed. Collaborations across institutions, units, and frameworks are likely to require a degree of management. The charge to the committee envisions each decadal survey conducting its own data-based analysis. But possibly the connections that are needed extend beyond the areas a decadal survey highlights. For example, when the Earth sciences community began to plan for its inaugural survey, it noted the constraints imposed by the dominance of conventional disciplinary boundaries. The organizers reflected on the need for a broad perspective where one did not prevail. Their conclusion: contemporary environmental challenges cannot be addressed exclusively by Earth scientists but demand the engagement of other disciplines—including the social, behavioral, and economic sciences—and experts on policy. As another example, the Diversity and Inclusion Task Force created by the American Astronomical Society incorporated three social scientists, chosen because of their knowledge about graduate admissions, retention, and data collection, to address topics central to the agenda of the task force. SMD might play a pivotal role in encouraging exchanges across specialties, both within the space and Earth sciences, and across other disciplines, and in so doing strengthen the quality of the data compiled to assess and sustain community health and vitality.
FOCUSING ON CLEAR, RELEVANT SCIENCE
Successful science is measured by its steady cadence of innovative, high-quality research results. The committee recognizes that accepting new opportunities, addressing changing environments, welcoming new ways to do business, while encouraging and embracing new generations of members to the science and research community will likely shift existing power structures, introduce new challenges and require adjusting paradigms. Data and visualization tools to track and measure intentional and unintentional consequences of this complex multi-objective problem will be necessary.
Recommendation: NASA’s Science Mission Directorate should collect key data and trends representative of the research solicitation process and quality of the research produced by the science community. Key indicators and/or efforts include the following:
- Metrics of participation (center, type of institution, specific institutions, partnership);
- Metrics of innovativeness and research pedigree (completeness of topics, novelty). The assessment of the quality of research and science results is likely best handled by periodic peer review;
- Review of peer review effectiveness and data sharing of other division-unique initiatives;
- A dashboard that tracks the multi-objective nature of driving science while incentivizing and supporting change; and
- Trends that capture the ratio of basic to applied research funding and foundational investment as defined in decadal recommendations. (Chapter 3)
ENABLING AND SUPPORTING A HEALTHY, REPRESENTATIVE WORKFORCE
This study recognizes that diversity is a driver of innovation, and that the science enterprise can only be at its most innovative when it maximizes and fully utilizes the broadest range of human talent (Hewlett et al. 2013). Data on demographics and other attributes are critical. In addition, as the nation’s demographic landscape shifts, with growing populations of racial and ethnic populations, sustaining our capacity to innovate and conduct research will require a strategy to increase the participation of underrepresented minorities in science (IOM 2011).
Recommendation: NASA’s Science Mission Directorate should implement the recommendations from the agency Science Workforce Study. Indicators to measure the success of the implementation could be the degree to which the nomenclature is adopted, feedback from participants in the training and rotational opportunities, and the number of best workforce practices that are shared across NASA centers. (Chapter 4, Recommendation 1)
Recommendation: NASA’s Science Mission Directorate (SMD) should use the demographic data collected through the Office of the Chief Scientist demographic survey to strengthen and evaluate progress in achieving the NASA SMD’s diversity, equity, and inclusion goals. Progress summaries should be presented to NASA’s constituencies and stakeholders on a regular cadence. The data and corresponding summaries should be recorded in NASA’s archives for historical research purposes. If the agency considers it important, NASA could fund further research into the historical trends in the data. Sample surveys of the key constituent populations beyond scientists should also be considered as part of assessing the health of the overall community. (Chapter 4, Recommendation 2)
Measuring and reporting qualities of inclusivity and equity to attract diverse community members requires going beyond demographics. To date, NASA has not established the benchmarks nor baselines by which to assess the health and vitality of its research community. SMD has placed significant focus on multiple programs that increase the diversity of its research community, like expansion of its double-blind (dual anonymous peer review) process. While this is a positive step forward, SMD has not identified an active feedback loop to assess progress, augment and adjust efforts as needed.
Recommendation: NASA’s Science Mission Directorate should develop a strategy to measure progress toward jointly agreed upon diversity, equity, inclusion, and accessibility (DEIA) goals. The following steps can be used to outline a plan to identify goals and mitigate challenges:
- Develop goals and objectives for DEIA—for example, increased minority serving institution (MSI)1 proposal submittals, increased nontraditional team members, increased nontraditional principal investigators, etc.;
- Identify relevant data that would inform the goal—for example, trend data on submissions from MSIs;
- Prepare cause/effect matrix to measure desirable changes and progress; and
- Measure progress and adjust as needed. (Chapter 4, Recommendation 3)
The 2020 study The Endless Frontier: The Next 75 Years in Science (NASEM 2020a) points to opportunities for science acceleration: advances in data science; newly developed analytical tools and numerical modeling to characterize, simulate and predict outcomes; artificial intelligence; interdisciplinary science; probabilistic methods; new science that relies on archival data; etc. Each will drive science and engage the community in discovery (NASEM 2020a). New platforms that encourage early innovation, particularly in student populations, like SmallSats and CubeSats have been useful in both teaching and encouraging collaboration. These complement the many large collaborative programs across all of the science divisions with programs like the Great Observatories, the
___________________
1 MSIs produce about 20 percent of underrepresented minorities science, technology, engineering, and mathematics degrees. See NASEM (2019a).
Geostationary Operational Environmental Satellites (GOES) and other Earth satellites, the “big data” missions, etc. that benefit from collaboration. Opportunities for interdisciplinary and collaborative science will accelerate research and discovery.
Recommendation: NASA’s Science Mission Directorate should continue support of interdisciplinary efforts and collaborative science by incentivizing implementation of programs similar to the SmallSats/CubeSats programs. (Chapter 4, Recommendation 4)
SPACE, EARTH, BIOLOGICAL, AND PHYSICAL SCIENCES EDUCATION AND OUTREACH
As a producer of compelling science, NASA is in a unique position to help students see a future science, technology, engineering, and mathematics (STEM) career pathway, and as such, is a contributor to the health of the STEM pipeline into the space science professions. Since NASA comprises multiple directorates (including the Office of STEM Engagement) that include education and public outreach (EPO) within their programming, the evidence-based strategies recommended by the National Academies report NASA’s Science Activation Program: Achievements and Opportunities (NASA 2020b), and other measures, can be used to ensure the consistency of data collected by NASA education and outreach programs. The results could benefit agency-wide efforts to support consistent EPO program evaluations.
Recommendation: NASA’s Science Mission Directorate (SMD) should propose to the Office of the Administrator pursuit of an agency-wide strategic approach to education and public outreach (EPO) that will include consistent definitions of success and impact and appropriate evaluation measures (e.g., for different audiences in formal education, informal EPO, as well as for the different areas of its overall science and research populations) for use across the SMD EPO portfolio as well as the agency as a whole to ensure consistent programmatic data collection. NASA SMD should work with social scientists, formal and informal educators, and science, technology, engineering, and mathematics (STEM) communication professionals to develop ways to track and evaluate short-, mid-, and long-term outcomes from their EPO efforts. The goal of this effort should be to aggregate data to provide evidence of impact in participants’ knowledge about the nature of science and engineering processes and practices, the role and value of science and engineering in society, and in changes in attitudes, behavior, and self-efficacy toward STEM careers. While important, the scope of a longitudinal effort compared to monitoring demographics for other efforts like workforce and career development for professional astronomers would be significant. Studies of mid- and long-term outcomes require following respondents for several years or more. Obtaining Institutional Review Board and parental approval to do so for participants who have not reached the age of majority at the time of the outreach could be problematic. NASA will have to decide agency priorities of diversity, equity, inclusion, and accessibility for these different audiences and devote resources to efforts accordingly. (Chapter 4, Recommendation 5)
ENGAGING SOCIAL AND BEHAVIORAL SCIENTISTS IN STRATEGICALLY SHAPING THE PEOPLE ENTERPRISE
This study recognizes that while diversity can be a driver of innovation, the science enterprise can be at its most innovative only when it finds the means to maximize and fully utilize the broadest range of human talent (Hewlett et al. 2013). As summarized in the National Research Council publication Enhancing the Effectiveness of Team Science (NRC 2015), the inclusion of individuals with diverse knowledge, perspectives, and research methods will only lead to the promised innovation and scientific breakthroughs when the issues of communication, and integration across the complexities of multiple domains can be resolved. If addressing the demography of the SMD community, the behaviors found within that community, and the operative political forces is required, analyses from sociologists, behavioral scientists, policy scientists are also likely needed. The risks of adjusting
culture, using rewards and messaging to shift personal perspectives, introduces challenges captured by Kunda in his ethnography of a high-tech company, “Engineering Culture: Control and Commitment in a High-Tech Corporation” (Kunda 1995). The concepts of considering culture as something to be engineered by readjusting norms requires the benefit of expert social and behavioral science engagement. NASA has provided instruction in many new space science areas, like training scientists on operations of new missions or observatories2 or how to use new data. Similar instruction on how to support teams could be beneficial.
Recommendation: NASA’s Science Mission Directorate (SMD) should engage or partner with social scientists to develop tools to assist proposing principal investigators in developing successful inclusion and management plans. These tools could include tools that NASA SMD uses to train scientists on space science capabilities—for example, workshops or master classes. Early partners from the social and behavioral sciences could come from the many institutions where there is relevant work on team cultures. Both quantitative and qualitative data should be considered as indicators of community health. (Chapter 4, Recommendation 6)
DECADAL STATE OF THE PROFESSION ASSESSMENTS
The discipline decadal surveys have given significant consideration to the current and future health of their communities and have made many valuable recommendations.
Recommendation: NASA’s Science Mission Directorate (SMD) should collect and review discipline decadal state of the profession recommendations at the SMD leadership level to help develop appropriate directorate-wide initiatives. (Chapter 4, Recommendation 7)
FUNDING THE SPACE AND EARTH SCIENCES RESEARCH COMMUNITY
The significant majority of the funding for space, Earth, biological and physical sciences research is soft funding, grants awarded through the annual call for basic and applied research, Research Opportunities in Space and Earth Sciences (ROSES). The grant-based research model, while agile, introduces concerns with burdens it places on the burgeoning research community (Benderly 2010; LeClere 2012; Smith 2007). Data associated with grants funding would be helpful in determining the impacts of grant oversubscription, churn, and institutional policy risk impacting the research community.
Strategic visualization tools like a dashboard to track science priorities over time to capture strategic critical skills, aging capabilities and facilities, vulnerable facilities, etc. will support science continuity across multiple domains in the face of unforeseen challenges. It can also show investments and progress against planned science missions to further strengthen SMD’s ability to exhibit a “constancy of purpose” in development and execution of its decadal missions (Sarikaya et al. 2019). In addition, visualization should capture the kind of science that seeks long term variation and change—for example, climate science, or that requires long term investment in human resources and collaboration—for example, interdisciplinary science, to ensure they are not disincentivized in favor of science that appears to produce more immediate returns. These assessments must also recognize that in the near term, minority scholars are less well funded than their counterparts (Bol et al. 2018).
___________________
2 An example of a workshop to train astronomers on how apply for observing time on the James Webb Space Telescope was funded in 2020 by the Space Telescope Science Institute (STScI), “JWST Workshop,” The Arecibo Observatory, https://www.naic.edu/ao/blog/jwst-workshop. Multiple electronic tools to assist astronomers are also available at WEBB Space Telescope, “Methods and Roadmaps—JWST User Documentation,” https://jwst-docs.stsci.edu/methods-and-roadmaps. The Nancy Grace Roman Space Telescope project held a virtual community workshop (November 2021) to share information on the Roman mission and its science (Science Team Workshop—Roman Space Telescope/NASA, https://roman.gsfc.nasa.gov/science/workshop112021/index.html).
Recommendation: NASA’s Science Mission Directorate (SMD) should track within each budget cycle the investments in projects inside NASA and projects providing funding outside NASA through the NASA Office of the Chief Financial Officer. The funding within NASA should be examined both by discipline and by location. The funding distributed beyond NASA should be studied by career stage of principal investigators (PIs), funding per institution, and breadth of research teams. Additional data to include reflect the status of the health of the grant-writing community. Key data to include are the ratio of directed to competed research dollars, numbers of grants per PI/co-investigator (Co-I)/Science PI, grant award success rates per PI/Co-I/Science PI, numbers of supported research contracts per PI/Co-I/Science PI and/or hours per week expected to support specific research contracts.
Because SMD’s budget is driven by the president’s budget request and congressional authorization and appropriations, NASA SMD should develop an integrated dashboard of annual priorities and budget allocations across all decadal science categories and SMD programs and projects to show at-a-glance investments and progress against science planned missions and the scientific results they deliver. This data visualization tool permits interested members of the science research community, the administration, and members of Congress to view the consistency (or inconsistency) of funds allocation and spending across the life-cycle of missions and can be used to address changes in directions over time and support the need for a consistent and stable budget for science missions. This also adds to the potential to improve greatly on adherence to cost and schedule planning. (Chapter 5, Recommendation 1)
DEVELOPING SYNERGIES WITH INTERNATIONAL AGENCIES, OTHER U.S. GOVERNMENT AGENCIES, AND DOMESTIC PARTNERS
Both SMD and NASA as an agency have included collaboration and synergies with other agencies, public-private partnerships with industry and open science as strategic goals. The opportunities include higher quality science results, informed, contributing new entrants, broader community outreach and global problem resolution. Visible data to capture progress on these fronts would inform progress and potential opportunities.
Recommendation: NASA’s Science Mission Directorate should define, collect, and report metrics of collaboration and partnership in each of the areas of international collaboration, inter-agency collaboration, commercial partnerships, and citizen science as indicators of a healthy and growing science community. (Chapter 5, Recommendation 2)
PROVIDING ADMINISTRATIVE OVERSIGHT OF A HEALTHY AND VITAL RESEARCH COMMUNITY
The role of leadership in sustaining any initiative should not be underestimated. Key strategies, include an articulated vision; leadership with the responsibility, authority and accountability and resources to accomplish necessary efforts; management of a data stream to understand status; and a regular cadence of review and criteria against which to evaluate outcomes. With many of these elements in place, the committee finds that SMD currently lacks an official with adequate staff and the responsibility to assess and shape the health of its research communities.
Recommendation: NASA’s Science Mission Directorate (SMD) should appoint a senior executive (perhaps with Senior Level rating) with oversight responsibility for the collection and analysis of data used to determine how well SMD is doing in creating a vibrant and healthy science community and the degree to which they are meeting diversity, equity, inclusion, and accessibility (DEIA) goals. Specific responsibilities of the person in this position will include the following:
- Ensure progress is being made across the directorate,
- Benchmark other agencies and recommend promising practices for consideration,
- Work in cooperation and collaboration with the other agency offices/departments (Chief Scientist, other directorates, Office of Human Capital Management, Small Business, etc.), and
- Apprise the SMD associate administrator (AA) and division directors on a regular basis of progress being made and barriers to accomplishment of established DEIA goals.
The executive should report to either the AA or deputy associate administrator for SMD and will negotiate with the AA SMD on appropriate staff support to facilitate the successful conduct of the duties of the position. (Chapter 5, Recommendation 3)
Recommendation: NASA’s Science Mission Directorate (SMD) should establish a collaborative working group among SMD, the Office of the Chief Scientist, the Office of the Chief Human Capital Management, and the Office of Diversity and Equal Opportunity to establish clear goals, collect and evaluate demographic results, and identify action required to mitigate deficiencies. A single senior executive official within SMD (with adequate supporting staff) should be assigned with oversight and coordinating responsibilities. Proposed specifics on the assigned senior executive official are captured in the previous recommendation. (Chapter 5, Recommendation 4)
Recommendation: NASA’s Science Mission Directorate should use diversity, equity, inclusion, and accessibility (DEIA) indicators from collected demographic data to identify characteristics associated with success for space and Earth science careers and research health and performance. NASA should also use the DEIA data to examine potential weaknesses in the processes and culture. (Chapter 6, Recommendation 1)
DATA AVAILABILITY
Although there are many sources of data available to NASA, there are opportunities to better identify and diversify its institutional pool by engaging social and behavioral scientists and similar experts in process development, strategic planning and tracking, cross-correlation of data from multiple sources, and encouraging scholarly use of some of the data.
Recommendation: NASA’s Science Mission Directorate should consult with external science and technology organizations to examine possible paths to acquire more complete information. In particular, the National Center for Science and Engineering Statistics at the National Science Foundation (NSF) would be a resource, given its experience in gathering information about science and engineering awardees. Moreover, NSF has a vested interest in the adequacy of the data, for NSF partners with NASA in supporting some of the decadal surveys. A standardized data collection instrument would allow data to be shared more easily across disciplines. (Chapter 6, Recommendation 2)
Recommendation: NASA’s Science Mission Directorate (SMD) should collect relevant culture and demographic data to obtain a comprehensive understanding of the longitudinal health and vitality of the relevant space and Earth science communities. This effort should include demographic aspects beyond race and gender. NASA SMD should work with social scientists to develop a plan to capture and examine these data regularly while minimizing the potential for survey fatigue. (Chapter 6, Recommendation 3)
Recommendation: NASA’s Science Mission Directorate (SMD) should define SMD’s space and Earth science research community to include not only the principal scientists, but also the network of professionals and skilled workers who enable the research, advocates who have public interest in the SMD mission, and potential members such as disciplinary-aligned students and researchers who could submit proposals. SMD’s goal should be to understand a broader membership in its community to ensure its overall health and vitality. Sampling of these populations should be considered in development of survey strategies. (Chapter 6, Recommendation 4)
Recommendation: NASA’s Science Mission Directorate (SMD) should collaborate on multiple fronts:
- SMD should collaborate with the Space Technology Mission Directorate, Office of the Chief Scientist, Office of STEM Engagement, Office of Chief Human Capital Management, and Office of Diversity, Equity, and Opportunity should work together to expand the questions on the Office of the Chief Scientist demographic survey and seek approval from the Office of Management and Budget for employment of their resultant proposed survey.
- SMD should work with the Office of Management and Budget, Office of Science and Technology Policy, Office of Personnel Management, NASA Office of Diversity and Equal Opportunity, NASA Office of Diversity and Inclusion, NASA Office of General Counsel, and other relevant government organizations, to identify and eliminate barriers to the collection of more relevant data. (Chapter 6, Recommendation 5)
PROMISING PRACTICES
Across the board, SMD and NASA as an agency, like its sister federal agencies, have developed multiple practices to address the health of their communities. Although practices against each attribute of health and vitality were assessed (Chapter 7), there were many promising practices associated with mentoring that merited attention. Mentoring was flagged by Dobbin as an effective tool to encourage inclusion, as so received special focus (Dobbin and Kalev 2016). Many studies have shown the value of mentoring to broaden the research community. Mentoring can not only prepare young scientists and engineers from all backgrounds for careers with NASA but also open other opportunities in the engineering and science communities globally.
Recommendation: NASA’s Science Mission Directorate (SMD) should develop a mentor-protégé program for minority serving institutions, including historically black colleges and universities, Hispanic-serving institutions, tribal colleges and universities, Asian American and Pacific Islander–serving institutions, Alaska Native and Native Hawaiian institutions, Native American–serving non-tribal institutions, predominantly Black institutions, etc., to help them train and develop principal investigators and researchers.
In addition, SMD should continue to work closely with outside professional societies—for example, the American Physical Society, the American Astronomical Society, the American Geophysical Union, etc., in development and expansion of mentoring programs. This will enable NASA to collect data and engage in longitudinal tracking of its research communities. (Chapter 7)
