3

Assuring Strong, Clear Science Priorities

FOCUSING ON CLEAR, RELEVANT SCIENCE

Successful science is measured by its steady cadence of innovative, high-quality research results. A robust research strategy centers on scientific discovery and relies on collaboration from both within and across key science areas. The road to discovery starts with problem solicitation, clarification, and prioritization, and proceeds through mission development, observation, data analysis, peer review, and dissemination of results. Each area has its own indicators of health. Successful execution also includes elements of project management, risk assessment, scheduling, and cost controls. The areas of project execution and management are outside of the scope of this study. However, cultivating skills in proposal preparation, partnering, resource identification, project management, risk forecasting and mitigation are all key capabilities that drive not only successful performance, but establish confidence in the science enterprise.

NASA has implemented processes that address every one of these elements, including additional enablers that accelerate discovery. In most areas, NASA is best in class; however, changing environments will reshape the future of what’s critical in science (NASEM 2020a). With this view of preparing for and enabling change, suggested processes include developing visualization tools that track and communicate multiple items: long-cycle science goals, the various separate initiatives being conducted by NASA’s Science Mission Directorate (SMD) science divisions, science progress (using metrics identified by the science community). This integrated, transparent view can encourage development of directorate-wide best practices. This chapter addresses these items and other opportunities for beneficial data collection in the categories of science problem solicitation; science assessment, clarification, and prioritization; and science acceleration.

Problem Solicitation

SMD uses multiple tools to prioritize research, from science teams to white papers soliciting input from the community. At its best, the process casts a wide net, capturing ideas and rationale from the broadest sets of potential researchers. Detailed reviews collect, and prioritize science objectives with input from both within NASA and the community, including partners from other federal and international agencies. White papers are requested on many subjects, particularly on “big ticket” items like observatory application and use. Using a metric of exciting and visionary ideas and requests for funding against available funds for research, SMD’s process is robust and encourages a healthy, forward leaning research community.

However, if diversity of thought is intended by a wide call, there are additional metrics that may be useful:

• Did the white papers or proposals consider an appropriate spectrum of science challenges? Are the key “corners of the research box” explored by the submissions?
• Are the proposals and responses representative of key populations? Multiple NASA centers? Geographic distribution of universities? Minority serving institutions (MSIs), minority institutions (MIs), R1, R2, partnerships? Industry partnerships? International collaboration?
• Are the concepts novel?

The importance of broad community engagement with input refreshed by valuing both the voices of experienced members of the community and more junior members ensures science that continues to push boundaries. These data would be helpful in assessing criteria for future calls, including whether more extensive implementation of pilot initiatives like NASA’s No Due Date (NoDD) proposals pilot program¹ in the Planetary Science Division (PSD), or incentives for engaging specific centers, technology or MSIs in future submittals would be beneficial. For example, the teaching responsibilities at smaller colleges and universities are greater than those of larger universities. With less time to prepare proposals, additional consideration for this challenge could be reflected in the proposal solicitation process—for example, sliding submission timelines based on school size, incentives for partnering, etc. The success metrics for the NoDD program are very appropriate for assessing effectiveness and next steps. Are proposers taking advantage of flexibility? (Metric: What is the decoherence time of proposal submission? [e.g., how are proposal submissions spread through the year?]) Do we have a reduced burden of proposals/reviews? (Metric: Does the overall rate [proposals per year] of submissions change?²) Are notification times timely (mean and maximum)? Are there any unanticipated negative impacts (or positive impacts) on underrepresented communities, demographics of proposers and proposing institutions?

One of the challenges in assessing the quality of research and science is that the classic metrics—for example, citations, h-index, impact, publications, altmetrics, etc., all have limitations. As acknowledged by the NoDD program, peer review of science performance against targeted goals will likely provide the best assessment of impacts on science/research integrity.

Research Assessment, Clarification, and Prioritization

With many more research opportunities and ideas than there is funding, assessment and prioritization is critical. The benefits of peer reviews are documented and well accepted (Ware 2008). In fact, the decadal surveys are the ultimate peer reviewed science planning, with multidisciplinary reviewers and a 10-year cadence. “The decadal provides an underpinning to all discussions about the priority of missions and the commitment of resources within the scientific community, NASA, the White House, and Congress. It is used as both ‘sword and shield’: a means to rally the community around new projects and investigations, and also to defend current priorities against budget cuts or other threats”³ (NRC 2007a). The significant levels of community engagement by members of the scientific community during decadal development defines high level priorities. More peer teams are involved in transitioning these priorities and recommendations into programs and evaluating their outcomes. A non-comprehensive list:

• Senior reviews are held every 2 years, to provide comparative reviews of operating missions within each science division to assess the scientific return from these missions.

___________________

¹NASA, 2022, “Research Programs with No (Fixed) Due Date (NoDD) or Rolling Submissions,” NASA Science Mission Directorate, https://science.nasa.gov/researchers/NoDD.

²NASA, 2021, “No-Due Date (NoDD) Programs,” Presentation at NASA Town Hall on January 21, NASA Science Mission Directorate, https://science.nasa.gov/science-pink/s3fs-public/atoms/files/Town_hall-1-21-21_v5.pdf.

³ J. Morse, Office of Science & Technology Policy (OSTP), NRC Workshop on Decadal Surveys, November 14-16, 2006.

• Science investigation teams provide the appropriate peer review and input in the identification of performance requirements related to the science areas, designing observational strategies, data analysis techniques, science simulations, calibrations, and dissemination.
• Formulation science working groups support science analysis and guide technical formulation. These processes have been effectively implemented on multiple programs, particularly programs with large investments in technology or observatories.
• Community briefings by science team members inform key stakeholders.
• Mechanisms to support ad hoc dialogue, particularly under current circumstances when the pandemic has minimized opportunities for ad hoc discussions, are under consideration, particularly for programs in formulation.
• Time allocation committees (TACs) segmented by discipline area—for example, planetary science, stellar physics, etc., evaluate astronomy and astrophysics General Observer (GO) programs for the best possible science.
• The NASA Advisory Council (NAC) Science Committee supports the advisory needs of the NASA Administrator, NASA SMD and other directorates in reviewing the space, Earth, biology, and physical science programs, facilities, and projects. Discipline science advisory committees advise the SMD divisions and provide advice and make recommendations to the division director on the relevant discipline programs, policies, plans, and priorities: Astrophysics Advisory Committee (APAC), Planetary Science Advisory Committee (PAC), Heliophysics Advisory Committee (HPAC), Earth Science Advisory Committee (ESAC), Biological and Physical Sciences Advisory Committee (BPAC), and Applied Sciences Advisory Committee (ASAC).⁴
• The National Academies of Sciences, Engineering, and Medicine’s Space Studies Board (SSB) provides an independent, authoritative forum for information and advice on all aspects of space science and applications, including oversight of the SSB Space Science Discipline committees who provide strategic direction and oversight to ad hoc committees and serve as an interface to the science community.⁵
• Science discipline analysis groups allow for community engagement in technical analyses like mission architectures.

Each of these processes has merit and is extremely effective; however, a review of the fiscal year 2022 SMD Congressional Justification Report, with supporting rationale by mission area indicates an uneven application of these tools across SMD divisions (NASA 2022b). Although there are many reviews and objectives, review committees, and review tools, data on their effectiveness may beneficial.

As a basic observation, the health and vitality of the SMD science community is measured by its continued pace of world-class missions and innovative scientific discoveries. The committee recognizes that introducing change, whether accepting new opportunities, addressing changing environments, and/or welcoming new ways to do business, while encouraging and embracing new generations of members to the science and research community, will likely shift existing power structures, introduce new challenges, and require adjusting paradigms. A tool to track and measure intentional and unintentional consequences of this complex multi-objective problem, capturing impacts on science, the existing community, and the anticipated community, would assist in measuring

___________________

⁴NASA, 2022, “Science Advisory Committees,” NASA Science Mission Directorate, https://science.nasa.gov/researchers/nac/science-advisorycommittees.

⁵ National Academies of Sciences, Engineering, and Medicine, 2022, “Space Studies Board,” https://www.nationalacademies.org/ssb/space-studies-board.

the broader goals of scientific success. These results should be compared to expectations and reported. Community involvement in the assessment of performance would provide valuable input.

ASSURING PRIORITY FOR ENABLERS OF SCIENTIFIC RESEARCH (FACILITIES, ACCESS, TECHNOLOGY DEVELOPMENT, DATA ANALYTICS, AND MODELING)

Science Acceleration

The 2020 National Academies’ study The Endless Frontier: The Next 75 Years in Science (NASEM 2020a) points to opportunities for science acceleration: advances in data science; newly developed analytical tools and numerical modeling to characterize, simulate, and predict outcomes; artificial intelligence; interdisciplinary science; probabilistic methods; new science that relies on archival data; etc. Each will drive science and engage the community in discovery. Under circumstances where physical processes are well known, as well as in emerging interdisciplinary fields, data analytics promise new opportunities. Of course, applications of numerical modeling must be governed to limit overuse, particularly for emerging science where there is no agreement on governing physical processes. A healthy research community should be availed of proven capabilities and encouraged to push for solutions (NASEM 2021; NRC 2007a, 2011a,b).⁶ Data on successful/unsuccessful applications of any of these emerging practices in individual science disciplines and by user communities would be helpful in tracking new opportunities for implementation in sister disciplines, buoyed by success in a related space science area.

In anticipation of growth in emerging practices and their role in scientific discovery and innovation, SMD has identified planned investments in all of its science divisions in order to drive adoption of novel technologies. As NAC’s Ad-Hoc Big Data Task Force observed, the fraction of science papers that rely on archived data is increasing and, in many cases, exceeds the percentage of papers based on new mission data. Resultantly, open and easy access by larger communities of users to publicly funded research and the underlying data will support continued innovation. Significant data storage (more than 100 petabytes of data by SMD’s estimates) will grow by four times that amount in 5 years as new missions are launched and new models are developed. Growth of NASA’s archives are anticipated to provide opportunities for unique discovery. SMD’s existing development plans appear to capture this opportunity.

The importance of foundational investments in facilities, technology, data analytics and modeling, and advancement in theory are highlighted in every decadal as a means to the most compelling future science challenges. New tools for scientists, including machine learning and artificial intelligence applications, require ongoing scanning of the research horizon, not only to maintain and support, but to drive discovery. The interdisciplinarity inherent in physics and astronomy, solar science, Earth science, biological and physical science means that the progress of a healthy science community relies on balanced progress across the subdisciplines. Novel ideas and the ability to explain data and scientific results all depend on this foundation.

Theory, simulations, and laboratory measurements in conjunction with observations support crucial understanding in every space, Earth, and biological and physical science discipline. Examples include modeling the processes that produce the universe which interact on a wide range of time and length scales. Simulations of dynamic events point to key findings about stars, and along with laboratory experiments and observations, we begin to understand planet formation. For planetary studies, science return and mission choices from among the many asteroids, comets, and other objects, require ongoing laboratory studies and theoretical work and laboratory simulations. Theoretical, laboratory, and terrestrial analog studies are required to interpret ancient Martian environments. From unraveling the mechanics of magma transport, to understanding contributors to climate change, observation systems in conjunction with theoretical models point to both solutions and often gaps in theory, knowledge, and

___________________

⁶ Note: Both the planetary and biological and physical sciences decadal reports were released in 2011.

research opportunities. Global observing systems, modeling systems, and theoretical and computational advances sharpen weather and climate science on a regular cadence (NASEM 2021; NRC 2007b, 2011a,b, 2013).

In each of the decadal surveys, the key metric for adequacy of support to theory, modeling, data analytics, and novel foundational elements that serve as the seed corn for new science was funding—in real year dollars (inflation-adjusted dollars) or as a ratio of overall budgets.

Although inclusion and equity across all dimensions are intended to improve with policies that examine and drive desired results, discussion specifically focused on assessing the challenges of accessibility was limited for this study given its evolving nature. The definition of accessibility, traditionally associated with physical access, is currently evolving. Evolving definitions include shifting to a broader definition that includes complete access, assuring that all obstacles, whether physical or psychological, do not serve as impediments to any individual’s potential achievement. The current administration, as stated in Executive Order 13985, “Advancing Racial Equity and Support for Underserved Communities Through the Federal Government,” has also acknowledged the need to consider a broader definition of accessibility to include access, particularly for communities and populations that have been underserved. With this broader definition of accessibility or access in mind, the study addressed the elements of inclusion and equity.