Origins, Worlds, and Life: A Decadal Strategy for Planetary Science and Astrobiology 2023-2032 (2022)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

C Technical Risk and Cost Evaluation of Priority Missions BACKGROUND The survey’s statement of task (see Preface and Appendix A) calls for “identifying, recommending, and ranking the highest priority research activities” in planetary science, astrobiology, and planetary defense. However, the identification and ranking of priority activities alone is insufficient. The statement of task mandates that consideration be given to the technical readiness, technical risk and likely cost of the major activities identified in the survey report. Concern about readiness and risk are not new. Alarm about the accuracy of the mission cost estimates used in past decadal studies were raised in a 2006 report by the National Academies. The latter noted that major missions in space and Earth science are being executed at costs well in excess of the costs estimated at the time when the missions were recommended……. in decadal surveys for their disciplines. Consequently, the orderly planning process that has served the space and Earth science communities well has been disrupted, and the balance among large, medium, and small missions has been difficult to maintain (NRC 2006). As a result, the 2006 report recommended that NASA should undertake independent, systematic, and comprehensive evaluations of the cost-to-complete of each of its space and Earth science missions that are under development, for the purpose of determining the adequacy of budget and schedule (NRC 2006, p. 33) The technical readiness of activities prioritized in decadal surveys and the associated cost estimates for candidate missions was discussed extensively during a lessons-learned workshop convened by the National Academies (NRC 2007a). Workshop participants commented that decadal surveys would benefit greatly if they conducted their own assessments of the technical risk and cost associated with priority missions, rather than to rely on NASA’s own estimates. Moreover, the adoption a uniform risk- and cost-estimating methodology would enable cross comparison between competing activities within a given survey (NRC 2007a, p. 21-30). The first occasion this advice was put into practice was when NASA included a call for an independent evaluation of cost and technology readiness in the statements of task for a review of the agency’s Beyond Einstein program (NRC 2007b). Soon thereafter, Congress recognized the benefit of such evaluations and mandated that the National Academies “include independent estimates of the life cycle costs and technical readiness of missions assessed in the decadal survey wherever possible” (Congress 2008). This requirement was first implemented during the 2010 decadal survey of astronomy and astrophysics via the use of the so- called cost and technical evaluation (CATE) process (NRC 2010). The CATE methodology was developed by the Aerospace Corporation and is particularly suitable to compare of costs and risks associated with a population of low-maturity mission concepts. Subsequently, the CATE methodology was successfully implemented by three additional decadal surveys, including Vision and Voyages (NRC 2011). A comprehensive review of the then most recent round of space-science decadal surveys was conducted during a workshop held in 2012 (NASEM 2013) and again in a 2015 consensus study (NASEM 2015). The CATE process was one of the topics examined in detail during both activities. One of the principal lessons learned about the CATE process, as identified in the 2015 report, was that “it is most useful as a reasonableness check on what is being recommended” (NASEM 2015, p. 52). In other words, how does the technical feasibility of one concept under consideration rank relative to its peers. Moreover, the 2015 report PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-1 

found that the details “used to support the CATE analysis are not necessarily indicative of how a mission will ultimately be implemented” (NASEM 2015, p. 52) Therefore, more emphasis needed to be placed on the technical readiness and risk aspects of the evaluation rather than the cost estimation. To reflect this important change in emphasis, the CATE process was revised somewhat and its name was changed to technical risk and cost evaluation (TRACE). The first decadal to implement the TRACE methodology was the 2020 astronomy and astrophysics survey (NASEM, 2021). THE CHALLENGE OF TECHNICAL RISK, COST AND SCHEDULE EVALAUTIONS The concepts assessed by a decadal survey are typically in preliminary stages of development: i.e., pre- Phase A or Phase A concepts. However, the cost of a mission is typically not well understood until it has gone through its preliminary design review (PDR). Even after PDR, unanticipated increases in mass, cost, and schedule can occur. Another challenge to accurate evaluation is the fact that not all pre-Phase A concepts are equal. Some may be more mature than others because more resources have been available during their formulation. Accordingly, ensuring that a mission evaluation is fair and equitable requires that the relative maturity of concepts be considered. Several different varieties of technical risk/cost/schedule/ evaluations are used when discussing missions concepts. The best known are the so-called ICE (independent cost estimates) and NASA’s TMC (technical, management, and cost). A third is the TRACE process adopted by the most recent round of decadal surveys. Each has its own strengths and weaknesses (Table C-1) TABLE C.1 Similarities and differences between three different approaches to assessing the technical, cost and risk characteristics of spacecraft missions TMC ICE TRACE Used consistently to compare several concepts Yes No Yes Concept cost is evaluated with respect to Cost Cap Project Budget NASA Budget Maturity of concept Phase A-B Phase B-D Pre-Phase A Evaluation Process Includes: Quantified schedule growth cost threat No Typically Yes Quantified design growth cost threat No No Yes Cost threat for increase in launch vehicle capability No No Yes Independent estimates for non-U.S. contributions No No Yes Reconciliation performed with project team No Yes No Technical and cost risk rating (low, medium, high) Yes No Yes The objective of the TRACE process is to perform a technical risk and cost evaluation for a set of concepts that may have a broad range of maturity, and to assure that the evaluations are consistent, fair, and informed by historical data. Typically, concepts evaluated via a TRACE are early in their lifecycle, and therefore are likely to undergo significant subsequent design changes. Historically, such changes have resulted in cost growth. ICEs, on the other hand, are usually done later in the lifecycle of a project after it has matured: i.e., typically at NASA’s key decision points KDP-B and KDP-C and design reviews such as PDR and the Critical Design Review (CDR). ICEs often do not consider certain aspects of cost growth associated with design evolution in the earliest phases of a project. Therefore, a robust process is required that fairly treats a concept of low maturity relative to one that has undergone several iterations and review. TRACE evaluations take into account several components of risk assessment (Table C-1). However, an essential prerequisite for a TRACE evaluation is a specific mission design specified in sufficient detail that its has a concept maturity level (CML) of four or five. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-2 

FORMULATION AND IDENTIFICATION OF MISSIONS FOR TRACE NASA prepared for the planetary science and astrobiology decadal survey via a twofold approach. First, the agency commissioned science definition teams (SDT) and related groups to study various mission concepts. 1 The best known of the latter was that for the Europa Lander (Hand et al. 2017). Second, NASA worked with the planetary science and astrobiology communities via a competitive process to identify, fund, study, and create the appropriate documentation for 11 pre-decadal planetary mission concept studies (PMCS). The survey’s six panels reviewed the SDT and PMCS mission study reports and assessed those concepts proposed by the community in white papers and prior proposals (e.g., V&V mission study reports). Then the panels identified 18 additional large- and medium-class mission concepts (including a pair jointly developed by two panels and one proposed by a cross-panel group) that could address key scientific questions within their respective purviews. Following documentation by their originating groups, the steering group assessed each one and selected nine for additional study. To ensure that all mission concepts are mature enough for subsequent evaluation by the TRACE team, the nine concepts were sent for detailed technical studies; three each at, the Jet Propulsion Laboratory, the Goddard Space Flight Center, and the Applied Physics Laboratory. One or more “science champions” drawn from the ranks of the panels was attached to the mission design teams at each of the centers to ensure that the concepts remained true to the scientific and measurement objectives of their originating panel. 2 A tenth study was later initiated by JPL to create a series of low fidelity (CML-2) concepts addressing planetary defense objectives (see Chapter 18 for details). None of the resulting small-class mission concepts were submitted to the TRACE team because the survey’s statement of task did not call for the prioritization of such likely low-cost missions. 3 Once the nine additional studies were completed, the scientific and technical feasibility of the resulting concepts was assessed by their originating panels. Also assessed by the panels were those PMCS and SDT concepts of relevance, and two additional concepts that had been contenders in the competition for the fourth New Frontiers opportunity. 4 Following deliberations and prioritization by their respective panel, the most promising mission concepts were forwarded to the steering group for additional discussion and an independent, multistep ranking process. The outcome of that process was that the 17 most promising concepts (one with two varients) were identified and submitted to the TRACE team for evaluation. In summary, from an initial group of 33 concepts, the survey’s panels and steering group identified 17 missions, each of which had been studied to a sufficient fidelity to undergo the TRACE process. A full list of all of the missions considered by the decadal survey, their origins and ultimate disposition can be found in Table C.2 5 1 It is important to note that not all such activities resulted in concepts of sufficient technical maturity—e.g., NEX- SAG (Campbell et al. 2015), Ice Giant (Hofstadter et al. 2017), and ICE-SAG (Diniega et al. 2019)—to be submitted for TRACE analysis without additional work. As such, the work of these groups was used as input to the survey’s mission formulation deliberations. 2 The full mission study reports for the nine concepts identified by the decadal survey are available at https://tinyurl.com/2p88fx4f 3 The complete final report of the planetary defense rapid mission architecture study is available at https://tinyurl.com/2p88fx4f 4 Required to ensure that representative concepts responsive to the New Frontiers 4 and 5 Ocean Worlds mission theme undergo technical risk and cost evaluation and thus remain potentially viable candidates for future New Frontiers’ opportunities. Full study reports for these two missions are available at https://tinyurl.com/2p88fx4f. 5 A notable omission from Table C.2 is the Venus In Situ Explorer. This concept did not undergo TRACE analysis because the survey committee concurred with V&V’s decision that the phase-A study—conducted when a concept responsive to VISE was in the step-two competition for the third New Frontiers launch opportunity—was equivalent to or better than a CATE. “The committee assumes that the ongoing NASA evaluation ….. has validated [its] ability to be performed at a cost appropriate for New Frontiers” (NRC 2011, p. 15). PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-3 

TABLE C.2 Mission Concepts Considered by the Decadal Survey for Technical Risk and Cost Evaluation Name Origin Disposition Additional Information Mercury Lander PMCS Selected for TRACE Ernst et al 2020 and Appendix C Venus Flagship PMCS Selected for TRACE Gilmore et al 2020 and Appendix C Venera D SDT Not selected for TRACE Venera-D 2019 and Appendix D VISAN: Venus in situ seismic and PV Not selected for study or Appendix E atmospheric network TRACE VSCA: Venus sub-cloud aerobot PV Not selected for study or Appendix E TRACE VLP: Venus life potential PV Not selected for study or Appendix E TRACE VIDEO: Venus investigation of PV Not selected for study or Appendix E dynamics from an equatorial orbit TRACE ADVENTS: assessment and PV Selected for study. Not O’Rourke et al. 2021 and discovery of Venus’ past selected for TRACE Appendix D evolution and near-term climatic and geophysical state Lunar Geophysical Network PMCS and V&V Prioritized following Neal et al. 2020 and CATE by V&V, no further Appendix C of V&V action needed Intrepid: lunar long-range rover PMCS Selected for TRACE Robinson et al. 2020 and traverse Appendix C Endurance: South Pole Aitken PMM Selected for study and Keane et al. 2021 and Basin sample collecting rover two variants selected for Appendix C TRACE INSPIRE: in situ solar system PMM Selected for Study and Heldmann et al. 2021 and polar ice roving explorer selected for TRACE Appendix C MOSAIC: Mars orbiter for PMCS Not selected for TRACE Lillis et al, 2020 and surface-atmospheric-ionospheric Appendix D connections MOIRE: Mars orbiter for ices, PMCS Not selected for TRACE Calvin et al. 2020 and resources and environments Appendix D Mars Life Explorer PM Selected for study and Williams et al. 2021 and selected for TRACE Appendix C Mars In Situ Geochronology PMCS Selected for TRACE Cohen et al. 2020, additional detail from originating team, and Appendix C Mars Deep Time Rover PM Not selected for study. Appendix E Not selected for TRACE Mars Polar Ice, Climate and PM Not selected for study. Appendix E Organics Not selected for TRACE Ceres Sample Return PMCS Selected for TRACE Castillo-Rogez et al. 2020 and Appendix C Cryogenic Comet Nucleus Sample PSSSB Selected for study. Not Stroud et al. 2021 and Return selected for TRACE Appendix D Europa Lander SDT Selected for TRACE Hand et al. 2017, additional detail from originating team, and Appendix C Enceladus Orbilander PMCS Selected for TRACE MacKenzie et al. 2020, additional details from PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-4 

originating team, and Appendix C Enceladus Multiple Flyby Prior study Selected for TRACE Davila et al. 2021 and documentation Appendix C made available to survey Titan Orbiter and Probe Prior study Version without the sea Hayes et al. 2021 and documentation probe elected for TRACE Appendix C made available to survey Saturn Ring Skimmer PGPS Not selected for study or Appendix E TRACE Centaur Orbiter and Lander PSSSB Selected for study and Telus et al. 2021, and selected for TRACE Appendix C Uranus Orbiter and Probe PGPS and Selected for study and Simon et al. 2021 and POWDP selected for TRACE Appendix C Calypso: Uranus Moon and KBO PSSSB and Selected for study and Martin et al. 2021 and Flyby POWDP selected for TRACE Appendix C Odyssey: Neptune Orbiter and PMCS Selected for TRACE Rymer et al. 2020, Probe additional details from originating team, and Appendix C Triton Ocean Worlds Surveyor POWDP Selected for study and Study report and selected for TRACE Appendix C Persephone: Pluto System Orbiter PMCS Not selected for TRACE Howett et al. 2020 and and Kuiper Belt Explorer Appendix D Interstellar Object Rapid PSSSB Not selected for study or Appendix E Response Mission for TRACE Solar System Space Telescope Cross panel group Not selected for study or Appendix E for TRACE NOTES: Green, yellow and brown shading, respectively, indicates that a concept studied and selected for TRACE, studied but not selected for TRACE; and suggested but not studied.PGPS, panel on giant planet systems, PM, panel on Mars, PMM, panel on Mercury and Moon; POWDP, panel on ocean worlds and dwarf planets, PSSSB, panel on small solar system bodies, PV, panel on Venus; PMCS, pre-decadal mission concept study; SDT, science definition teams, V&V, Vision and Voyages decadal survey; OVERVIEW OF THE TRACE PROCESS The National Academies engaged the services of the Aerospace Corporation to perform independent TRACE evaluations of mission concepts identified by the committee’s steering group during this survey. Aerospace’s TRACE team consists of experts in the evaluation of technical, cost and schedule risks The TRACE began when the survey committee forwarded the full mission study reports for the 17 concepts selected for detailed examination to the Aerospace Corporation team. If the supplied documentation was insufficient or more details were required, the TRACE team requested additional information from the decadal survey. In three cases, the decadal survey was able to work with the relevant originating team to obtain the information required to complete the TRACE. The members of the TRACE team worked interactively to determine an initial assessment of technical risk and cost and schedule estimates for each of 17 missions analyzed. The TRACE team was diligent and worked, to the extent feasible, to treat all 17 concepts in a consistent and evenhanded manner. Following an initial internal review within Aerospace to assure that the 17 assessments were mutually consistent, the results were presented to the survey’s steering group. The latter provided feedback to the PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-5 

TRACE team who, in turn, incorporated this feedback into revised technical, cost and schedule risk evaluations. FIGURE C.1 Schematic illustration of the flow of the Aerospace Corporation’s technical risk and cost evaluation (TRACE) process. The blocks in purple indicate interaction by the TRACE team with the committee. The TRACE team’s approach (Figure C.1) is based on the following principles: • Use multiple methods and databases containing the pertinent details of past space systems so that no one model or dataset biases the results. Specifically, the TRACE team used proprietary Aerospace models (e.g., Small Satellite Cost Model, Missions Operations Cost Estimating Tool) and space-industry standards (e.g., NASA’s Project Cost Estimating Capability). • Use analogy-based estimating; tie costs and schedule estimates to NASA systems that have already been built with known cost and schedule. • Use both system-level estimates as well as a build-up-to-system level by appropriately summing subsystem data so as not to underestimate system cost and complexity. • Use verification tools, to check cost and schedule estimates for internal consistency and risk assessment. • In an integrated fashion, quantify the total threats to costs from schedule growth, the costs of maturing technology, additional scope, and the threat to costs owing to mass growth resulting in the need for a larger, more costly launch vehicle. In summary, an analogy-based methodology estimates the costs of future systems by comparison to the known cost of systems that have previously been built. The TRACE process provides an independent estimate of cost and complexity of Pre-Phase A, proof of mission concepts anchored with respect to previously built hardware informed by the NASA historical record of concept design growth. Thus, the TRACE process is designed to inform the decadal survey about the potential future impact of the concepts PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-6 

evaluated to best enable the prioritization of a balanced program within a defined future budget scenarios. The use of multiple methods—e.g., analogies and standard cost models—ensures that no one model or database biases the estimate. The use of system-level estimates and arriving at total estimated costs by statistically summing the costs of all individual work breakdown structure (WBS) elements ensures that elements are not omitted and that the system-level complexity is properly represented in the cost estimate. The evaluation of technology, cost, and schedule are inextricably intertwined. However, it is easier to describe each element of the overall analysis (e.g., technical, schedule, and cost) separately noting in each instance the linkages to the overall TRACE evaluation. Technical Evaluation The evaluation of technology readiness, risk, and maturity in the TRACE process focuses on identifying the most important technical threats to achieving the necessary mission performance and stated scientific goals. The assessment is limited to a consideration of the top-level technical maturity and risk. Deviations from the current state of the art, system and operational complexity, and integration concerns associated with the use of heritage components are identified. Technical maturity and the need for specific additional development are evaluated by the TRACE team by assessing the readiness levels of key technologies and hardware. Technical risk assessment also included available resource margins for the reference design, resilience of the program architecture, testing challenges, and operational requirements. During the assessment of technology risks and concept maturity, the technical, cost, and schedule teams interact so that technological threats can be translated into schedule and cost risks. The technical evaluation phase of the TRACE process is limited to the identification of high level technical risks that could potentially impact schedule and cost. The TRACE process places no cost cap on mission concepts and hence risk as a function of cost is not considered. Concept maturity and technical risk are evaluated by considering the ability of a concept to meet a specified performance with adequate mass, power and performance margins, given the proposed launch date. TRACE evaluations also assess proposed mass and power contingencies with respect to technical maturity using AIAA guidelines to achieve a consistent evaluation. If the TRACE technical team concludes that the proposed contingencies are insufficient, they are increased in accordance with historical data on mass and power growth as summarized in the AIAA guidelines. In some cases, growth in mass and power requirements necessitate the selection of a larger launch vehicles to execute the proposed mission. In addition, the need for a more capable launch vehicle—e.g., to accommodate potential growth in mass and power requirements—are passed on to the TRACE cost and schedule teams for incorporation into their estimates. Schedule Evaluation To aid in the assessment of concept risk, independent schedule estimates are incorporated as part of the TRACE cost estimate. This is especially true for assessment of risk with respect to proposed mission development and execution timelines. Like the TRACE assessment of cost risk, schedule risk is also derived from analogies in the historical NASA record. Historical data from past analogous NASA missions, properly adjusted, are used to gauge the realism of the proposed durations of the development phases. Similarly, the time to critical mission reviews (e.g., PDR and CDR) and the time required for integration and testing are evaluated for each concept and contrasted with appropriate historical experience. A statistical approach is used to create a schedule probability “S-curve”—i.e., a curve of the probability that the development time will exceed some specific value as a function of that value. The overall schedule, as proposed, is then adjusted with the historical data in mind. If the proposed schedule for a particular mission cannot be met, the next available launch window is selected. Additional costs incurred because the proposed schedule cannot be met are then added to the total cost of the mission. The committee requested PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-7 

that the TRACE team use the 70th percentile value in its schedule estimate—i.e., there is a 70 percent probability that the schedule will be shorter than indicated and a 30 percent probability that it will be longer. Cost Evaluation The primary goal of the TRACE cost evaluation is to provide independent estimates (in FY 2025 dollars) that can be used to prioritize various concepts within the context of the expected NASA budgetary constraints for the coming decade (see Chapter 22). The TRACE team developed high-level cost estimates based on the information provided by the various mission study teams with a focus on treating all projects equally. To be consistent for all concepts, the TRACE cost process allows an increase in cost resulting from increased contingency mass and power, increased schedule, increased required launch vehicle capability, and other cost threats depending on the concept maturity and specific risk assessment of a particular concept. All cost assessments for the TRACE process are probabilistic in nature and are based on the NASA historical record and documented project lifecycle growth studies. Traditional S-curves of cost probability versus cost are provided for each concept with both the project estimate and the TRACE estimate at the 70th percentile indicated. The focus of the TRACE costing process is to estimate the cost of conceptual hardware—e.g., instruments, spacecraft bus, landers—using multiple analogies and cost models based on historical data (see Figure C.2). A probabilistic cost-risk analysis is employed to estimate appropriate cost reserves. Ensuring consistency across the range of concepts—from those that are immature to those that are significantly more mature—the cost estimates are updated and adjusted with information from the technical TRACE team with respect to mass and power contingencies, technical maturity and development risk, and potentially required additional launch vehicle capability. Using independent schedule estimates, costs are adjusted using appropriate “burn rates” (cost per month) to properly reflect the impact of schedule changes. Finally, the results are integrated, cross-checked and verified for consistency before being presented to the survey committee. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-8 

FIGURE C.2 Schematic illustration of the TRACE cost process. The top row represents the basic elements of the cost-evaluation process. Typical of all cost estimates, the TRACE evaluation begins with the hardware costs as defined in WBS 5 (payload), WBS 6 (flight systems) and WBS 10 (integration and test) using multiple analogies and different cost models. Typical cost wraps or percentages of the hardware (WBS 1, 2, 3, 4, 7 & 9) are based on the historical record. Operations cost is estimated with a combination of parametric models and the monthly operations cost of historical analogies. Then each of the WBS cost elements are probabilistically evaluated to produce cost reserves in the form of a typical S-curve. The 70th percentile represents the initial cost estimate without consideration of future cost threats. The lower row represents the evaluation of the additional cost threats, typically not evaluated by the NASA concept study teams, associated with design and schedule growth. Finally, all analysis is integrated and leveled against other concept evaluations. TRACE RESULTS FOR PRIORITY MISSIONS Summaries of the results of the TRACE evaluations of the 17 priority missions identified by the decadal survey on the basis of their potential to address the 12 key science goals (see chapters 4 to 15) and potential technical viability presented in Boxes C.1 through C.18. These missions are as follows (not in priority order): • Mercury Lander (Box C.1); • Venus Flagship (Box C.2;) • Intrepid: lunar long-range rover traverse (C.3); • Endurance: South Pole Aitken Basin sample collecting rover (A-variant Box C.4, R-variant Box C.5); • INSPIRE: in situ solar system polar ice roving explorer (Box C.6); • Mars In Situ Geochronology (Box C.7); PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-9 

• Mars Life Explorer (Box C.8); • Ceres Sample Return (Box C.9); • Europa Lander (Box C.10) • Enceladus Orbilander (Box C.11); • Enceladus Multiple Flyby (Box C.12); • Titan Orbiter (Sea Probe Descoped) (Box C.13); • Centaur Orbiter and Lander (Box C.14); • Uranus Orbiter and Probe (Box C.15); • Calypso: Uranus Moon and KBO Flyby (Box C.16) • Odyssey: Neptune Orbiter and Probe (C.17); and • Triton Ocean Worlds Surveyor (C.18) PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-10 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-11 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-12 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-13 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-14 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-15 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-16 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-17 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-18 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-19 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-20 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-21 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-22 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-23 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-24 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-25 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-26 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-27 

PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-28 

SUMMARY TRACE evaluations are a forward-looking technical readiness and budget (cost and schedule) evaluation process typically used to assess Pre-Phase A mission concepts. Linked technical, cost, and schedule evaluations were developed for each of the priority mission concepts selected by the committee. The use of historical databases and evaluation of the technical risk, cost, and schedule histories of analogous space systems which have already flown provide a high degree of confidence that the resulting assessments are realistic and credible. The TRACE-process-derived mission costs are typically higher than the cost estimates provided by mission advocates and design center study teams. The reason is that project-derived cost estimates are typically done via a bottoms-up or “grass roots” approach, and beyond standard contingencies they do not include probabilities of risk incurred by necessary redesigns, schedule slips and other unforeseen required adjustments. In other words, these estimates typically do not account for the “unpleasant surprises” that historically happen in nearly all space mission developments. TRACE evaluations include a probabilistic assessment of required reserves assuming that the concept achieves the mass and power as allocated or constrained by the respective stated project contingencies within the schedule as stated by the project. In addition to these reserves, additional cost threats are also included that quantify potential cost growth based on design maturity (mass and power growth) and schedule growth. Potential cost threats for larger required launch vehicle capability are also included, if required. It is the combination of these reserves and cost threats that are often the main reason for the large differences between the TRACE evaluation and the project estimate, when they occur. Differences in the estimates for hardware costs (instruments and flight systems) can also be a contributing factor. Cost increases and schedule slippage has plagued spacecraft missions since the dawn of the Space Age. As such, fiscal uncertainties are a significant threat to long-term planning and budget management. Even with the most careful evaluation, the ultimate cost of a spacecraft is poorly constrained until relative late in its development cycle. As a result, a decadal survey assessing concepts in their earliest phases of development faces a quandary: Throw caution to the wind and accept the assurance of mission advocates that their concept is doable within a specific cost and schedule; or adopt a specific approach to program evaluation. While not a panacea, the TRACE process’ use of the history record of “unpleasant surprises” provides the best tool currently available to add a degree of realism to long-term program planning. REFERENCES Calvin et al. 2020, MOIRE: Mars Orbiter for Resources, Ices, and Environments, Mission Concept Study, Planetary Science Decadal Survey, Jet Propulsion Laboratory, Pasadena, California. Available at https://science.nasa.gov/solar-system/documents. Campbell et al. 2015, Report from the Next Orbiter Science Analysis Group, MEPAG NEX-SAG Final Report, Mars Exploration Program Analysis Group, Pasadena, California. Castillo-Rogez et al., 2020, Ceres: Exploration of Ceres’ Habitability, Mission Concept Study, Jet Propulsion Laboratory, Pasadena, California. Available at https://science.nasa.gov/solar- system/documents. Cohen et al. 2020, In Situ Geochronology for the Next Decade, Final Report Submitted in response to NNH18ZDA001N-PMCS: Planetary Mission Concept Studies, NASA Goddard Space Flight Center, Green Bank, Maryland. Available at https://science.nasa.gov/solar-system/documents. Congress (Congress of the United States) 2008, National Aeronautics and Space Administration Authorization Act of 2008, Public Law 110-422, Section 1104b. Davila et al. 2021, Enceladus Multiple Flybys: Is there Life Beyond Earth? Planetary Mission Concept Study for the 2023-2032 Decadal Survey, NASA Goddard Space Flight Center, Green Bank, Maryland. Study report available at https://tinyurl.com/2p88fx4f. Diniega et al. 2019, Report from the Ice and Climate Evolution Science Analysis Group, MEPAG ICE-SAG PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-29 

Final Report (2019), Mars Exploration Program Analysis Group, Pasadena, California. Ernst et al., Mercury Lander: Transformative Science from the Surface of the Innermost Planet, Planetary Mission Concept Study for the 2023-2032 Decadal Survey, Applied Physics Laboratory, Columbia, Maryland. Available at https://science.nasa.gov/solar-system/documents. Gilmore et al. 2020, 2020 Venus Flagship Mission Study: A Mission to Explore the Habitability of Venus and the Origins of Earth-Sized Planets Both Near and Far, A Planetary Mission Concept Study Report Presented to the Planetary and Astrobiology Decadal Survey, NASA Goddard Space Flight Center, Green Belt, Maryland. Available at https://science.nasa.gov/solar-system/documents. Hand et al. 2017, Report of the Europa Lander Science Definition Team, JPL D-97667, Jet Propulsion Laboratory, Pasadena, California. Hayes et al. 2021, Titan Orbiter and Probe, Mission Concept Report for the Planetary Science and Astrobiology Decadal Survey, Jet Propulsion Laboratory, Pasadena, California. Study report available at https://tinyurl.com/2p88fx4f. Hofstadter et al. 2017, Ice Giant Pre-Decadal Study Final Report, JPL D-100520, Jet Propulsion Laboratory, Pasadena, California. Howett at al 2020, Persephone: A Pluto-System Orbiter and Kuiper Belt Explorer, Planetary Mission Concept Study, Applied Physics Laboratory, Columbia, Maryland. Available at https://science.nasa.gov/solar-system/documents. Lillis et al. 2020, MOSAIC: Mars Orbiters for Surface-Atmosphere-Ionosphere Connections, Mission Concept Study, Planetary Science Decadal Survey, Jet Propulsion Laboratory, Pasadena, California. Available at https://science.nasa.gov/solar-system/documents. MacKenzie et al 2020, Enceladus Orbilander: A Flagship Mission Concept for Astrobiology, Planetary Mission Concept Study for the 2023-2032 Decadal Survey, Applied Physics Laboratory, Columbia, Maryland. Available at https://science.nasa.gov/solar-system/documents. Martin et al. 2021, Calypso: In Search of Ocean Worlds in the Uranian System and Kuiper Belt, A Planetary Mission Concept Study Report Presented to the Planetary and Astrobiology Decadal Survey, Applied Physics Laboratory, Columbia, Maryland. Study report available at https://tinyurl.com/2p88fx4f. Neal et al., The Lunar Geophysical Network, Final Report Submitted in response to NNH18ZDA001N- PMCS: Planetary Mission Concept Studies, NASA Goddard Space Flight Center, Green Bank, Maryland. Available at https://science.nasa.gov/solar-system/documents. NRC (National Research Council) 2006, An Assessment of Balance in NASA’s Science Programs, The National Academies Press, Washington, D.C., p. 32. NRC 2007a, Decadal Science Strategy Surveys: Report of a Workshop, The National Academies Press, Washington, D.C., pp. 21-30. NRC 2007b. NASA’s Beyond Einstein Program: An Architecture for Implementation, The National Academies Press, Washington, D.D., pp. 66-114. NRC 2010, New Worlds, New Horizons in Astronomy and Astrophysics, The National Academies Press, Washington, D.C., Appendix C. NRC 2011, Vision and Voyages for Planetary Science in the Decade 2013-2022, The National Academies Press, Washington, D.C., pp. 331-353. NASEM (National Academies of Sciences, Engineering, and Medicine) 2013, Lessons Learned in Decadal Planning in Space Science: Summary of a Workshop, The National Academies Press, Washington, D.C. NASEM 2015, The Space Science Decadal Surveys: Lessons Learned and Best Practices, The National Academies Press, Washington, D.C. O’Rourke et al. 2021, ADVENTS: Assessment and Discovery of Venus’ Past Evolution and Near-Term Climatic and Geophysical State, Mission Concept Study to Report to the NRC Planetary Science and Astrobiology Decadal Survey 2023-2032, NASA Goddard Space Flight Center, Green Bank, Maryland. Study report available at https://tinyurl.com/2p88fx4f. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-30 

Robinson et al 2020, Intrepid Planetary Mission Concept Study Report, Jet Propulsion Laboratory, Pasadena, California. Available at https://science.nasa.gov/solar-system/documents. Rymer et a.l 2020, Neptune Odyssey: Mission to the Neptune-Triton System, NASA Mission Concept Study for the Astrobiology and Planetary Decadal Survey, Applied Physics Laboratory, Columbia, Maryland. Available at https://science.nasa.gov/solar-system/documents. Stroud et al. 2021, CROCODILE: Cryogenic Return of Cometary Organics, Dust, and Ice for Laboratory Exploration, Mission Concept Study to Report to the NRC Planetary Science and Astrobiology Decadal Survey 2023-2032, NASA Goddard Space Flight Center, Green Bank, Maryland. Study report available at https://tinyurl.com/2p88fx4f. Telus et al. 2021, CORAL: Centaur Orbiter and Lander, Mission Concept Study to Report to the NRC Planetary Science Decadal Survey, Applied Physics Laboratory, Columbia, Maryland. Study report available at https://tinyurl.com/2p88fx4f. Venera-D 2019, Venera-D: Expanding Our Horizon of Terrestrial Planet Climate and Geology Through the Comprehensive Exploration of Venus—Report Of The Venera-D Joint Science Definition Team, Phase II Final Report, NASA Headquarters, Washington, D.C., and Space Research Institute, Moscow. Williams et al. 2021, Mars Life Explorer, Mission Concept Study, Planetary Science and Astrobiology Decadal Survey, Jet Propulsion Laboratory, Pasadena, California. Study report available at https://tinyurl.com/2p88fx4f. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION C-31