A Performance Assessment of NASA's Astrophysics Program (2007)

The NRC reports Astronomy and Astrophysics in the New Millennium and Connecting Quarks with the Cosmos identified a compelling set of scientific goals and opportunities that the field is in a position to address. NASA’s 2003 roadmaps presented a set of plans to address many of these goals. However, between 2003 and 2006 the fiscal posture of the Astrophysics Division changed considerably, and the resulting 2006 plan is substantially different from the 2003 plan in two key ways. First, suites of missions designed to address goals and opportunities in cosmology, high-energy astrophysics, and searches for Earth-like planets have been delayed or deferred. Second, the division’s program portfolio has become heavily skewed toward support of the largest missions, with a resultant loss of balance in terms of mission size, type, and goal and with respect to long-term investment in technology and infrastructure development. In the committee’s estimation, both of these issues, if unaddressed, will reduce the scientific productivity of NASA’s Astrophysics Division for at least the next 15 years. In addition, a number of intangible concerns about the stability of the agency and its interactions with the community have established the perception that NASA is not responding to the guidance provided in the NRC reports.

The dramatic slowing, at the middle of the current decade, of progress toward realizing the scientific opportunities laid out in the Q2C report and starting the missions recommended in the AANM survey report has been due to no one single factor. Rather, the slowdown reflects the confluence of external and internal factors and events that together had the effect of a perfect storm hitting the Astrophysics program at NASA. Moreover, all of this occurred in a larger context: At the beginning of the decade 2000-2010, the ambitions and expecta-

tions of the scientific community were rising, buoyed by the stunning discoveries of the late 1990s; the fact that by the mid-1990s the survey list of projects for that decade was exhausted (at NASA); and the prospects for rising science budgets in the larger fiscal environment of a federal budget deficit that had then turned to a surplus. In addition to budgetary issues, there were other important changes, including management reorganizations; changes in mission philosophy, from faster, better, cheaper back to a more traditional approach; new accounting systems; and increasing mission complexity leading to cost escalation. This chapter addresses the elements that came together to slow progress toward realizing the goals articulated in the AANM survey and the Q2C report, and it offers the committee’s appraisal of some of the likely effects.

An important and quantifiable part of the story of how progress toward AANM and Q2C goals has slowed involves budgetary expectations versus budgetary reality for astrophysics. Estimates of the dollar value of the mismatch between the assumptions of the AANM survey and the budget reality for the period from 2001 through 2010 are summarized below:^1,2

• At least $2 billion: Higher than expected costs for the recommended projects, dominated by cost escalation for JWST;

• At least $0.6 billion: Cost of delay in the HST fourth servicing mission due to the Columbia accident, from the planned 2003 servicing to the projected mid-2008 servicing;

• Approximately $0.5 billion: Carryover projects from previous decades, largely SIM and SOFIA, which were not accounted for in the AANM survey;

• Approximately $0.5 billion: Approximate cost of NASA’s share of the Joint Dark Energy Mission, which was not included in the AANM survey but was recommended by the Q2C report; and

• $0.383 billion: Projected decreases (as given in the president’s FY 2006 budget request) in the astrophysics budget for budget years FY 2006 through

FY 2010 (Table 4.1), redirected toward the president’s new vision for space exploration.

For comparison, the NASA budget for astrophysics for the period from 2001 through 2010 has averaged (and is projected to be) about $1.5 billion per year, for a total of about $15 billion for the 10-year period. Thus, the more than $4 billion gap relative to the expectations of the AANM survey is currently projected to amount by the end of the decade to about 25 percent of NASA’s astrophysics budget. It is no wonder that progress has slowed.

Moreover, this analysis does not include the past expectation that, in the wake of the federal budgetary surplus in the mid-1990s, science budgets would be rising. The stark reality now is that the nation faces a budget deficit that will persist for some years to come.

In the wake of the Columbia accident, the effort to return the remaining shuttles to flight, completion of the International Space Station, and the new exploration activities associated with the vision for space exploration, NASA’s focus has moved to the human spaceflight program. This situation and the resulting squeeze on NASA’s science portfolio are unlikely to change in the foreseeable future unless special action is taken. As quickly as circumstances have changed in this decade, it is certainly possible that future changes could return the astrophysics budget to the levels planned in FY 2005-FY 2006. However, in the estimation of the committee such changes cannot be taken for granted, and NASA’s Astrophysics Division should take steps to resolve the imbalance in the budget currently projected in the FY 2007 request (see Table 4.1).

The dominant reason for the delay or deferral of large sections of the Astrophysics Division’s program (as well as for the imbalance in the program) is the growth in the cost of the projects that the division is implementing. The higher-than-anticipated costs to the division for missions currently in development will total roughly $2 billion more over the course of the decade than the AANM decadal survey anticipated based on the cost data the survey committee was given by NASA. In addition, the cost estimates for the missions still in the formulation phase (such as LISA and Con-X) are also significantly greater than anticipated at the time of the AANM survey.

Many factors have contributed to the lack of cost realism for these missions. Some causes are imposed from the top down by institutional factors. Others are driven from the bottom up due to mission complexity, engineering capabilities, infrastructure limitations, and other elements. In the committee’s judgment, the following factors have combined to limit NASA’s ability to implement AANM survey objectives at the resource level anticipated in the survey report:

• The AANM survey recommendations were developed in the “faster, better, cheaper” era, using NASA budget estimates that were not rigorously assessed and have since proven to be systematically underestimated.

• Changes in accounting methods, including the move to full-cost accounting, have affected NASA cost estimates in ways that currently result in unplanned cost increases for many missions.

• The effect of increases in mission complexity was not taken into account during the earliest stages of mission formulation.

• New agency policies, processes, and mission assurance requirements have disproportionately driven up the cost of small missions.

• Access to launch vehicles appropriate for smaller missions has been reduced.

NASA has already improved its budgetary and planning processes through the use of independent cost-estimating models, maintaining adequate budget and schedule reserves, and development of suitable tracking metrics. The importance of these techniques is increasingly understood in the scientific community as well.

Since the agency’s shift to full-cost accounting, astrophysics missions must budget individually for staff support, launch services, mission operations, and NASA infrastructure overhead. While this accounting change may be neutral from an overall agency perspective (the agency transferred funds to the Science Mission Directorate to cover the cost of the new responsibilities), programs now bear a greater percentage of the cost burden than in the past. The result is that funds budgeted for Astrophysics program missions now buy less astrophysics. Moreover, from the perspective of maintaining balance, small and moderate missions are affected to a greater degree than large missions, because the work on small and medium missions is often done at universities or other institutions that lack the technical infrastructure to deal with special problems. And whereas NASA’s support was formerly essentially free, full-cost accounting now means that such costs are directly assessed to the program. Since this support must be accounted for up front, the net result is either less mission than what was achievable in the past or higher cost.

A trend toward increasing mission complexity is to be expected for flagship missions as science goals extend beyond first-generation survey objectives to more difficult second-generation objectives. In the committee’s judgment,

the risks associated with increasing mission complexity have been significantly underestimated in the past, leading to a corresponding underestimation of costs and schedules. Complexity and risk are key discriminators between mission categories. Small and medium missions are generally cost-capped either directly or indirectly by being limited to using only available technology. In such cases, risk can be looked at as mostly programmatic and can be mitigated by allocating sufficient project reserves.

Larger missions including flagship missions employ advanced technologies that require substantial development before the risks can be mitigated and associated mission costs thoroughly understood. Therefore, controlling risk not only is important but also requires a rigorous process operated at two levels. First, the process must correctly identify the technology areas requiring study and early development. Second, the management process must effectively plan the effort and then execute it according to a roadmap with specific metrics that are critically reviewed.

The end product of such efforts should be technology development in mission-critical areas so as to achieve a technology readiness level of 6 before a project is fully defined, planned, and costed. The committee credits NASA for implementing management procedures for risk and technology development for larger missions such as SIM and JWST. These missions now appear to have been accurately defined and planned with respect to both cost and schedule.

However, the application of similar risk management processes to smaller missions, although it helps to reduce mission risk, has done so at relatively great expense in both time and money. Given the projected paucity of smaller missions like Explorers and the consequent imbalance in its portfolio, NASA should consider whether cost savings can be realized by scaling down the risk management and mission assurance approaches applied to smaller missions. Such a change in approach would not so much increase risk in smaller missions as recognize the opportunity costs entailed by fewer small missions and, in a sense, the consequent reduction in science return to the overall Astrophysics program.

NASA’s mission assurance requirements have grown increasingly process-oriented and restrictive in an attempt to use external regulation to prevent mission failures. Current mission assurance requirements for small and medium unmanned missions have become restrictive in ways that are counterproductive, given that the time and money associated with such efforts are necessarily subtracted from essential engineering activities. A balanced risk approach with respect to mission assurance, such as that used on WMAP, is described in the NASA Integrated Action Team report³ and should be appropriately applied to smaller missions.

An additional barrier to successful small and medium missions is the cost growth of smaller launchers as the launch industry moves toward larger and more expensive vehicles. For example, the U.S. Air Force has announced its intention to stop supporting Delta II launches after 2009, meaning that the WISE mission planned for launch in 2009 may be the last NASA mission to be launched on a Delta II. Therefore, unless the Delta II line can be retained or an alternative launcher of comparable capability developed, small and moderate astrophysics missions will be affected either by being restricted to small launchers or by becoming much more costly.

In conclusion, the central reason for the difference between the 2003 and 2006 NASA strategies for astrophysics is that, for many reasons, there has been a substantial increase in the cost for nearly all astrophysics missions.

Past NASA successes have encouraged ever more ambitious planning, but at the same time missions are taking longer to complete. It is now clear that four of the space-based missions recommended in the AANM survey, namely JWST, Con-X, LISA, and TPF, are sufficiently complex and challenging that even if additional funds had been available, the missions could not have been completed within the current decade. (In the case of TPF, the AANM survey realized this constraint and recommended only design and development funds.) At the beginning of the decade, the ambitions and expectations of the science community were rising, buoyed by the stunning discoveries of the late 1990s and the prospects for rising science budgets in the larger fiscal environment of a federal budget deficit that had turned to a surplus. And the new missions recommended by the Q2C report made NASA’s plate even fuller. Clearly the real impact of the current situation at NASA has been magnified by the high expectations of the astrophysics community for the decade 2000-2010.

Confronted with more realistic mission costs, the Astrophysics Division appears to have chosen to concentrate its resources on the highest-priority items listed in the AANM survey report and to maintain other missions in development at their current level. The committee believes that when the comparison is between flagship missions, this strategy is consistent with the decadal survey priorities.⁴

The division’s strategy comes at a steep scientific cost, however. By choosing to defer the entire Beyond Einstein suite of missions until 2009 at the earliest, the

agency is also deferring the ability to address the scientific opportunities at the intersection of astrophysics and elementary particle physics, as presented in the Q2C report. By choosing to keep SIM PlanetQuest in stasis until after the launch of JWST and to defer the TPF mission until the next decade, NASA is foregoing the opportunity to take the lead in a field with broad resonance in the general public and around the world. These areas of scientific inquiry are quite exciting to both the astrophysics community and the general public. The committee believes that these choices are one key reason for the science community’s perception that the agency is not making the expected progress in addressing the science goals and opportunities in astrophysics. However, the committee does not believe that this situation can be remedied without either a dramatic change in the fiscal outlook for the Astrophysics Division, or a significant reduction in the capabilities of many of the missions that have been proposed or are in development.

The decision to concentrate resources on the highest-priority items listed in the AANM survey report and to maintain the carryover missions in development at their current level has also caused an imbalance in the Astrophysics program’s mix of mission sizes, as described above in Chapter 3. The committee believes that the current imbalance, if not addressed, will have a significant long-term impact on NASA’s Astrophysics program and the field as a whole.

As displayed in Figure 4.1, three flagship Great Observatory missions were launched in the 1990s, although as noted above, HST and CGRO were developed in the 1980s. Chandra is the only flagship mission both started and launched in the 1990s, although the three HST servicing missions (SM1, SM2, and SM3A) can also be considered large missions executed within that decade.

The Explorer program in the early 1990s launched the Extreme Ultraviolet Explorer (EUVE) and the X-Ray Timing Explorer (XTE), both of which were in the moderate-class category, which is above the typical Explorer mission cost baseline. The “faster, better, cheaper” (FBC) initiative started in the early 1990s was set in motion, in part, to develop smaller and less costly moderate missions in astrophysics as well as the other NASA program areas.

Although the impact of the FBC initiative began to appear in the late 1990s with the reduction in cost and complexity of the SWAS and FUSE missions, the major impact has come in the current decade. Seven Explorer-class missions have been launched or are projected to be launched in the decade 2000-2010, with six of them—HETE-2, WMAP, RHESSI, CHIPS, GALEX, and Swiftbeing carryover missions from the FBC era, launched through 2004. A significant secondary result of the FBC era is that no traditional flagship missions will be launched in the 2000-2010 decade. Spitzer, although considered a Great Observatory, was really a medium-class mission owing to simplifications implemented under FBC principles.

FIGURE 4.1 Run-out cost normalized to 2006 dollars (excluding launch) for NASA astrophysics missions launched or projected to be launched since 1990. Missions with an asterisk were developed under a general strategy of “faster, better, cheaper.” Missions represented in parentheses are shown at their currently planned launch date. Mission classes designated on the right-hand side are approximate based on plotted missions as well as currently baselined mission cost assumptions. Projections into the 2010 decade (represented with “+” symbols) are estimates based on the current understanding of mission status and expected mission starts.

The committee believes that the current era of discovery in astronomy and astrophysics owes much to the combination of flagship missions and Explorer missions that are now operational. Smaller missions using available technology provide opportunities for rapid turnaround when new discoveries are made. For example, when the COBE mission discovered anisotropy in the CMB radiation from the infant universe, the WMAP mission could be quickly deployed to follow up. Smaller missions have the important side benefit of developing spaceflight experience and expertise in young scientists and engineers who then apply their experience to other missions, including flagship missions. However, the highly targeted nature of the Explorer missions leads them to have relatively short scientifically productive lifetimes. Without a steady stream of new Explorer missions to build on the discoveries made by the flagship missions, the productivity of the flagships and of the Astrophysics program as a whole will suffer.

Currently just three astrophysics missions are projected for launch before 2013: GLAST in 2007, Kepler in 2008, and WISE in 2009. With no medium- or Explorer-class missions now in early stages of development, the picture looks particularly bleak for maintaining the current balance in NASA’s astrophysics portfolio. Indeed, the only mission currently manifested for launch in the next decade is JWST, with a projected date of 2013. The elimination of the Small Explorer mission NuSTAR combined with the delays in issuing announcements of opportunity for Explorer missions will lead to a 4- to 5-year “launch desert” in the first half of the next decade. Without new Explorer mission proposal opportunities in the immediate future, this launch desert will extend even further into the next decade.

It is important to note that all of the remaining astrophysics missions in the division’s plans, with the possible exception of the Einstein Probes, fall into the flagship mission category with respect to complexity, cost, and schedule. JWST is still early enough in its development that its cost, schedule, and associated launch date must be viewed as preliminary; considering that the spacecraft has not yet begun the integration phase, historically a phase that has led to cost growth, it is not impossible that further delays and cost increases could occur. Such an event would lead to an even greater impact on overall Astrophysics program content and balance. A decade limited to one new start on a flagship mission plus two to four small and medium missions would constitute a major reduction in the NASA Astrophysics program.

NASA’s Astrophysics Division will always be subject to a variety of budgetary, managerial, and political forces that it can neither predict nor control. But the environment in which the division has operated in recent years has been extraordinarily unstable. During the committee’s data-gathering sessions, the following sources of instability were identified as having a negative effect on the productivity of the division. While the magnitude of the effect caused by these factors is difficult to quantify, the committee includes this discussion as a reflection of concerns within the astronomy and astrophysics community that contribute to the perception that the agency’s program is not progressing.

• New focus for NASA. The new vision for space exploration has changed the budgetary priorities at NASA and, as discussed above, this has led to a projected decrease of $383 million in funding for astrophysics for 2006 through 2010 (see Table 4.1). More importantly, if the NASA budget for science remains constant in the years thereafter (or only grows with inflation), the pressures to devote some significant fraction of the NASA science budget to other priorities is likely to further erode the dollars available for the priorities in astrophysics identified by the AANM, Q2C, and future decadal surveys.

• Leadership changes and reorganizations. Over the past 5 years, NASA has had three administrators. In the same period, the Science Mission Directorate (SMD) has had four reorganizations and three changes in basic administrative guiding documents. Also, the agency continues to suffer from a large number of vacancies in key positions in SMD. For example, in June 2006, 6 of the 10 directorships and deputy directorships of the five SMD divisions were either vacant or occupied by acting personnel.

• Programmatic instability. Numerous decisions have been made over protracted periods. For example, the administration’s failure, for well over a year, to approve an Ariane launch for JWST led to a costly delay in the mission. In the wake of the Columbia accident, a number of internal NASA decisions were made and then reversed on whether to proceed with the next Hubble servicing mission. At a programmatic level, the committee heard of numerous instances of changes made to program budgets late in the fiscal year that hindered the ability of managers to manage their project teams. Such changes were said to be even more damaging to the program than budget levels that are lower than requested.

In addition to the instability within the agency, NASA’s advisory structure underwent a number of changes during the decade. An important feature of the U.S. scientific enterprise is effective communication between the scientists who are making discoveries and the managers within the federal science agencies who are responsible for making decisions. Science managers need expert advice to inform their decisions; scientists need to understand how the science agencies function as well as the basis for the decisions that managers are making. The effectiveness of two-way communication between scientists and science managers is unique to the U.S. scientific enterprise and a key to its success over the past 50 years.

The engagement of the expert science community with federal science managers takes various forms, ranging from high-level policy and strategic advice provided by agency councils (such as the NASA Advisory Council) and NRC committees and studies to more tactical and ad hoc advice from NASA Advisory Council subcommittees and ad hoc committees of experts (e.g., management operations working groups and review panels). Several features appear to be key to a robust and effective advice structure: the openness and transparency of the process; recognition by all that advice is advice, but that managers must finally make the decisions; and a short path between managers and experts that allows for efficient and rapid exchange of information.

The current advisory structure at NASA, which consists of the NASA Advisory Council and its committees and subcommittees, is very vertical. That is, advice at all levels, from tactical and ad hoc expert advice to high-level policy and strategic advice, flows through the NASA Advisory Council to the NASA

administrator and then down through the NASA organization.⁵ This architecture lacks the short path between the relevant manager and outside experts that is needed for efficient communication and dialogue. Furthermore, it puts the high-level NASA Advisory Council, whose primary function is advising the NASA administrator on broad strategic planning questions, in the position of having to digest and transmit more mission- and division-specific advice, which can only distract the council from addressing the global issues. Inevitably, there is a loss of valuable information, and necessarily the critical dialogue between expert advisers and relevant managers becomes difficult if not impossible.

The current vertical structure has deprived the science community of insight into the goals and objectives of the agency, just as it has deprived NASA of needed tactical advice in making critical decisions. A more effective structure would be more horizontal, separating the different advice functions and providing more direct connections between the experts and the relevant managers. The NASA Advisory Council would continue to advise the administrator on policy and strategic matters, with NRC science committees and studies providing advice to the associate administrator for the Science Mission Directorate and the NASA Advisory Council. At the critical tactical level, NASA Advisory Council subcommittees and ad hoc panels would provide advice to SMD’s associate administrator and science managers (including the Astrophysics Division director).

The committee concluded that the following are key principles for an effective advice structure:

• A hierarchy of advice where input is provided to the appropriate level of manager;

• A short path connecting the advising body to the relevant manager; and

• The ability for the manager to engage the advising body directly.

The committee notes that the previous advice structure had these attributes.