CHAPTER SIX

Interface Between Research and Operations

The aim of research efforts to make subseasonal to seasonal (S2S) predictions is to operationalize forecasts on these timescales to provide consistent and timely information that various sectors can rely upon for decision-making. As described in Chapter 3, there is an increasing demand for easily accessible and comprehensible S2S forecasting information that is updated on a regular basis, which provides an increasing demand for operational S2S products. As described in Chapter 4, there is an ongoing progress to identify and characterize sources of predictability for S2S forecasts. As described in Chapter 5, the models that have been developed as research tools to study scientific questions regarding the processes responsible for weather and ocean variability, climate change, and predictability are being improved at a rapid pace (e.g., Delworth et al., 2006; Gent et al., 2011; Kiehl et al., 1998). Any strategy to improve the provision of operational S2S products should incorporate the efficient migration of advances from the research community into operational forecasts (NRC, 2012b).

A key part of the interface between research and operations should account for the interaction between the research and operations communities. A natural tension exists between the two communities. Researchers in the academic community are generally rewarded for exploring new concepts, because scientific journals often favor publications that are viewed as making major advancements as opposed to incremental changes. Forecasters in the operational community are often under pressure to maintain a natural conservatism. Because numerous users depend on operational forecasts and invest in using specific outputs, there is pressure on the forecasters to maintain consistency in their forecast products. This tension can be healthy, but some of these cultural differences can impede dialogue and collaboration. Facilitation of the interface between research and operations should start with acknowledgment of these cultural differences.

This chapter describes several ongoing efforts to promote collaboration across the research and operations communities in the United States and elsewhere. The topic of forging better links between research and operations in climate modeling was covered comprehensively in a recent NRC report (NRC, 2012b), and the committee builds upon rather than repeats that report’s highly relevant but more general findings. Thus, in this chapter, the committee makes the case for several recommendations that are

more specific to the S2S context. In particular, the committee emphasizes research-to-operations related to the development of multi-model ensemble (MME) forecast systems, because of their importance to reducing uncertainty and increasing the skill and reliability of S2S forecasts (Chapter 5).

CURRENT ACTIVITIES AT THE INTERFACE OF S2S RESEARCH AND OPERATIONS

A number of efforts, both nationally and internationally, work at the interface of research and operations in S2S forecasting. Many of these efforts were introduced in Chapter 2. This section describes in greater detail several prominent efforts, highlighting their importance for bridging research and operations, particularly in the area of developing MME forecast systems; it does not, however, provide a comprehensive list of all efforts.

Demonstration MME S2S Forecast Systems

As introduced and described in Chapter 2, a number of international efforts are aimed at improving MMEs and issuing MME forecasts. The North American Multi-Model Ensemble (NMME)¹ (Box 2.2) started as a research and demonstration project for S2S prediction involving universities and laboratories in the United States, NCEP, and the Canadian Meteorological Center (CMC). It is currently supported by NCEP, by CMC through a Cooperative Agreement, and through research dollars from NOAA, NASA, and DOE. Participating modeling groups include both operational and research centers,² with forecasts from each provided to the NOAA NCEP Climate Prediction Center for evaluation and consolidation as part of its operational seasonal prediction system.

NMME-2, which started in 2012, initially focused on bridging research and operations, and developing requirements for operational seasonal prediction that were then used to define the specifications of a rigorous retrospective forecast experiment and evaluation regime. Other more specific goals of the NMME-2 experiment include to:

___________________

¹ More information on the National Multi-Model Ensemble is available at http://www.cpc.ncep.noaa.gov/products/NMME/NMME_description.html; http://cpo.noaa.gov/ClimatePrograms/ModelingAnalysisPredictionsandProjections/NMME.aspx; https://www.earthsystemcog.org/projects/nmme/ (all accessed April 14, 2016).

² Details of each of the participating models can be found at http://cpo.noaa.gov/ClimatePrograms/ModelingAnalysisPredictionsandProjections/NMME/AbouttheNMME.aspx (accessed February 10, 2016).

1. “Build on existing state-of-the-art U.S. climate prediction models and data assimilation systems that are already in use in NMME-1 (as well as upgraded versions of these forecast systems), introduce a new forecast system, and ensure interoperability so that future model developments can be easily incorporated.
2. Take into account operational forecast requirements (e.g., forecast frequency, lead time, duration, number of ensemble members) and regional/user-specific needs. A focus of this aspect of the experiment will be the hydrology of various regions in the United States and elsewhere to address drought and extreme event prediction. An additional focus of NMME-2 will be to develop and evaluate a protocol for intraseasonal or subseasonal multi-model prediction.
3. Utilize the NMME system experimentally in a near-operational mode to demonstrate the feasibility and advantages of running such a system as part of NOAA’s operations.
4. Enable rapid sharing of quality-controlled retrospective forecast data among the NMME team members, and develop procedures for timely and open access to the data, including documentation of models and forecast procedures, by the broader climate research and applications community.” (Kirtman, 2014)

Based on the results of earlier NMME experiments and demonstrations, NOAA officially began incorporating NMME-2 output into its seasonal operational prediction suite in May 2016.

The Asia-Pacific Climate Center (APCC, see Box 2.1) is a joint activity of the Asia-Pacific Economic Cooperation (APEC) involving 17 operational and research centers from nine APEC member countries. It collects dynamic ensemble seasonal prediction data from these centers and produces seasonal forecasts and outlooks that are disseminated to APEC members every month (see Box 2.1). Together with its aligned research institute, Climate Prediction and its Application to Society (CliPAS), APCC has established protocols and databases for contributing model centers’ forecast data, which in turn supports research on predictability. APCC also conducts research on MME methods, which in turn feeds work to issue MME forecasts using the most beneficial methodology (Min et al., 2014).

Through these and other MME research and demonstration efforts (e.g., ENSEMBLES, DEMETER—see Chapter 2), much has been learned about MME forecast systems. As also described in Chapter 5, a primary finding has been that MME forecasts in general show improved forecast skill and reliability when compared with the individual model forecasts. Thus, first and foremost, these demonstration and aligned research-operations efforts have shown the potential for operational MMEs.

Finding 6.1: Multi-modal ensembles have been demonstrated to be a viable mechanism for advancing S2S forecasts.

Although it is not yet a forecast demonstration project, the World Climate Research Programme/World Weather Research Programme (WCRP/WWRP) joint research project—the S2S Project—started in January 2013 with a primary goal to advance subseasonal forecasting by coordinating prediction and predictability research enabled by the establishment of a multi-model database. The database consists of ensembles of subseasonal (up to 60 days) forecasts and is supplemented with an extensive set of reforecasts following TIGGE—the THORPEX Interactive Grand Global Ensemble—protocols (Box 2.3). Although this project leverages operational systems, the forecasts are currently disseminated with a 3-week delay, thus emphasizing the use of operational forecast system data for use by both the research and operations communities.

One advantage of the S2S Project database is the diversity of operational models. However, the models are inconsistent in terms of in forecast start date, frequency, lead time, and reforecast strategy, which makes it difficult for data exchange, performance inter-comparison, and research. The inconsistencies between models also reflects the fact that the subseasonal forecast is still in its infancy stage.

There are significant opportunities for leveraging the S2S Project, not only the database but also the associated subproject research and activities. Most of the subprojects already strongly link to entities and activities outside the S2S Project (e.g., the MJO Subproject links to the WCRP/WWRP Working Group on Numerical Experimentation [ WGNE] MJO Task Force). During a recent workshop on Subseasonal Prediction hosted by the European Centre for Medium-Range Weather Forecasts (ECMWF),³ in concert with an S2S Project Steering Group meeting, for example, a working group discussed and recommended avenues for broader international collaboration that would more fully take advantage of the S2S Project and NMME. These included (1) the establishment of a Task Team on S2S process-oriented diagnostics as well as forecast skill verification metrics—keeping in mind both model development and stakeholder interests, (2) more routine interaction between the leads of the subseasonal NMME Core Team, S2S project co-chairs, S2S Verification subproject leads, and the World Meteorological Organization (WMO) Commission for Basic Systems (CBS) leads, (3) joint workshops among NMME, S2S, CBS, etc., (4) coordinated research experimentation, with leadership in part provided by WGNE. Additional recommendations and full details will be provided in the final report of the ECWMF workshop expected for release in early 2016.

___________________

³http://www.ecmwf.int/en/learning/workshops-and-seminars/past-workshops/workshop-subseasonalpredictability.

There is an opportunity for enhanced collaboration between the operational centers contributing to the S2S Project and the WMO Joint CBS/CCl Expert Team on Operational Predictions from Subseasonal to Longer-time Scales (ET-OPSLS), which operates through the WMO Lead Centre (LC) in Korea. Building on the existing mechanism whereby the LC has access to the same S2S database, but without the 3-week embargo, could enable a closer synergy between the research community and operational centers’ research efforts.

Finding 6.2: The S2S Project has begun to archive data from operational forecast systems and coordinating research using these databases to accelerate improvements in subseasonal prediction, as well as plays key role in guiding the development of decision support projects.

Example Research to Operations Strategies and Arrangements

ESPC

The National Earth System Prediction Capability (ESPC) interagency program was established in 2010 “to improve coordination and collaboration across the federally sponsored environmental research and operational prediction communities for the scientific development and operational implementation of improved global prediction at the weather to climate interface.”⁴ ESPC advocates for a number of things at the interface of research and operations, including common coupled modeling architectures and standardization of data, archives, and forecast skill metrics.

As part of ESPC, NOAA and the U.S. Navy use a number of mechanisms to improve the flow of technology into operational weather and ocean systems. These include focused workshops, visiting scientist programs, special sessions at professional conferences, testbeds, and focused transition teams such as the Navy’s development- and operations-transition teams and the NSF/NOAA Climate Process Teams.

European Efforts

The UK Met Office operates a single science program, covering both weather and climate and both research and transition to operations. This approach, with the same management responsible for all parts, means that research to operations challenges

___________________

⁴http://espc.oar.noaa.gov/.

are significantly lessened, in part because the entire program can be designed with research to operations in mind. In addition, an active science partnerships program seeks to entrain developments and expertise from international partners and the academic community. The latter is facilitated by relationships with a number of key universities (including jointly funded positions and PhD studentships). The UK Met Office also has a crucial strategic relationship with the Natural Environment Research Council (which funds much of the academic research in the UK), which enables co-design and co-funding of major research programs such as those developing the next generation dynamical core and working to improve the representation of convection in weather and climate models. This integrated approach—both in the design of the programs and the mixture of academic and Met Office scientists carrying them out—is of great benefit for research to operations.

ECMWF also has a strongly focused research program, targeted at generating operational improvements. It hosts a significant number of visiting scientists and holds numerous workshops (involving international experts), seminars, and training programs.

Finding 6.3: The United States can learn from international efforts to connect research and operations more closely and can build upon current national efforts to coordinate research and operations activities.

CHALLENGES IN RESEARCH TO OPERATIONS AND OPERATIONS TO RESEARCH

Motivated by the growing expectation for governments to provide S2S services (see, e.g., Dr. Jane Lubchenco’s testimony during Congressional hearings to confirm her as Undersecretary of Commerce for Oceans and Atmosphere and Administrator of NOAA [Lubchenco, 2009]), there is a desire within the research community to migrate experimental prediction models into operational use, for example, the NOAA Climate Test Bed effort to build the NMME (described previously). There is also a desire to improve on operational models by transitioning model components and/or parameterization schemes from experimental models developed in the broader community. In theory, this migration of experimental model components and parameterizations into operational use has the potential to efficiently leverage the U.S. S2S research community and to provide more skillful and comprehensive operational predictions.

However, gaps exist between research goals and operational imperatives, for example, changing an operational model requires a more careful and elaborate process than that for a research model. In addition, there are mismatches between the resource

requirements needed to maintain an operational model and the current distribution of resources between research, development, and operations.

There is also a mismatch between the expectations of the operational numerical weather prediction (NWP) and the seasonal prediction community and the model research and development community. The principal measure of success of work that is supported by a typical short-term (e.g., 3-year) research grant is the number, quality, and impact of the research publications that result from the project. Because researchers are not rewarded for developments that become “operational,” they are not motivated to perform what they consider to be substantial extra work to transform research results into operational methods or procedures. Their view that scholarly publications speak for themselves has been described as a “loading dock” approach—the researchers make their results available to the operational prediction community via peer-reviewed publications (i.e., left on the loading dock), who, in turn, are responsible for deciding how to use them (see Chapter 7 for additional discussion of workforce issues).

From the operational community perspective, the great many constraints imposed by operations must be considered by the researchers who seek to improve operational predictions. To effect a transition from research to operations, they argue, the research community must modify its developments to conform to the constraints of the operational models and resources. However, for the research community to use operational models for research, operational centers must provide infrastructure support for full testing of developments in the operational environment. The mismatch between the two communities’ expectations has been called the “valley of death,” that is, a communication and interaction gap (NRC, 2012b). There is a need to better align the two communities and provide adequate resources so that good ideas can be more rapidly and effectively transformed into operational practice.

Operations-to-research faces a similar issue. To make research relevant and focused on scientific issues exposed by operations, operational centers must provide access to their data and analysis, operational models, and multiyear reanalysis and retrospective forecasting runs. In addition, operational and agency development laboratories must provide access to key model developers and software engineers to facilitate use of code and data by the outside community. These activities are demanding for personnel, computational and storage resources, something operational centers have traditionally lacked (see also Chapter 7).

At the core of the challenges within research to operations for S2S is the question of how to expand participation in the development and improvement of the operational prediction systems in the operational centers. Currently, the major route to move

research results and successes into operations is by diffusion through the professional literature and meetings and some focused symposia, such as those of ECMWF, for example, Seasonal Prediction in 2012 and Subseasonal Predictability in 2015. Continuation of these efforts is important to continue the transfer of information along the research to operations pipeline. However, common access to operational systems and data is a requirement for improving the flow of technology and information.

Finding 6.4: There is a clear need to provide the research community with greater access to operational systems or mirror systems to aid in transitioning component and parameterizations from the research community into operational centers.

WAY FORWARD FOR RESEARCH TO OPERATIONS

Over the past two decades, substantial progress has been made in understanding some of the phenomenological drivers for S2S prediction, and operational centers have made progress in improving S2S forecast skill (see Chapters 2 and 5). However, there is significant opportunity to increase operational skill from current levels in seasonal and subseasonal forecasts (Chapter 4). Better connections between the research and operations communities are crucial to improving operational skill and advancing research and quasi-operational prediction systems into operational mode. Operational centers should carefully choose which updates to make because improvements to one type of forecast may come at the expense of another, and users who invest heavily in developing products rely on the output from operational centers being in a specific form. Ensuring that the best research results get into operational use and allowing researchers to contribute and learn from the experiences of the operational centers are ongoing challenges for the weather/climate community at large (NRC, 2012b). For S2S in particular, a few areas require enhanced attention, including planning and work to develop operationalized multi-model S2S forecast systems, providing the S2S research community with greater access to the data and models from operational systems, and organizing the operational community to be ready to provide S2S forecasts on the consequences of large, unanticipated events.

Operationalizing Multi-Model Ensembles

As described in Chapter 5, MMEs have demonstrated potential for improving the skill and uncertainty quantification of S2S forecasts. Although many design considerations must be addressed to develop the best operational S2S forecast systems (see Recommendation K), it has been established that an MME outperforms a single model en-

semble at extended timescales (Kirtman, 2014; see Chapter 5). The committee believes that all evidence points to the necessity of MMEs for enabling more skillful S2S forecasts in the next decade. Thus a long term goal of the U.S. operational centers should be to develop a fully operational MME that spans the S2S timescales.

An immediate question to ask is whether the existing North American Multi-Model Ensemble (NMME; see above) could be built upon to develop a fully operational S2S MME. NMME was originally funded and intended to provide a highly valuable research vehicle for advancing S2S prediction and especially for investigations into optimal multi-model ensemble configurations. While the NMME-2 is quasi-operational (see Box 2.2) and providing data that is incorporated into NCEP operational products, external users, and the research community on a real-time basis, its existence is partially dependent on short-term research funding. The CFSv2 is the current NCEP operational seasonal prediction system and is supported as such, and output of the Canadian Meteorological Center system’s two models are being provided through a Cooperative Agreement. All other systems in the NMME-2 are supported through federal research funding activities including NOAA Office of Oceanic & Atmospheric Research (OAR), NASA, and DOE. In the committee’s view, this quasi-operational approach may be misguided in the long run, given the varying levels of operational robustness of all of the contributing models. Participants such as universities or research laboratories have little motivation or funding to sustain the provision of 99.9% reliable, on-time data delivery of forecasts with adherence to rigorous software validation and verification or to scheduled software update cycles. Even the more applied laboratories such as NASA and NOAA’s Geophysical Fluid Dynamics Laboratory (GFDL) do not have the mission, funding, or infrastructure to meet the rigorous requirements imposed upon operational data providers.

That said, it will be difficult to create an MME in a true operational environment. The MME’s value lies in the differing data assimilation, dynamics, and physical parameterizations of the contributing models, which leads to cancelation of model biases and better assessment of predicted probability distributions. This implies that an operational system of systems should include distinctly different systems. However, each individual system requires an expensive host of scientists and software engineers, especially as computer systems become more complex (Chapter 5). S2S forecast systems are also very computationally expensive and would significantly impact a single operational center’s computing resources.

A critical challenge for the next decade therefore will be the design and implementation of an operational multi-model S2S forecast system that operates within finite operational resources. Meeting this challenge will require exploration of the entire

trade space for S2S prediction systems including using common model components (e.g., ocean, wave, land, aerosol, ice), statistical-dynamical prediction such as analogs, or stochastic parameterizations to achieve suitable skill with fewer or even a single system. Thus, the committee’s related recommendation above (Recommendation K) is an important step in the deliberate design of an operational MME system (and one that is not based solely on expediency). Exploring the various options in this space, with the goal of optimizing skill while reducing the number and uniqueness of system members, should lead to a tremendous reduction in the cost of human resources to maintain a multi-model operational system.

This exploration of strategies should take place within the context of the current NMME, WWRP/WCRP S2S Project and/or APCC S2S forecast efforts, but further demonstration of benefits in increased skill and reliability will be a key component of the national research agenda for S2S prediction. However, it is critical that the broader research community be engaged in this effort. As described for climate models generally (NRC, 2012b), operational centers would best promote these advances by providing operational models, supporting data sets, and a user-friendly environment that allows external researchers to test experimental parameterizations and/or model components in an operational setting. This requires a “collaborative framework where datasets and metrics/targets are standardized for careful intercomparison” (Sandgathe et al., 2013). The NOAA Climate Test Bed activity⁵ provides the potential for such a connection. In addition, the Next Generation Global Prediction System (NGGPS⁶) is an effort by the National Weather Service to accelerate research to operations for weather forecasting. The current efforts would need significant enhancement to fully address the challenges of designing an operational MME.

As described above and in Chapter 5, systematic design of a robust multi-model S2S system will be a large, complicated, and expensive experiment, which would benefit tremendously from a central, coordinating authority and preferably central funding as well. The various interagency efforts within the U.S. government, for example ESPC, could offer this coordination and determine a plan forward with the long-term goal of establishing an operational MME. The plan for a fully operational MME could start with models from North American operational centers, but could be expanded to include models from other countries.

Recommendation L: Accelerate efforts to carefully design and create robust operational multi-model ensemble S2S forecast systems.

___________________

⁵http://www.nws.noaa.gov/ost/CTB/, accessed January 27, 2016.

⁶http://www.nws.noaa.gov/ost/nggps/, accessed January 27, 2016.

Specifically:

• Use test beds and interagency and international collaborations where feasible to systematically explore the impact of various S2S forecast system design elements on S2S forecast skill, in particular the questions of how many and what formulations of unique models are optimum in an operational MME (see also Recommendation K).
• Assess realistically the available operational resources and centers that are able to contribute to an operational MME.

Provide Research Community with Greater Access to Operational Systems

A number of ongoing activities are working at the interface of research and operations. As such, rather than recommend a wholesale restructuring of the relationship between these communities, the committee chooses to make several targeted recommendations to help continue progress in this area. That said, the committee emphasizes that these recommendations will take significant time and effort to accomplish and that they should be viewed as part of a longer term challenge to address over the course of the next decade.

As the S2S community looks to bridge S2S research into S2S operational predictions, a new paradigm is needed for the U.S. research to operations pipeline. At the center of the challenges within research to operations for S2S is the participation by researchers with new ideas and tools in development and improvement of the prediction systems in the operational centers. Closing the gap between research and development and operational prediction will require the capability to establish workflow provenance and automate analysis where feasible and reasonable, for which research and development are needed. The major route for research results and successes to move into operations are by diffusion through the professional literature and meetings and some focused symposia, like those of ECMWF, for example, Seasonal Prediction in 2012 and on Subseasonal Prediction in 2015. Continuation of these efforts is important to further the transfer of information along the research to operations pipeline.

The section on current activities describes efforts by the U.S. government, ECMWF, and the UK Met Office to improve the flow of research to operations. Promoting and expanding these mechanisms would help to include more scientists in plowing the new ground of S2S.

Beyond current activities, the committee recommends two additional approaches. One is the use of a data archive of operational deterministic and ensemble forecasts and retrospective forecasts and their initialization data by the research community

outside the operational centers. This would facilitate further analyses of sources of predictability and identify new sources of predictability, skill diagnostics, and more. Data storage will be a challenge for these types of request, although not an insurmountable one. These activities could focus on specific periods of time, for example, a field campaign relevant to S2S or a special year of interest such as a given phase of the Quasi-Biennial Oscillation (QBO), to help minimize the archiving effort required by operational modeling centers. Alternatively, national data depositories could be established for this and other “big data” projects.

The WWRP/WCRP S2S project and NMME have already started work on making operational center data available to the research community, including the reforecast and forecast data. Overall, there is a pragmatic and near-term opportunity for operational centers to help to make such archived data more available—through the S2S Project or otherwise—for the research community. This access could potentially be achieved via test centers. In addition, there is an opportunity for the research community to take better advantage of the operational center data that is now becoming available from the S2S Project.

A second approach of substantial benefit would be to provide researchers with the capability to request reruns of operational models or conduct numerical experiments using operational models themselves. Some of the visiting scientists programs have enabled researcher to insert their diagnostics into operational models, but the ability for researchers to request reruns of operational models for specific time periods or even test new parameterization will be difficult given the resource constraints of operational centers. The ability for users to run operational models themselves will be a more difficult challenge, one that will involve access to the modeling code as well as sufficient computing power to run the code. Some modeling centers have already released the codes of previous versions of operational models. However, making codes of current operational models available and accessible requires a significant effort by the operational centers. To improve the flow of advances between research and operations, operational centers should work toward meeting requests for reruns and making model codes available for researchers over the course of the next two decades.

Lastly, most decision-makers are likely to acquire information via an intermediary. As described in Chapter 3, the committee recommends an ongoing process that involves those that use forecast products to make decisions and those who produce forecasts to work iteratively to develop improved forecast products. The private sector will be a key part of that process. Transferring enhancements in private-sector products or performance to improvements at the operational centers presents a significant challenge,

but part of the iterative process of product development could include feedback from private industry for identifying and improving system performance.

Recommendation M: Provide mechanisms for research and operations communities to collaborate, and aid in transitioning components and parameterizations from the research community into operational centers by increasing researcher access to operational or operational mirror systems.

Specifically:

• Increase opportunities for S2S researchers to participate in operational centers.
• Enhance interactions with the international community (e.g., the S2S Project and APCC) and with the WMO LCs.
• Provide better access in the near term to archived data from operational systems, potentially via test centers.
• Develop, in the longer term, the ability for researchers to request reruns or perform runs themselves of operational model forecasts.
• Encourage effective partnerships with the private sector through ongoing engagement (see also Recommendation B).

Establish Capability to Respond to Unanticipated Events

Large, unanticipated events that may influence the weather/climate system may be natural, accidental, or deliberately caused by humans. Natural events of this scale within the past two centuries include major volcanic eruptions (such as Pinotubo, El Chichon, and Agung in the 20th century or Krakatoa and Tambora in the 19th century). The range of such natural events could also include meteoroid or comet impacts. Prominent and recent accidental events that raised public concern about their widespread impact on timescales of weeks and longer include the Deepwater Horizon oil spill (NRC, 2013) and the Fukushima Diachi nuclear accident (NRC, 2014; see also Chapter 3). Deliberate events have included the 1991 Kuwait oilfield fires or, more benignly, the decision to substantially curtail Chinese industrial emissions to improve air quality during the 2008 Olympics. On geological timescales there is strong evidence of much larger volcanic eruptions and impacts by extra-planetary bodies. Similarly, future human-induced climate forcing events could greatly exceed the magnitude of historical events. Of particular note, the 2015 NRC report Climate Intervention: Reflecting Sunlight to Cool Earth finds that large-scale albedo modification to cool the climate system is technically feasible with a scope that could be done unilaterally by a single nation or even a wealthy non-state actor, but that the consequences of such actions would not be evenly distributed and could alter atmospheric circulation and precipita-

tion patterns. These types of large unanticipated events have the potential to affect the weather/climate system (and potentially much of the Earth system depending on the event) over S2S timescales.

The committee recommends that the nation should develop and maintain a system for projecting the consequences of any unusual forcing events—in particular over S2S timescales—in order to aid emergency response and disaster planning. This system should be mobilizabile within 1week (giving time to ascertain the details of the forcing and select the appropriate set of predicted quantities) and return preliminary results for timescales from the near-term to seasonal and out to a 1-year forecast horizon by the end of a second week. The quality of the system components should be established before any such event via documentation of hypothetical test cases in the peer-reviewed literature. For timescales longer than 1 year, there is time to mobilize the broader scientific community to expand the recommended on-demand prediction system and develop new capabilities tailored to the specifics of the major event in question. This system should be initialized using the same data sets and systems as the operational S2S prediction, and have configurations that include a full range of physical and chemical atmospheric, oceanic, cryosphere, and ecosystem processes, drawing upon capabilities from the nation’s operational and research weather and climate forecasting systems. Other scientific disciplines should be engaged to prepare components for this system that may be appropriate for such events as volcanic eruptions, meteor impacts, a limited nuclear war, oil or other chemical spills in large water bodies, atmospheric or oceanic releases of radioactivity, or releases of biologically or radiatively active gases and aerosols. Although this system will draw upon the expertise of the nation’s research community, it will need to be considered an operational system, with the same robustness and reliability as is expected from other operational forecast systems. The development of this new capability for projecting the consequences of unusual forcing events will leverage many existing research activities or efforts to develop longer-term Earth system projection capabilities, but it will still constitute a substantial new effort by the nation. As such, the fiscal and computational resources to support this new capability should not be drawn from the limited resources currently dedicated to improving existing S2S forecasts.

By their very nature, it is not possible to statistically validate predictions of the consequences of unusual events by examining skill in simulating large numbers of observed events, or by doing bias corrections in the same way as is done for operational predictions. Rather, bias control could be done analogously to how it is handled for centennial-scale climate projections: predictions of consequences should be taken from the difference between an ensemble of simulations in which a forcing event occurs and an ensemble of identically initialized “control runs” in which the event does

not occur. The credibility of the prediction system can be evaluated by examining its ability to simulate well-observed smaller analogous events (e.g., reasonable simulations of the 1991 Mt. Pinotubo eruptions are a necessary condition for credibly simulating the consequences of a hypothetical Yellowstone Caldera mega-eruption of the magnitude that occurred 640,000 years ago; this is analogous to the use of 20th-century simulations to establish the credibility of coupled climate models for 21st-century projections of climate change). In addition, because significant nonlinearities in the Earth system are possible, the prediction systems should be used for a diverse series of hypothetical forcing event scenarios of sufficiently large magnitude to ascertain that they will work sensibly when called upon.

Quality assurance and a critical evaluation of skill are essential for any official forecast product. For routinely generated products, these are usually performed by making a large number of retrospective forecasts of well-observed previous situations. For unprecedented forcing events, this may not be possible. Publication of simulations of hypothetical or poorly observed historical events in the peer-reviewed scientific literature may provide one adequate path toward providing quality assurance. However, it is important that protocols for quality assurance be agreed upon and these steps toward quality assurance be taken for a wide range of potentially useful projection capabilities. This quality assurance must occur before an unanticipated forcing event, so that the capabilities are available to provide timely and useful guidance to decision-makers and the public once such an event occurs.

Recommendation N: Develop a national capability to forecast the consequences of unanticipated forcing events.

Specifically:

• Improve the coordination of government agencies and academia to enable rapid response to unanticipated events and to provide S2S forecasts using the unanticipated events as sources of predictability.
• Utilize emerging applications of Earth system models for long-range transport and dispersion processes (e.g., of aerosols).
• Increase research on the generation, validation, and verification of forecasts for the aftermath of unanticipated forcing events.

This page intentionally left blank.