Evaluating Progress of the U.S. Climate Change Science Program Methods and Preliminary Results (2007) / Chapter Skim
Currently Skimming:
2 Process for Evaluating Progress: Task 1a
2 Process for Evaluating Progress: Task 1a
Pages 17-34
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 17... ...
A number of approaches have been proposed, ranging from the comprehensive evaluation framework laid out in Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program (NRC, 2005) to simply tracking completion of CCSP products.
|
From page 18... ...
Some natural divisions in the CCSP and the information available for the evaluation are described below. Organizing the Evaluation The CCSP has three major components: overarching goals, research elements, and cross-cutting issues (Figure 2.1)
|
From page 19... ...
The most unambiguous source of information on program progress is found in the CCSP's annual report to Congress, Our Changing Planet, which provides selected examples of progress and plans each year. Our Changing Planet also tallies agency budgets into some CCSP categories (overarching goals, focus areas, and research elements; see Figure 2.1)
|
From page 20... ...
In response to a committee request, the CCSP Interagency Working Group on Human Contributions and Responses/Decision Support Resources sent out a questionnaire to eight participating agencies. The resulting list of programs and budgets is given in Appendix B and was used in the committee's evaluation.
|
From page 21... ...
metrics to the five CCSP overarching goals. Their rough evaluation found that nearly all of the metrics were relevant to the CCSP and that quantitative scores could be assigned with a reasonably high level of confidence for most metrics. Scoring was most difficult for metrics dealing with impacts, communication of results, and use of results by stakeholders.
|
From page 22... ...
In contrast, the human contributions and responses research element has had relatively little funding, multiagency coordination, or science community participation. However, nonscientist stakeholders are more engaged in selected agency initiatives, and as a result the program has had some successes in informing resource management and decision making, as well as in advancing science.
|
From page 23... ...
CCSP Approaches Agency-Specific Metrics The 1993 Government Performance and Results Act requires federal government agencies to set strategic goals and to measure performance against them on an annual basis. Every federal agency has metrics to measure performance, and CCSP program managers are trying to determine whether the metrics of participating agencies can also be used to assess CCSP progress as a whole. However, the aggregate of agency metrics reflects neither the level of agency involvement in CCSP programs nor the breadth of the CCSP, a conclusion also reached in NRC (2005)
|
From page 24... ...
Leader with sufficient authority to Leadership exists in IWG members, who The IWG has been expanded to cover decision allocate resources, direct research effort, have agency budget authority support tools, which makes it difficult to assess the and facilitate progress effectiveness of leadership on human contributions and responses 2. A multiyear plan that includes goals, CCSP strategic plan as well as North CCSP strategic plan contains good research focused statement of task, implementation, American Carbon Program and Ocean questions, but little information on discovery, applications, and integration Carbon and Climate Change plans have implementation.
|
From page 25... ...
directed plan requires more resources than currently research program specifically to allow the planned program to available be carried out 3. Sufficient resources to implement and Opportunities for basic research cut across With the exception of a few focused programs, sustain each of the following: (a)
|
From page 26... ...
inventory and land cover changes. Strong that the CCSP is emphasizing the creation of links exist to international carbon cycle new databases necessary to advance human research efforts contributions and responses research Output Metrics 1.
|
From page 27... ...
However, planned expansion of research element networks has not materialized 3. Appropriate stakeholders judge these Work benefits and is benefited by research Little documented interaction with stakeholders, results to be sufficient to address scientific in other science fields (e.g., land use with possible exceptions such as the RISAs and the questions and/or to inform management and land cover, ocean acidification and International Research Institute for Climate and and policy decisions ecosystems)
|
From page 28... ...
are of uncertainties, increased understanding of carbon budget, although uncertainties in beginning to lead to increased understanding uncertainties, or reduced uncertainties that some areas have narrowed. Feedbacks of uncertainties and development of decision support decision making or facilitate the remain important areas of research strategies and tools that incorporate uncertainties, advance of other areas of science (coupled modeling advances)
|
From page 29... ...
appear to have informed policy and improved decision of the Intergovernmental Panel on Climate informed policy making, although not to the making Change extent envisioned in the CCSP strategic plan 2. The program has benefited society in Carbon management is being addressed at Positive societal benefit, especially if measured terms of enhancing economic vitality, the state or regional level, if at all against the level of investment promoting environmental stewardship, protecting life and property, and reducing vulnerability to the impacts of climate change 3.
|
From page 30... ...
Annual performance measure: • Improve climate models: Produce a new continuous time series of retrieved cloud properties at each Atmospheric Radiation Measurement site and evaluate the extent of agreement between climate model simulations of water vapor concentration and cloud properties and measurements of these quantities on time scales of 1 to 4 days NASA Progress in understanding and improving predictive capability for changes in the ozone layer, climate forcing, and air quality associated with changes in atmospheric composition. Annual performance measure: • Demonstrate that NASA-developed data sets, technologies, and models enhance understanding of the Earth system, leading to improved predictive capability in each of the six science focus area road maps Progress in quantifying global land cover change and terrestrial and marine productivity, and in improving carbon cycle and ecosystem models.
|
From page 31... ...
The CCSP is tracking the 21 synthesis and assessment products associated with the overarching goals, and participating agencies are tracking the 208 milestones, products, and payoffs associated with the research elements. Reporting whether each milestone, product, or payoff is completed; on or behind schedule; or discontinued responds directly to Office of Management and Budget directions in the Fiscal Year 2006 federal budget plans (OMB, 2005)
|
From page 32... ...
: Category A: improve data sets in space and time, and improve estimates of physical quantities Category B: improve understanding and representation of processes Category C: improve predictability, predictive capabilities, or assessment of uncertainty Category D: improve synthesis and assessment to inform Category E: improve assessment and management of risk, and improve decision support for management and policy making These product categories provide a measure of the maturity of the program, starting from data collection, to predictions of future climate changes, to improved use of information to better serve society. The complete matrix is given in Appendix C
|
From page 33... ...
A similar number of scores would be required to evaluate the major components of the program (five overarching goals, seven research elements, and six cross-cutting issues) using the 24 metrics identified in NRC (2005)
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.