Skip to main content

Currently Skimming:

3 Major Challenges to Achieving an Effective Assessment Process
Pages 39-62

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 39...
... FRAMING A CREDIBLE AND LEGITIMATE PROCESS Framing the assessment process such that it is perceived as credible and legitimate by all relevant stakeholders is a major challenge (Farrell et al.
From page 40...
... This simplifies the task of achieving credibility and legitimacy compared to impact assessments that inevitably consider value judgments and trade-offs. Science has strong norms for how to carry out deliberations about the state of knowledge, so it is relatively easy, in principle if not in practice, to conduct a credible process assessment
From page 41...
... . A well-established and successful model for process assessments has emerged that involves the following key elements: • Getting a critical mass of the world's most respected scientists in the relevant fields to participate; • Ensuring broad participation and sponsorship; • Having an intensive, science-focused process of deliberation that is of such high quality that it attracts the number and quality of participation required and produces reports that can serve as authoritative scientific references in the field; • Urging the process to provide clear consensus evaluations of the state of knowledge on key policy-relevant questions, to the extent the underlying knowledge base allows; • Writing clear, compact summaries with the involvement and consent of the scientific author teams; and • Disseminating the summary messages prominently and consistently.
From page 42...
... Impact Assessments Impact assessments have been much less successful in achieving credibility and legitimacy than process assessments (Parson et al. 2003; Moser 2005)
From page 43...
... Scenarios provide process assessments with a set of possible futures that are plausible even if a response assessment does not attempt to assign probabilities to future states of the world. This ability to separate process and response assessments by the use of scenarios makes both process and response assessments much easier to conduct successfully than impact assessments, albeit at the cost of resting the process assessment on "what if" scenarios rather than detailed analysis of social system responses.
From page 44...
... Even the most thoroughly integrated assessments often neglect issues that are of considerable importance in decision making, such as equity (Morgan and Dowlatabadi 1996)
From page 45...
... workshop Understanding and Responding to Multiple Environmental Stresses (NRC 2006) notes that integrated assessments are required when the impacts and decisions are place-based (i.e., specific to a locality or region)
From page 46...
... . The rationale for an integrated assessment is that the separation that differentiates process, impact, and response assessments from each other is ultimately artificial and may lead to science that is less robust than might be ideal.
From page 47...
... . Therefore, negotiating this boundary is a balancing act between achieving credibility, legitimacy, and salience (Jasanoff 1987)
From page 48...
... The appropriate stakeholders may be scientists, decision makers, politicians, resource managers, the public, and so forth. Even those without the technical expertise to engage in the assessment process may still perceive themselves to be stakeholders because they are affected by the outcome, particularly in the case of impact assessments.
From page 49...
... Most often, response assessments are initiated before regulatory policies to mitigate the environmental risk are in place, with a goal of evaluating whether technology options are feasible and sufficiently cost-efficient that regulatory policies can be adopted without major economic impacts. Asking industry representatives to participate energetically in these sorts of assessments and to disclose information openly is comparable to asking potentially affected industries to provide a green light to impose regulations (Parson 2006)
From page 50...
... How should the technical questions at issue and the assessment process be defined to protect against unwillingness to disclose technical information to regulators and regulatory advocates, unwillingness to disclose technical information to competitors, and hijacking of technical deliberations to serve interests of individual firms or technologies? Stakeholder Capacity Building Capacity building to develop a common language and technical understanding among assessment stakeholders can greatly enhance the effectiveness of assessments.
From page 51...
... . Investments in capacity building can have payoffs in multiple areas, including expanding the informed audience for the assessment, contributing to future assessment effectiveness, expanding the ability of decision makers to act on scientific information, equipping participants with new knowledge in assessment methodology and tools, and building a scientific community that is more sensitive to needs and concerns of the broader society.
From page 52...
... In fact, programs such as the Regional Integrated Science Assessments illustrate how regional assessments can be nested within a national or global assessment. A need for further development of decision-support tools for global change assessments has been pointed out by many authors, particularly at the regional scale (Scheraga and Smith 1990; Easterling 1997; Morgan et al.
From page 53...
... Stratospheric ozone assessments are peer reviewed, but do not undergo public review because the scope of the issue is limited and the perception of legitimacy in this process by all major stakeholders has been established over time. Effective review processes increase credibility by allowing many individuals to evaluate the veracity of the report and increase legitimacy by involving a larger range of stakeholders (Edwards and Schneider 2001)
From page 54...
... A range of approaches have been employed for characterizing uncertainty related to environmental change, including standard statistical techniques, model-based sensitivity analysis, expert judgment, and scenario development. It is difficult, if not impossible, to objectively and quantitatively define uncertainty for many of the complex issues related to climate change.
From page 55...
... The most widely adopted approach to estimating uncertainty in model predictions is sensitivity analysis, in which the range of probable model outcomes is assessed using a series of model realizations with a range of values for the various inputs. Both the sensitivity to specific model parameters (e.g., how clouds or air-sea exchange are represented)
From page 56...
... For example, the stratospheric ozone models failed to predict the appearance of the Arctic ozone hole because they did not include important heterogeneous chemical reactions. Expert Judgment In many global change assessments that evaluate potential outcomes in complex systems, the characterization of uncertainty for policy makers must
From page 57...
... The communication process should be active during the entire assessment, and not solely be designed around the report dissemination. Effective assessments have a comprehensive, multifaceted communication strategy right from the start, encompassing an analysis of the target audiences, alternate modes of reaching and engaging them, desired responses (e.g., policy decisions, legislation, technological innovation, standards, international
From page 58...
... The MA, the stratospheric ozone assessments, and the ACIA are examples of assessment processes that produced an array of different publications and communications for different audiences -- from policy makers to business to the general public. Effective assessment reports are concise, accessible, visually attractive, and user-friendly; investment in the writing and review process is critical.
From page 59...
... Accessibility is also enhanced when both the basic report and other documentation are made available on the web in usable form and, if appropriate, translated into key languages. A summary is possibly the most crucial element of the written assessment product and an effective dissemination process, especially if it is concise, unbiased, clear about assumptions and uncertainties, free of jargon, and relevant to the various needs of decision makers.
From page 60...
... Process assessments have a well-worn path for success, which includes incorporating a critical mass of experts, ensuring broad participation, focusing intensively on a specific science issue, developing consensus through a state-of-the-science evaluation, instituting authoritative review, and offering a clear summary of the results. As a result, process assessments are less likely to be subject to criticism for their credibility or legitimacy, unless the science topic is associated with the perception of great political importance.
From page 61...
... Further, the assessment outcomes are much more likely to involve analyses that depend on assigning values; hence, it is more likely that they will generate a diversity of opinions that impact legitimacy and credibility. It is important that impacts and response assessments be designed in a manner that accepts the challenge of broadening the participation and the level of communication, while also recognizing that there is still much to be learned about how to conduct these types of assessments successfully.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.