National Academies Press: OpenBook

A Framework to Guide Selection of Chemical Alternatives (2014)

Chapter: 11 Identifying, Comparing, and Implementing Alternatives

« Previous: 10 Life Cycle, Performance, and EconomicConsiderations
Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×

11

Identifying, Comparing, and Implementing Alternatives

The final steps (Steps 10, 11, and 12) of the framework integrate information from previous evaluations in order to identify acceptable alternatives, compare alternatives to make a decision (an optional step), and implement selected alternatives. By Step 10, assessors should have sufficient information to determine which, if any, of the potential alternatives have a lower overall negative impact to human health, ecotoxicity and other considerations, as well as meet other requirements established in Step 2. Figure 11-1 shows where these steps fall in the framework, and Figure 11-2 provides more information about what is involved in Steps 10-12.

IDENTIFYING ACCEPTABLE ALTERNATIVES WITHIN EXISTING FRAMEWORKS

All of the reviewed frameworks integrate information across different domains to identify acceptable alternatives, but they give varying levels of guidance on how to do this. The CA SCP and REACH frameworks set acceptability criteria at the beginning of the chemical alternatives assessment process and then measure alternatives against those criteria. The UCLA MCDA framework provides a structure for integrating information from different domains, but focuses on ranking alternatives rather than determining alternatives’ acceptability. Both TURI and UNEP do not give specific guidance on determining acceptability, but they both demonstrate within their case studies how to organize disparate data in matrices, as well as how to use simple markers (e.g., +, -, or =) to denote better or worse performance against the chemical of interest in each criteria (Table 11-1).

The EPA’s DfE framework relies on the stakeholders participating in an assessment to evaluate certain aspects of the alternatives, and leaves the integration of disparate data to the individual companies implementing the alternatives. The BizNGO, German Guide, and Lowell frameworks provide little or no guidance on how to integrate the disparate information from different steps in the alternatives assessment to determine the acceptability of alternatives.

STEP 10: IDENTIFYING ACCEPTABLE ALTERNATIVES IN THE COMMITTEE’S FRAMEWORK

Consistent with the reviewed frameworks, a step to identify acceptable alternatives based on information from different domains has also been included in the committee’s framework. Inclusion of this step is also aligned with the committee’s Statement of Task, which states that the framework should be able to consider the full range of benefits and shortcomings of substitutes, including balancing factors such as product functionality, product efficacy, process safety, and resource use. It is beneficial to retain a dedicated step for determining basic acceptability, without forcing a ranking or further narrowing the list of alternatives, because having more than one acceptable alternative may be desirable under certain circumstances. For example, if the entity performing the alternatives assessment is a regulator considering taking action on the chemical of interest, offering a range of alternatives to replace the chemical allows complex industries and supply chains the flexibility to select the option that best suits each company’s needs. Also, alternatives assessments that identify multiple acceptable alternatives can spur innovation if alternatives that have minor shortcomings in certain areas in the initial assessment can be further developed so that they become preferred replacements. And finally, if irresolvable issues are encountered during the implementation of a selected alternative, it may be useful to have other alternatives that have been identified as acceptable to consider.

While Step 7 has the goal of integrating information about the potential human health and ecotoxicity impacts of the alternatives to determine if alternatives meet the definition of safer, Step 10 has the goal of integrating the additional disparate data from Steps 8 and 9 to determine which

Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×
Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×

images

FIGURE 11-2 Additional detail about Steps 10-12.

alternatives, if any, are acceptable. For the purposes of this framework, an alternative is considered acceptable if it meets the requirements established in Step 2, and does not have undesirable aspects or trade-offs so that it no longer has a lower overall negative impact to human health and/or the environment. This definition of acceptability depends on the requirements set in Step 2 as opposed to factors that entities may simply have a preference for, because this step is focused on identifying acceptable, not preferred or optimal, alternatives. For example, companies will generally have a preference for lower cost alternatives, but unless a clear requirement is set (such as a maximum price that the entity will consider), the preference for lower cost should be addressed as a part of comparing alternatives in Step 11.

By Step 10, assessors should have sufficient information to determine which, if any, of the potential alternatives can be considered acceptable. Coming out of Step 7, each alternative will have been assessed to determine how it compares to the chemical of concern in the original domain of concern and environmental and human health hazards, as well as exposure. Results from Steps 8 and 9.1 may provide additional information about the broader potential environmental impacts of the alternatives, as well as their constituents and breakdown products. This information should be used to determine if the remaining alternatives continue to meet the requirement to have lower overall negative impact to human health and/or the environment.

Another important aspect of Step 10 is that it is a critical point for documenting the findings of all of the analyses that have been performed throughout the assessment, as well as documenting any monitoring or other measures that may be required to make particular alternatives acceptable. As noted in Chapter 3, thorough documentation of findings allows for more effective critical evaluation of alternatives assessment results and comparability across assessments. The organization of the reports and documentation is left to the discretion of the

Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×

TABLE 11-1 Example of a Summary Matrix for Multiple Alternatives across Several Criteria in a Case Study based on the TURI Framework

Assessment Criteria Lead (Referenced) Comparison Relative to Lead
Bismuth Ceramic Steel Tin Tungsten
Technical and Performance Criteria Density 11.34 g/cm3 - - - - +
Hardness (desirable for “feel” and noise) Soft Mohrs: 1.5 + + + = (pure) + (alloy) +
Malleability (split-shot application) Yes - - - = -
Low melting point (for home production) 622ºF + - - + -
Corrosion resistant Yes = = - = =
Environmental Criteria Highly toxic to waterfowl Yes + ? + + +
Toxic to aquatic species Yes + ? + + +
Primary drinking water standards (MCL Action Level) 15 µg/L ? ? + (iron) + (FL & MN) ?
Human Health Criteria Carcinogenicity EPA B2 IARC 2B + + + + +
Developmental toxicity Yes (Prop 65) + + + + +
Occupational exposure: REL (8-hour TWA) 0.050 mg/m3 ? + + + +
Cost Retail price Low - - -/=/+ - -
Availability of end product Excellent - - - - -

Note: + Better = Similar - Worse? Unknown
SOURCE: Adapted from TURI (2006).

assessor, but summary tables or other graphic methods should be used to compile and present results for multiple alternatives against multiple criteria.47

If no alternatives are determined to be acceptable at the conclusion of Step 10, research can be initiated to develop new alternatives and/or improve existing ones, a process informed by observations about how each alternative failed to meet the requirements established in Step 2 or the expected negative impacts to human health and the environment that were considered unacceptable to the entity conducting the alternatives assessment.

STEP 11: COMPARING ALTERNATIVES IN THE COMMITTEE’S FRAMEWORK

If a single alternative must be selected for implementation, or if it is necessary to identify

_____________

47 Similar requirements are also found in the CA SCP, REACH, TURI, and UNEP frameworks.

Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×

preferred alternatives, ranking or other comparative methods may be applied to the alternatives identified in Step 10. Additional information about this optional step is provided in the next section.

Comparing Alternatives within Existing Frameworks

Four frameworks (IC2, CA SCP, Lowell, and UCLA MCDA) use information from different domains to evaluate alternatives so that they can be ranked, categorized, or narrowed to a single choice for implementation. The IC2, CA SCP, and Lowell frameworks allow ranking, but give no guidance on specific methods on how this, as well as categorizing or narrowing the choice of alternatives, should be done. As a result, the choice of approach is left up to the discretion of assessor. The UCLA MCDA framework deals more comprehensively with ranking. The framework referred to as UCLA MCDA is actually a specialized form of the more general approach of decision analysis, which is a field that applies decision theory to real-world, complex problems. For this reason, it is well suited for integrating disparate information for each alternative and evaluating that information against multiple criteria (Siddall 1972; Keeney and Raiffa 1976; Triantaphyllou 2000; Wang 2002; Figueira et al. 2005; Hatamura 2006; Edwards et al. 2007). Applying MCDA methods requires the creation of a model that reflects the decision maker’s preferences, value trade-offs, and goals (Belton and Stewart 2002). The UCLA report Developing Regulatory Alternatives Analysis Methodologies for the California Green Chemistry Initiative (Malloy et al. 2011) demonstrates how such an approach could be applied within a chemical alternatives assessment.

In the UCLA report, two case studies are presented in which an MCDA model created to compare a regulated hazardous substance and its alternatives is used to analyze alternatives to perchloroethylene (PCE) for dry cleaning and lead (Pb) solders in electronics. The variables for the model were first selected from the human health, environmental, resource usage, performance, and economic factors that must be evaluated under California Assembly Bill AB 1879, the enabling statute for CA SCP. For each major area of interest (upper-level criteria), sub-criteria with metrics against which alternatives could be scored were identified (measurement sub-criteria). Weights for the criteria within the model were based on averaged scores of expert and stakeholder ratings of the relative important of the different criteria.

The authors were able to rank the alternatives in both cases using two commonly used MCDA methods: multi-attribute utility theory (MAUT)48 and outranking.49 When the authors varied the assigned criteria weights, they found relatively small variations in the rank order using different stakeholder weighting levels. The authors were also able to run the model with different assumptions about missing data (such as assuming missing data were to receive the worst or best possible score for an end point) to see if these differences affected the rank order of alternatives. When they used different assumptions and policies for handling data gaps, they found that different assumptions could result in significant differences in the relative rank of alternatives. The authors also examined the impact of converting continuous data (such as LD50) to categories (high, moderate, low), and found that the rank order of alternatives with respect to top performers was unchanged, but that the remaining alternatives were significantly reordered. Based on the successful application of MCDA methods in the case studies, the authors concluded that MCDA was a viable way to assist in the evaluation of complex data within a chemical alternatives assessment.

Step 11: Comparing Alternatives in the Committee’s Framework

A step for comparing alternatives has been included as an option in the committee’s framework to address the need to differentiate among acceptable alternatives in order to select a single alternative for implementation or to identify preferred alternatives from the list of acceptable ones.

The decision analysis methods used in the MCDA example are one way to integrate disparate information to rank or differentiate alternatives. Those methods may be most helpful when evaluating complex data across many criteria, for cases with many alternatives, or when the substitution decision is expected to have a high impact. Although MCDA methods may be useful in some cases, they may be more complicated than required for many

_____________

48 MAUT is an optimization approach that represents the decision-maker’s preferences as utility functions, and attempts to maximize the decision-maker’s overall utility.

49 “Outranking models compare the performance of two alternatives at a time, in terms of each criterion, to identify the extent to which one alternative out-performs the other, then aggregates that information for all possible pairings to rank the alternatives based on overall performance on all criteria” (Malloy et al. 2011).

Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×

assessments. There are other ways to rank, compare, and select alternatives, including simple matrix methods (such as the one shown in Table 11-1), as well as the decision rules described in Chapter 9.

Ultimately, the choice of integration method is beyond the scope of the committee and is left to the assessor. All assumptions, data, and methods should be documented regardless of the method used. The criteria and weighting used within these decision analysis methods are context-dependent and based on values, and therefore left to the discretion of the assessor or entity conducting the alternatives assessment.

STEP 12: IMPLEMENTING ALTERNATIVES

By the end of Step 11, the assessor will have either identified preferred alternatives or initiated research on de novo green chemistry alternatives. In those cases where acceptable alternatives are identified, the next step is implementation of the selected alternative(s) in particular applications.

Implementing Alternatives within Existing Frameworks

Implementation of alternatives is addressed only to a limited degree in the frameworks reviewed by the committee. Most of the frameworks end with the selection of a preferred alternative. CA SCP requires an implementation plan as well as confirmation that a substitution has occurred. Two frameworks, BizNGO and Lowell, contain steps entitled “Select and Implement Safer Alternative” and “Select and Implement/Review Selection,” respectively. The Lowell framework states that the final step, Review Selection, reflects the fact that technologies are not perfect in terms of environment and social acceptability. Specific chemical selections will need to be re-visited and reevaluated over time, based upon emerging science and changing social expectations. Alternatives assessment is an iterative process on the journey towards sustainable technologies (Rossi et al. 2006).

The most detailed attention to implementation is in REACH. In particular, the European Chemical Agency (ECHA 2011) document Guidance on the Preparation of an Application for Authorization states that entities seeking an authorization (noting that no feasible alternatives are available) must consider: “What research and development activities are needed and/or planned to develop an alternative substance(s) or technology(ies), or develop equipment or processes enabling the use of alternative(s); and (2) What testing must be done and what criteria need to be satisfied before an alternative can be used for a particular function” (ECHA 2011).

The guidance further documents some of the particular implementation challenges for an alternative that might substantiate a longer substitution transition period, including:

  • “The transfer to the alternative requires investments that take considerable time (time needed to plan the necessary changes, to purchase the equipment needed, to build any constructions, to install, to train the personnel, etc.);
  • The transfer to an alternative substance requires regulatory approval (e.g., production of aircraft or medical equipment), or change to an alternative technique requires a review of permit;
  • The transfer to an alternative requires customer approval (e.g., for use in products that must be tested for technical performance over long time periods, or where the transfer to an alternative up in the supply chain may affect the quality of the end products and testing by several downstream user levels is required);
  • An alternative substance is currently not produced in sufficient quantity; and
  • Costs related to investment in new equipment/techniques may depend on other planned investments, age of the current equipment, etc.” (ECHA 2011).

Under REACH, if an applicant for authorization identifies a suitable alternative, that entity must develop a substitution plan for the alternative, documenting timing, supply chain consultation, and how the transition will occur, including evaluating risk trade-offs. Figure 11-3 provides a graphic of the substitution planning steps under REACH.

Finally, several occupational health chemical substitution frameworks not considered in Chapter 2 include steps focused on implementation and evaluation of the consequences of the substitutions. For example, OSHA’s framework, Transitioning to Safer Chemicals, has the steps, “Piloting the Alternative” and “Implementing and Evaluating the Alternative,” with information on each step (OSHA 2014). The European Commission’s Directorate General for Employment, Social Affairs and Equal

Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×

images

FIGURE 11-3 Steps to follow in a substitution planning process required for companies seeking authorization for a Substance of Very High Concern under REACH. SOURCE: Adapted from ECHA 2011.

Opportunities has published a substitution framework (developed by the Finnish consulting group GAIA) entitled “Guidance for minimizing chemical risk to workers’ health and safety and the environment,” which includes implementation guidance in a Plan-Do-Check-Act approach (Pessala et al. 2012).

Implementation is an underdeveloped topic within chemical alternatives assessments, but one that is critical for minimizing unintended health, environmental, and performance consequences, as well as ensuring continuous improvement in transitioning to safer chemicals and products. Many alternatives assessments only peripherally consider the actual adoption of alternatives and the challenges that might occur either up or downstream of the

Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×

production process, or the potential unforeseen health and environmental hazards that may be created at this stage. Implementation is often the most challenging part of the substitution process and may require ongoing monitoring to identify and minimize potential trade-offs. It is important to give attention to implementation early in the chemical alternatives assessment, including involving stakeholders affected by a chemical substitution. Such attention can make informed substitution more successful and develop a culture of continuous improvement toward safer processes and products. It can also help reduce the potentially high costs of additional substitutions by identifying potential problems before full-scale implementation.

Implementing Alternatives in the Committee’s Framework

An implementation step has been included in the committee’s framework, consistent with the best practices in the existing frameworks. This step is intended to support action related to the implementation of safer substitutes by helping entities identify alternatives and mitigate expected or unintended consequences in the substitution process, and ensuring a more successful, informed substitution. Ultimately a chemical alternatives assessment is not worthwhile if the alternatives are not adopted.

An implementation step will prepare for the following challenges that may result:

  • Identified acceptable alternatives may work in a specific range of applications but not in others that have specific processing or operating requirements. Or, alternatives may change product functionality. For example, lead free solders may not perform well in high pressure or low gravity applications (NASA 2009).
  • Identified acceptable alternatives may require significant process design or formulation chemistry changes to achieve functionality that may not have been considered. These changes may affect product quality or may lead to increased or modified exposures or new hazards.
  • Implementing alternatives may require work practice changes that can affect worker exposure pathways, increase potential hazards (toxicity and physical), and affect productivity if they do not work as well (Bartlett et al., 1999).
  • Changes may make end of life collection and recycling more challenging or lead to unexpected end of life exposure concerns.
  • New understanding about the toxicity of a chemical substitute or a chemical used alongside the substitute in a process or product. New understandings about environmental fate or life cycle may require adjustment of earlier assumptions.
  • The large amount of information collected in the evaluation phase, including potential conflicts in information and inertia within a firm or sector to make changes, might lead to paralysis that inhibits action on alternatives adoption.

While some of these challenges will have been addressed in earlier technical and environmental and health and safety evaluations, some particular changes may not have been foreseen and thus encountered for the first time during implementation.

Goal and Objectives of the Implementation Step

The overall goal of this step is to enhance the implementation of safer alternatives while avoiding unintended consequences of substitutions. Planning for implementation supports the transition to safer chemicals, processes, and products and allows for continuous improvement, updating understanding as scientific knowledge evolves on hazards and exposures, and minimizing or avoiding adverse health and ecosystem impacts that might be identified in the application phase of an alternative.

The objectives of the implementation step are to:

  1. a. Document final choices of preferred alternatives, including the rationale and potential information gaps that need to be filled;
  2. b. Identify potential unintended consequences that might occur at the application phase of a substitute and implement modifications to minimize these; and
  3. c. Develop evaluation and continuous improvement plans, including a plan for updating and modifying assumptions and data used in the assessment if substantial new, unanticipated information arises that could affect the evaluation and choice of alternatives.

Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×

While the implementation and evaluation step of the committee’s framework is primarily focused on avoiding unintended consequences in the application phase of substitutes, there are some overlaps with adoption support,50 particularly in the areas of pilot trials and greater integration of alternatives assessment processes as a precursor to adoption efforts.

The Implementation Process

Implementation generally consists of the following series of steps, with a strong emphasis on stakeholder engagement:

  1. Pilot testing, or small-scale testing of a substitute to identify (a) issues related to performance of alternatives, including process or product modifications that are needed to make the alternative function to specifications; and (b) changes in product or process chemistries or work practices (both in product manufacture or use) that might affect worker or consumer health.
  2. Developing an implementation plan, including outlining and documenting the processes and actions needed to implement the substitution, including research and mitigation needs.
  3. Monitoring and evaluation, which are essential to the early identification of potential unintended consequences of substitutions and to the documentation of the beneficial impacts of substitutions and potential improvements. Monitoring needs are context dependent and could involve simple measures, such as air and water monitoring or waste audits, as well as workplace industrial hygiene evaluations. It could also include more complex and formal adverse events post-market monitoring, such as formal adverse reporting systems.

These steps can be completed using a pilot testing/supply chain partnership. In this model, trade organizations and/or government or academic research centers (such as the National Institute for Standards and Technology [NIST] or the Massachusetts Toxics Use Reduction Institute [TURI]) work with a sector in a pre-competitive manner or with a particular firm to evaluate the functionality (and, in some cases, the health and safety implications) of alternatives for a chemical of concern. This type of testing is designed to both share the costs of evaluating the concrete application of a substitute to ensure adequate performance in situ, as well as identify process or formulation conditions that might have to change to ensure functionality, such as the use of new solvents that might present health and safety or environmental concerns. This information can then be fed back into a revised chemical alternatives assessment, if necessary.

Another model of implementation is Intervention Research, an occupational health prevention strategy, reflected in the P2OSH framework (Quinn et al. 2006). The P2OSH framework has an iterative series of steps that involve piloting and then implementing alternatives, and exploring how the adoption process might result in changes to materials used, health and safety of workers, direct costs of adoption, and changes to performance. With this information, the company or organization can determine whether full-scale implementation of an alternative should move forward, or whether design, process, or product modifications should be instituted to minimize potential unintended consequences of a substitution.

Ultimately, these steps will not only support action related to implementation of substitutes, but also identify and mitigate expected or unintended health and safety, ecosystem, performance, or economic consequences during the substitution process.

_____________

50 The implementation step within the committee’s framework is distinct from the concept of “adoption support,” which includes the policies (restrictive, purchasing, or other), incentives, technical assistance, and other support provided to businesses to increase the rate of adoption of safer alternatives.

Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×

This page intentionally left blank.

Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×
Page 149
Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×
Page 150
Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×
Page 151
Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×
Page 152
Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×
Page 153
Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×
Page 154
Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×
Page 155
Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×
Page 156
Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×
Page 157
Suggested Citation:"11 Identifying, Comparing, and Implementing Alternatives." National Research Council. 2014. A Framework to Guide Selection of Chemical Alternatives. Washington, DC: The National Academies Press. doi: 10.17226/18872.
×
Page 158
Next: 12 Case Studies »
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!