National Academies Press: OpenBook

Guidance on PFAS Exposure, Testing, and Clinical Follow-Up (2022)

Chapter:2 Principles for Decision Making Under Uncertainty

« Previous: 1 Introduction
Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×

2

Principles for Decision Making Under Uncertainty

Given that there is some degree of uncertainty in all science, science-based decision making also involves uncertainty (Fischhoff and Davis, 2014). This uncertainty can arise from many different factors, including lack of available evidence, statistical variability, model uncertainty, and “deep” uncertainty about the fundamental scientific processes relevant to a decision (IOM, 2013). Yet, despite this uncertainty, when evidence exists, it helps predict what may happen when a decision is made (Fischhoff and Davis, 2014). Values also influence how scientific evidence is gathered and interpreted, thereby influencing how uncertainty is characterized and what decisions are made on the basis of that particular interpretation of the evidence (Douglas, 2009; Elliott, 2017; IOM et al., 1995). Thus, it is important to consider uncertainty in the broader context of the decision to be made and the values underlying the decision-making process (IOM, 2013).

In response to uncertainty, some public health professionals may wish to avoid acting as soon as science determines the risks of an exposure for fear that the association is not a true one and that acting too quickly could lead to inappropriate and costly public health measures (Boffetta et al., 2008). On the other hand, some may fear that if society waits for more certain evidence, meaningful action will be delayed (Blair et al., 2009). As a group of scientists with different backgrounds and perspectives, the committee grappled with both of these potential outcomes of its decisions. The committee established the principles in this chapter to help accomplish its charge, but also believes they may be useful to support the Agency for Toxic Substances and Disease Registry (ATSDR) in making decisions about medical followup and advice to communities exposed to per- and polyfluoroalkyl substances (PFAS) or other chemicals with uncertain effects.

DEVELOPMENT OF THE COMMITTEE’S PRINCIPLES

In its effort to identify principles for decision making under uncertainty, the committee solicited input from members of PFAS-impacted communities through its virtual town halls (see Chapter 1). The committee also reviewed decision-making frameworks, including a review paper (Norris et al., 2021), the criteria used by the U.S. Preventive Services Task Force (USPSTF) (Krist et al., 2018; Sawaya et al., 2007), GRADE (Grading of Recommendations Assessment, Development and Evaluation) (Moberg et al., 2018), a National Academies evidence framework for decisions regarding genetic testing (NASEM, 2017), the Evidence and Value: Impact on DEcision-Making (EVIDEM) framework (Goetghebeur et al., 2008), the principles used by the C-8 Medical Panel (C-8 Medical Panel, 2013), the ethical principles from the Belmont Report (HHS, 1979), and principles for precautionary reasoning (Resnik, 2021). A brief overview of these sources is included below.

The existing principles and frameworks the committee reviewed were not entirely satisfactory for the purposes of this study. Most of the existing frameworks were not designed to address situations characterized by limited evidence and substantial scientific uncertainty—the contexts the committee was charged to address. The committee understood that when evidence is incomplete, dangers can be associated both with taking action and with failing to act (Douglas, 2009; Elliott and Richards, 2017). Therefore, the committee adapted the principles and frameworks it reviewed to develop a set of principles appropriate for decision making under substantial scientific uncertainty.

Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×

PRINCIPLES PUT FORWARD BY THE COMMITTEE

The committee decided to use the ethical principles proposed in the Belmont Report (HHS, 1979) and developed by Beauchamp and Childress (2001) as a foundation for its approach. Although these principles (nonmaleficence, beneficence, autonomy, and justice) were not initially proposed for decision making under uncertainty, they provide an ethical starting point for addressing such situations. In addition, they encompass many of the criteria and concepts included in other frameworks, such as considerations of benefits, harms, health equity, and human rights. The committee combined the principles of nonmaleficence and beneficence into a principle of proportionality—drawing on the Resnik (2021) framework for precautionary reasoning in response to uncertainty. The committee also added two other principles included in other decision-making frameworks but not captured by the principles from the Belmont Report. One is the principle of feasibility, found in several of the frameworks reviewed by Norris and colleagues (2021), including GRADE. It captures the importance of considering the capabilities of the current medical system and the ways it might need to develop to respond to decision needs. The other is the principle of adaptability, which builds on the call for review and revision in the National Academies framework for genetic testing and the emphasis in the USPSTF framework on revising recommendations in response to emerging scientific information. Box 2-1 briefly describes the committee’s resulting five principles for exposure biomonitoring and patient follow-up under substantial scientific uncertainty about the health effects of PFAS exposure. Each of these principles is discussed in turn below.

Proportionality

The committee’s first principle, proportionality, is adapted from Resnik’s recommendations for precautionary decision making under uncertainty (Resnik, 2021, p. 81):

Proportionality: Decisions should balance plausible harms and benefits proportionally, meaning that the harms and benefits are considered together and weighed based on a qualitative assessment of their potential likelihoods and magnitudes.

Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×

This principle draws on a common theme that cuts across most frameworks the committee considered: balancing harms and benefits. For example, the USPSTF states that in determining the benefits of a preventive services framework, one should consider evidence about the accuracy of screening, harms of early-intervention treatment and not treating, and the treatment benefits and harms of a particular preventive service. The USPSTF framework also considers the accuracy of a screening test and its relationship to the clinical health effect, and the harms associated with the screening for the health condition and treatment of the condition.

The principle of proportionality addresses harms and benefits in situations of substantial scientific uncertainty. To balance harms and benefits proportionally is to consider them together such that, all else being equal, policies with fewer or less severe harms (assessed in terms of their likelihood and magnitude) can be justified based on lesser benefits relative to policies with increased harms. This principle is particularly relevant for decision making under uncertainty because it can justify taking actions to realize potentially significant benefits even when the evidence for those benefits is limited, as long as the potential harms associated with the actions are minimal.

Proportionality provides an alternative to probability-based decision principles that require numerical ranking of the probabilities or utilities associated with effects. Although applying the principle of proportionality calls for considering the likelihood and magnitude of benefits and harms in some form, it does not require assigning numerical rankings to their likelihoods or magnitudes. Because of the qualitative nature of this principle, reasonable decision makers can potentially disagree about how to apply it in specific situations because judgments are required to weigh harms and benefits. Especially when the benefits associated with two policies are similar, and the likelihood and the magnitude of their harms cannot be estimated precisely, the principle of proportionality may not offer decisive guidance. Instead, this principle provides a general way of reasoning compatible with a range of more specific strategies for balancing harms and benefits under different forms of uncertainty (Douglas, 2009; Workman et al., 2020). It affirms that both harms and benefits need to be considered and that they need to be weighed against one another when making decisions (Resnik, 2021).

For this report, the harms of a decision could include physical effects, psychological effects, and opportunity costs (Harris et al., 2014). The committee does not consider the financial costs of paying for tests or treatment a harm. Costs could be considered under the principle of feasibility, but the committee excluded costs from feasibility as well. The reason for excluding costs is that, although a certain test or treatment may be expensive today, costs typically decrease over time if demand for the test or treatment increases. Many other organizations, such as the USPSTF, do not consider financial costs when formulating recommendations based on assessment of health benefits and harms, and the committee followed this approach. The committee did consider as potential harm the loss of income or financial damage incurred as a result of decisions (e.g., decreased property values caused by a community’s learning about chemical contamination).

The principle of proportionality calls for considering benefits and harms that are plausible. The concept of plausibility provides a minimal threshold for determining which benefits and harms should even be considered. Resnik (2021, pp. 80–81) defines a scientific statement as plausible “if it is consistent with well-established scientific facts, hypotheses, laws, models, or theories”—a lower standard of evidence than is typically required for even weak scientific confirmation. Instead, this definition is designed to rule out “armchair speculation” about potential harms and benefits. If a harm or benefit is judged to be plausible, it can be considered in the overall assessment of proportionality. Nevertheless, harms and benefits deemed to have a reasonably low level of evidential support might still be given relatively little weight in proportionality assessment unless their potential magnitudes were estimated to be exceptionally high.

In situations of substantial scientific uncertainty, it is generally unrealistic to expect more than a qualitative assessment of the likelihoods and magnitudes of harms and benefits. This assessment may be informed by evidence, including epidemiological studies, toxicological studies (both animal models and mechanistic studies), modeling, and support from analogy. It should be guided by norms for the responsible acquisition and utilization of evidence, knowledge, and expertise. Still, the appropriate

Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×

standards of evidence for drawing tentative conclusions about potential likelihoods and magnitudes may be much lower than would apply in other scientific contexts (Douglas, 2009; Elliott and Richards, 2017; Ginsberg et al., 2019).

Justice

Justice is the second principle for decision making in situations of substantial scientific uncertainty:

Justice: Decisions should be informed by an emphasis on promoting justice, including by balancing benefits and harms fairly across the population of at-risk individuals, advancing health equity, and respecting human rights. In addition, justice requires consideration of sociohistorical context, stakeholders, existing structural inequalities, and issues of agency (the power a community has to advocate for itself in conflicts).

Justice is central to the Belmont Report and Beauchamp and Childress’s framework for biomedical ethics (Beauchamp and Childress, 2001; HHS, 1979), and elements of justice are included in many of the frameworks discussed in the review by Norris and colleagues (2021). Beauchamp and Childress (2001, p. 226) describe justice as “fair, equitable, and appropriate treatment in light of what is due or owed to persons.” Thus, an injustice involves “a wrongful act or omission that denies people benefits to which they have a right or distributes burdens unfairly.”

In this report, the focus is on three elements of justice: fairness, equity, and human rights. Resnik (2001) emphasizes that decisions under uncertainty should be handled in a manner that is procedurally and distributively fair: the process for making decisions should be fair in the sense of incorporating all relevant stakeholders and employing transparent and accountable procedures, and the endpoints of the decisions should also be fair.

In the health care setting, the second element of justice—equity—calls for promoting health equity. Health equity is “the state in which everyone has the opportunity to attain full health potential, and no one is disadvantaged from achieving this potential because of social position or any other socially defined circumstance” (NASEM, 2017, p. 32). Promoting health equity is not the same as achieving similar health outcomes for everyone. Instead, promoting health equity in the context of this report means designing approaches that enable people to lead full, healthy lives regardless of their social circumstances, such as race, socioeconomic status, and geographic location. Health equity is a particularly important element of justice because it establishes the positive responsibility to provide the conditions necessary for people to lead healthy lives regardless of their social circumstances; without this positive responsibility, some ethicists have argued that the principle of justice does not provide significant guidance beyond what is already found in other bioethical principles, such as proportionality or autonomy (London, 2022).

Finally, justice involves respect for human rights. The committee’s conception of human rights is drawn from the World Health Organization (WHO)-Integrate evidence-to-decision framework (Rehfuess et al., 2019), which emphasizes rights related to the availability and accessibility of health care and such general rights as nondiscrimination.

In cases of environmental pollution such as PFAS contamination, it is crucial to incorporate the concept of environmental justice into efforts to promote fairness, health equity, and human rights. According to the U.S. Environmental Protection Agency (EPA), “Environmental justice is the fair treatment and meaningful involvement of all people regardless of race, color, national origin, or income, concerning the development, implementation, and enforcement of environmental laws, regulations, and policies. This goal will be achieved when everyone enjoys:

  • The same degree of protection from environmental and health hazards, and
  • Equal access to the decision-making process to have a healthy environment.”1

___________________

1 See https://www.epa.gov/environmentaljustice (accessed September 17, 2021).

Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×

The environmental justice movement in the United States arose in recent decades out of the realization that communities of color, which already faced other social disadvantages, were also exposed to disproportionately high levels of environmental pollution (Bullard, 2018; Shrader-Frechette, 2002). Many places where PFAS contamination has been identified are rural and served by private well water, and they face other environmental and social challenges, such as co-occurrences of other contaminants; occupational exposures; and health inequities, such as less access to health care and a lack of economic resources with which to mitigate exposures. These less-advantaged rural communities may also have structural and agency-related factors that can impact their ability to minimize their exposure, as well as to seek and access adequate health care linked to exposures. Promoting environmental justice requires addressing these inequities, such as by taking steps to eliminate disproportionate lack of access to health care, exposure biomonitoring, patient follow-up for PFAS-associated health effects, and environmental remediation and mitigation. Required as well is addressing the needs of vulnerable populations, such as pregnant and nursing women, young children, and those who are immunocompromised.

Autonomy

The committee’s third principle is autonomy:

Autonomy: Decisions should be based on informed decision making by individuals and reflect respect for their values.

The Belmont Report emphasizes the importance of respecting individuals’ autonomy and ability to make judgments; it also warns against withholding information that would enable them to judge. Beauchamp and Childress (2001) likewise emphasize that respect for autonomy requires eliminating constraints on individuals’ decision making and fostering their self-determination. In practice, this principle has often been operationalized by focusing on obtaining individuals’ informed consent to medical treatments or research experiments, but its implications are much broader. In the context of this report, the principle of autonomy calls for health care services to provide individuals with information and collaborate with them to facilitate decisions that accord with their values. This approach is fundamental in situations of substantial scientific uncertainty, when difficult decisions need to be made about how to weigh harms and benefits given limited information about their likelihoods and magnitudes. Different individuals can reasonably approach such situations in different ways. Therefore, the principle of autonomy affirms that patients should play a role in making these decisions.

In addition to fostering decisions that accord with patient values, the principle of autonomy supports other steps to empower patients and their communities. To facilitate patients’ self-determination, health care providers and researchers should take steps to report information back to patients in ways that facilitate their future informed decision making. When patients are tested for exposure to potentially toxic substances, it is essential to provide them with (Brody et al., 2014; Morello-Frosch et al., 2009)

  • information about their exposure level;
  • a description of the potential health impacts, as well as the related signs and symptoms;
  • how their exposure level compares with those of others in their area and across the country;
  • the potential significance of their exposure level; and
  • how they can reduce future risk, either of the exposure itself or its potential health impact(s).

The PFAS Research, Education, and Action for Community Health (PFAS-REACH) project is an example of an effort to provide accessible information to community members about how to interpret exposure biomonitoring results and what follow-up activities could be warranted to address PFAS-associated health effects (Boronow et al., 2017). It is also essential to respect the decision of some

Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×

individuals not to be tested. Some individuals may not want to have information that increases their uncertainty or could lead to future medical monitoring that might not improve their health outcomes.

The principle of autonomy can also guide actions taken at the community level to help foster decision making by individuals. For example, there is growing recognition that community organizations and advocacy groups can generate important information about public health threats and develop actionable solutions (Corburn, 2005; Elliott, 2017; Wandersman, 2003). Thus, health care providers and public health institutions can foster individuals’ autonomy by collaborating with these organizations that can help facilitate informed patient decision making.

Feasibility

The committee’s fourth principle for making decisions under substantial scientific uncertainty is feasibility:

Feasibility: Decisions should into take account resource availability, including follow-up services.

Feasibility, included in many evidence-to-decision frameworks reviewed by Norris and colleagues (2021), denotes the ability to conduct testing, clinical evaluation, and follow-up activities. An assessment of feasibility encompasses the infrastructure and resources currently available and whether they are sufficient to achieve the goals of a decision. In the context of PFAS exposure biomonitoring, for example, feasibility could include such considerations as the time and knowledge providers have available to facilitate environmental exposure assessments or their access to environmental occupational health physicians. Feasibility assessments may also include consideration of whether an action or policy is sustainable over time and it entails important legal, ethical, or bureaucratic barriers (Moberg et al., 2018). At the same time, however, it is crucial to recognize that feasibility assessments are relative to a particular point in time, and inputs and outputs can change. In some cases, a particular policy may not be feasible immediately because of a lack of resources or other institutional barriers, but those barriers may reflect past or present injustices that need to be remedied. Thus, for example, a feasibility assessment can guide policy makers to allocate additional resources when exposure biomonitoring or follow-up services are likely to have benefits or are needed to promote a just health care system. Therefore, the principle of feasibility needs to be considered alongside the principle of justice so it can guide future investments and institutional changes instead of detracting from efforts to address inequities and promote just outcomes. Unfortunately, the lack of a coordinated health system in the United States complicates any assessment of feasibility that goes beyond the requirements imposed on public health agencies. Individuals must rely on insurance or self-pay for many follow-up services, and access to high-quality primary care and specialty care services can be fraught with challenges.

Adaptability

The fifth and final principle is adaptability:

Adaptability: Decisions should respond to new information about harms, benefits, and other relevant considerations (e.g., health equity and feasibility).

The National Academies evidence framework for decisions regarding genetic testing (NASEM, 2017) emphasizes that adaptability is important in decision contexts in which scientific information changes rapidly, as is the case with genetic testing. The USPSTF also emphasizes that its evaluation of the benefits and harms of preventive services is a process that needs to be revisited in light of new advances in research, testing, and treatment capabilities (Sawaya et al., 2007). This responsiveness to new information is especially important in situations of substantial scientific uncertainty, such as the decision

Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×

context surrounding PFAS. In such situations, it is often necessary to develop plans for acting even without decisive information about what course of action is best, and the harms associated with failing to act when warranted can often be more severe than the harms associated with taking actions that ultimately turn out to be unwarranted. When actions are taken without compelling information, however, it is crucial to reexamine regularly the state of the evidence and the consequences of the actions taken to reassess the best course of action. The adaptive management movement in environmental policy exemplifies this commitment to an ongoing process of learning and reevaluation. According to a National Research Council report on managing water resources, “Adaptive management promotes flexible decision-making that can be adjusted in the face of uncertainties as endpoints from management actions and other events become better understood” (NRC, 2004, pp. 1–2).

COMMITTEE’S CONSIDERATIONS IN DEVELOPING ITS PRINCIPLES

Among the committee’s considerations in developing its principles for decision making under substantial scientific uncertainty were input from contaminated communities and a number of previously published evidence-to-decision frameworks.

Community Input from Town Halls

During its three town halls, the committee obtained perspectives from people living in or working with communities with PFAS contamination during its three Town Halls (see Appendix B). The committee recognizes that the views expressed at the town halls do not necessarily represent the views of all affected communities or all individuals within those communities; nonetheless, they provided input for the committee to consider. Important themes relevant to the development of principles for decision making under substantial scientific uncertainty included the following:

  • Community members shared that they had to educate their clinicians about the exposures in their communities and the potential health effects that could result. They stated further that clinicians do not know how to provide advice on patient follow-up following PFAS exposure, whether PFAS exposure biomonitoring should be provided, or how to interpret PFAS exposure biomonitoring results if shared by the patient.
  • Community members expressed the view that PFAS exposure biomonitoring should be available through traditional clinical care to all individuals. In addition, given that PFAS are ubiquitous in the environment and that many of the contamination sites have not been identified, exposure biomonitoring should be more widely available.
  • Significant harms were not associated with exposure biomonitoring for PFAS. Community members understand that PFAS exposure biomonitoring may not directly inform their health, but they wish to have knowledge of their exposure levels as a first step.
  • Speakers stated that PFAS exposure information could inform health care decisions and patient follow-up related to potential adverse health effects associated with PFAS exposure.

Evidence-to-Decision Frameworks Considered by the Committee

Norris and colleagues recently conducted a review of evidence-to-decision frameworks to inform the development of a new framework for recommending interventions to prevent or mitigate the harmful effects of adverse environmental exposures (Norris et al., 2021). The authors reviewed 18 frameworks and compared the criteria they used to justify decisions. Those criteria included the priority of the problem and several considerations related to benefits and harms, including desirable effects, undesirable effects, the certainty of evidence regarding desirable and undesirable effects, and the balance of effects. Other criteria included values, resources, equity, acceptability, feasibility, and human rights. Briefly, the frameworks reviewed by Norris and colleagues are summarized in Table 2-1.

Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×

TABLE 2-1 Evidence-to-Decision Frameworks Reviewed by Norris and Colleagues (2021)

Priority Benefits Harms Certainty Values Resources Equity Acceptability Feasibility Human rights
ACP
BCPP
CalEPA Persticides
CalEPA AA
EVIDEM
GRADE
GCPS
ICER
ISPOR
NICE
Navigation Guide
SIGN
USPSTF
WHO
WHO-INTEGRATE

NOTES: ACP = Advisory Committee on Immunization Practices; BCPP = Breast Cancer Prevention Partners; CalEPA = California Environmental Protection Agency; CalEPA AA = California Environmental Protection Agency, Alternative Assessments; EVIDEM = Evidence and Value: Impact on DEcision-Making; GRADE = Grading of Recommendations Assessment, Development and Evaluations; GCPS = Guide to Community Preventive Services; ICER = Institute for Clinical and Economic Review; ISPOR = International Society for Pharmacoeconomics and Outcomes Research; NICE = United Kingdom National Institute for Health and Care Excellence; SIGN = Scottish Intercollegiate Guidelines Network; USPSTF = U.S. Preventive Services Task Force; WHO = World Health Organization.
SOURCE: Norris et al., 2021. Creative Commons CC BY.

GRADE (Grading of Recommendations Assessment, Development, and Evaluations)

GRADE has become the dominant evidence-to-decision framework over the past 15 years, including for clinical decision making and public health (Guyatt et al., 2008; Moberg et al., 2018; Norris et al., 2021). It includes evidence frameworks for several different decision contexts including clinical recommendations, individual perspective; clinical recommendations, population perspective; coverage decisions; and health system and public health recommendations or decisions. According to Norris and colleagues (2021), all applications of GRADE include roughly the same considerations:

  • Is the problem a priority?
  • How substantial are the desirable anticipated effects?
  • How substantial are the undesirable anticipated effects?
  • What is the overall certainty of the evidence of effects?
  • Does the balance between desirable and undesirable effects favor the intervention or the comparison?
  • Is there important uncertainty about, or variability in, how much people value the main outcomes?
  • How large are the resource requirements (costs)?
  • What is the certainty of the evidence of resource requirements (costs)?
  • Does the cost-effectiveness of the intervention (the out-of-pocket cost relative to the net desirable effect) favor the intervention or the comparison?

It is important to recognize that GRADE emphasizes financial considerations, equity, acceptability, and feasibility more for health systems and public health decisions than for individual decisions (Norris et al., 2021).

Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×

U.S. Preventive Services Task Force

The USPSTF is an independent panel of primary care, prevention, and evidence-based medicine experts. It uses an analytic framework (see Figure 2-1) to evaluate systematically the benefits and harms of a particular preventive service. In evaluating evidence concerning the benefits and harms of widespread implementation, both the evidence’s certainty and the magnitude of the benefits and harms are assessed. As shown in Table 2-2, the USPSTF assigns a letter grade to each preventive service signifying its recommendation about the service’s provision (Sawaya et al., 2007). It is important to note that the USPSTF does not consider cost when determining a letter grade.2 If evidence is not available, the USPSTF considers the “chain of indirect evidence, including evidence about the accuracy of screening tests, the efficiency and harms of early treatment, and the association between changes in intermediate endpoints due to treatment and changes in health endpoints” (Mabry-Hernandez et al., 2018). As Table 2-2 indicates, when the certainty, based on the evidence, for the net benefit of screening is rated low, the USPSTF considers the evidence insufficient to make a recommendation. The USPSTF views its evaluation of the benefits and harms of a preventive service as a process, and continually revisits and reevaluates based on new advances in research, testing, and treatment capabilities (Sawaya et al., 2007).

Image
FIGURE 2-1 U.S. Preventive Services Task Force’s generic analytic framework for a screening preventive service.
SOURCE: Mabry-Hernandez et al., 2018. CC BY-NC-ND 4.0.

___________________

2 See https://www.uspreventiveservicestaskforce.org/uspstf/about-uspstf/task-force-resources/uspstf-and-costconsiderations (accessed September 16, 2021).

Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×

TABLE 2-2 U.S. Preventive Services Task Force’s (USPSTF’s) Recommendation Grida

Certainty of Net Benefit Magnitude of Net Benefit
Substantial Moderate Small Zero/Negative
High A B C D
Moderate B B C D
Low Insufficient

a A, B, C, D, and Insufficient represent the letter grades of recommendation or statement of insufficient evidence assigned by the U.S. Preventive Services Task Force after assessing certainty and magnitude of the net benefit of the service.

SOURCE: Sawaya et al., 2007.

Evidence and Value: Impact on DEcision-Making (EVIDEM) Framework

Goetghebeur and colleagues (2008) developed this framework to facilitate health care decision making. They conducted an extensive review and analysis of the literature and determined that a framework for addressing the value judgments related to assessing health care interventions should be able to do the following: “1) disentangle intrinsic and extrinsic value components; 2) develop a simple and rigorous system that applies multiple criteria decision analysis (MCDA) from a pragmatic standpoint based on actual thought processes; 3) provide practical access to the evidence on which value judgments are based; and 4) provide a practical method for decision-makers to provide feedback to data producers and all other stakeholders” (Goetghebeur et al., 2008, p. 273). Based on these needs, the authors built the EVIDEM framework, tailored to provide a “comprehensive, transparent structure grounded in global standards and local needs” (Goetghebeur et al., 2008, p. 283). The framework aims to consider all perspectives, values, and rationales related to the decision at hand.

National Academies Evidence Framework for Genetic Testing

In 2017, a National Academies committee developed an evidence framework for genetic testing that calls for a clear definition of the genetic scenario being considered and a triage process for evaluating whether the purpose of a test is worthwhile and an expedited provisional decision can be made (see Figure 2-2). In addition, given the rapidly advancing nature of the field of genetic testing, the framework includes the opportunity for review and revisions (NASEM, 2017).

C-8 Medical Panel

In February 2005, the West Virginia Circuit Court approved a class action settlement agreement in a lawsuit resulting from contamination of drinking water with perfluorooctanoic acid (PFOA, also known as C-8) from DuPont’s Washington Works facility in Wood County, West Virginia (see Box 1-2 in Chapter 1). The agreement included establishing a science panel that conducted epidemiologic evaluations to determine probable link conditions associated with exposure to PFOA, and a separate medical panel that defined medical monitoring for members of the class (C-8 Medical Panel, 2013). The settlement agreement stated that the medical panel needed to consider the following factors in developing the medical monitoring protocol:

  1. Increased risk … that the Class Member has a significantly increased risk of contracting the particular diseases relative to the risk in the absence of exposure;
  2. Necessity of Diagnostic Testing … that the Class Member should undergo specific periodic diagnostic testing that would not be required in the absence of exposure to
Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×
  1. C-8. The Settlement further specified that the desires of Class Members for reassurance that they did not have a probable link condition was a sufficient rationale for testing, and factors such as financial cost and the frequency of testing need not be given significant weight in assessing the need for testing; and
  2. Existence of Monitoring Procedures … that testing procedures must exist that it is not necessary to show that detection and treatment in a pre-symptomatic state reduces the burden of the probable link condition. (C-8 Medical Panel, 2013, p. 1)
Image
FIGURE 2-2 Visual representation of the seven components of the evidence framework for genetic testing developed by the National Academies of Sciences, Engineering, and Medicine.
SOURCE: NASEM, 2017.

Belmont Report

The Belmont Report (HHS, 1979) was developed as a set of guidelines for protecting research participants. This seminal report on ethics and health care research outlines basic ethical principles and guidelines meant to assist in resolving ethical problems surrounding the conduct of research with human subjects. The three main principles are respect for persons, beneficence, and justice.

  • Respect for persons requires that people be considered autonomous. Autonomous individuals can consider their personal goals and make their own decisions. To respect persons is to give weight to their opinions and choices while refraining from obstructing their actions or choices unless detrimental to others. Conversely, a lack of respect for persons or autonomous agents is to refuse to accept or validate their judgments, deny them the freedom to act on their judgments, or withhold information necessary to make a judgment.
Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×
  • Beneficence is an obligation to respect individuals and their decisions and protect them from harm. Two general rules guide the committee’s understanding of beneficence: (1) do no harm, and (2) maximize possible benefits and minimize harms. The Hippocratic Oath requires physicians to make medical recommendations and decisions that benefit their patients “according to their best judgment.” The committee recognizes that zero harm or risk may not be possible, but medical decisions need to consider when the benefits outweigh the risks.
  • Justice refers to “fairness in distribution.” In the present context, it means that PFAS research should have a fair distribution such that a certain group of people is not selected for the research to the exclusion of others. An injustice occurs when some individuals benefit from the research or intervention or are denied that benefit without good, ethical reason.

Beauchamp and Childress (2001) have developed these principles further in the established textbook Principles of Biomedical Ethics to guide biomedical decision making beyond the context of research ethics. For example, these authors refer to respect for autonomy instead of respect for persons and divide the principle of beneficence into nonmaleficence (avoiding harm) and beneficence (providing benefits).

Precautionary Reasoning

An extensive body of scholarship in medical and environmental decision making is devoted to precautionary reasoning and the precautionary principle. This work can inform decision making under substantial scientific uncertainty. Resnik (2021) proposes four criteria for assessing the reasonableness of precautionary measures, as shown in Table 2-3.

CONCLUSION

Building on the work of other experts and evidence-to-decision frameworks, the committee developed five principles to guide decision making under uncertainty for use throughout this report: proportionality, justice, autonomy, feasibility, and adaptability. The principles of proportionality, justice, and autonomy build on the ethical principles outlined in the Belmont Report (HHS, 1979) and developed further by Beauchamp and Childress (2001). These principles affirm that decision making in response to PFAS should weigh plausible benefits and harms proportionally while treating all individuals fairly, promoting health equity, respecting human rights, and supporting informed decision making by individuals in accordance with their values. The principles of feasibility and adaptability affirm that decisions should be responsive to emerging information about harms, benefits, and other considerations.

TABLE 2-3 Criteria for Reasonableness of Precautionary Measures

Proportionality Reasonable measures balance plausible risks and possible benefits proportionally
Fairness Reasonable measures are based on a fair balancing of risks and benefits; fairness includes distributive and procedural fairness
Epistemic Responsibility Reasonable measures comply with norms for the responsible acquisition and utilization of evidence, knowledge, and expertise
Consistency Reasonable measures are based on a consistent rationale for decision making

SOURCE: Resnik, 2021.

Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×

In the clinical setting, the committee’s five principles converge under the principle of autonomy and shared decision making3 between patient and clinician. For this reason, shared decision making is likely to be the practical way a clinician can incorporate the principles into the clinical encounter.

The committee’s principles may provide a framework that ATSDR can use when writing clinical guidance for environmental exposures. Despite findings of associations between adverse health effects and chemical exposures, substantial scientific uncertainty exists about the causal role of environmental chemicals in many health outcomes. Regulatory agencies, public health agencies, and clinicians need to make thoughtful decisions about how to act in response to this uncertainty. The principles of proportionality, justice (particularly environmental justice), autonomy, feasibility, and adaptability can play a central role in these decisions.

REFERENCES

Barry, M. J., and S. Edgman-Levitan. 2012. Shared decision making—pinnacle of patient-centered care. New England Journal of Medicine 366(9):780–781. https://doi.org/10.1056/NEJMp1109283.

Beauchamp, T. L., and J. F. Childress. 2001. Principles of biomedical ethics. New York: Oxford University Press.

Blair, A., R. Saracci, P. Vineis, P. Cocco, F. Forastiere, P. Grandjean, M. Kogevinas, D. Kriebel, A. McMichael, N. Pearce, M. Porta, J. Samet, D. P. Sandler, A. S. Costantini, and H. Vainio. 2009. Epidemiology, public health, and the rhetoric of false positives. Environmental Health Perspectives 117(12):1809–1813. https://doi.org/10.1289/ehp.0901194.

Boffetta, P., J. K. McLaughlin, C. La Vecchia, R. E. Tarone, L. Lipworth, and W. J. Blot. 2008. False-positive results in cancer epidemiology: A plea for epistemological modesty. Journal of the National Cancer Institute 100(14):988–995. https://doi.org/10.1093/jnci/djn191.

Boronow, K. E., H. P. Susmann, K. Z. Gajos, R. A. Rudel, K. C. Arnold, P. Brown, R. Morello-Frosch, Laurie Havas, and J. Green Brody. 2017. DERBI: A digital method to help researchers offer “right-to-know” personal exposure results. Environmental Health Perspectives 125(2):A27–A33.

Brody, J. G., S. C. Dunagan, R. Morello-Frosch, P. Brown, S. Patton, and R. A. Rudel. 2014. Reporting individual results for biomonitoring and environmental exposures: Lessons learned from environmental communication case studies. Environmental Health 13:40. https://doi.org/10.1186/1476-069x-13-40.

Bullard, R. D. 2018. Dumping in Dixie: Race, class, and environmental quality. New York: Routledge.

C-8 Medical Panel. 2013. C-8 Medical Panel report. https://www.hpcbd.com/wp-content/uploads/sites/1603732/2021/01/Medical-Panel-Report-2013-05-24.pdf (accessed June 15, 2022).

Charles, C., A. Gafni, and T. Whelan. 1997. Shared decision-making in the medical encounter: What does it mean? (or it takes at least two to tango). Sociological Science in Medicine 44:681–692. https://doi.org.10.1016/s0277-9536(96)00221-3.

Corburn, J. 2005. Street science: Community knowledge and environmental health justice. Berkeley, CA: Department of City and Regional Planning, University of California, Berkeley. https://doi.org/10.5070/BP31911296.

Douglas, H. E. 2009. Science, policy, and the value-free ideal. Pittsburgh, PA: University of Pittsburgh Press.

___________________

3Barry and Edgman-Levitan (2012) builds on concepts in Charles (1997) and the Institute of Medicine’s (2001) Crossing the Quality Chasm: a New Health System for the 21st Century and describes shared decision making as the pinnacle of patient-centered care: “The process by which the optimal decision may be reached … is called shared decision-making and involves, at minimum, a clinician and the patient…. In shared decision-making, both parties share information: the clinician offers options and describes their risks and benefits, and the patient expresses his or her preferences and values. Each participant is thus armed with a better understanding of the relevant factors and shares responsibility in the decision about how to proceed” (p. 780).

Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×

Elliott, K. C. 2017. A tapestry of values: An introduction to values in science. New York: Oxford University Press.

Elliott, K. C., and T. Richards. 2017. Exploring inductive risk: Case studies of values in science. New York: Oxford University Press.

Fischhoff, B., and A. L. Davis. 2014. Communicating scientific uncertainty. Proceedings of the National Academy of Sciences of the United States of America 111(Suppl 4):13664–13671. https://doi.org/10.1073/pnas.1317504111.

Ginsberg, G. L., K. Pullen Fedinick, G. M. Solomon, K. C. Elliott, J. J. Vandenberg, S. Barone, Jr., and J. R. Bucher. 2019. New toxicology tools and the emerging paradigm shift in environmental health decision-making. Environmental Health Perspectives 127(12):125002.

Goetghebeur, M. M., M. Wagner, H. Khoury, R. J. Levitt, L. J. Erickson, and D. Rindress. 2008. Evidence and value: Impact on DEcisionMaking—the EVIDEM framework and potential applications. BMC Health Services Research 8(1):1–16.

Guyatt, G. H., A. D. Oxman, G. E. Vist, R. Kunz, Y. Falck-Ytter, P. Alonso-Coello, and H. J. Schünemann. 2008. GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. British Medicial Journal 336(7650):924–926. https://doi.org/10.1136/bmj.39489.470347.AD.

Harris, R. P., S. L. Sheridan, C. L. Lewis, C. Barclay, M. B. Vu, C. E. Kistler, C. E. Golin, J. T. DeFrank, and N. T. Brewer. 2014. The harms of screening: A proposed taxonomy and application to lung cancer screening. JAMA Internal Medicine 174(2):281–286.

HHS (U.S. Department of Health and Human Services). 1979. The Belmont report: Ethical principles and guidelines for the protection of human subjects of research. Washington, DC: U.S. Department of Health and Human Services.

IOM (Institute of Medicine). 2001. Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press. https://doi.org/10.17226/10027.

IOM. 2013. Environmental decisions in the face of uncertainty. Washington, DC: The National Academies Press. https://doi.org/10.17226/12568.

IOM, NAS, and NAE (Institute of Medicine, National Academy of Sciences, and National Academy of Engineering). 1995. On being a scientist: Responsible conduct in research (2nd ed.). Washington, DC: National Academy Press.

Krist, A. H., T. A. Wolff, D. E. Jonas, C. M. Mangione, C-W. Tseng, and D. Grossman. 2018. Update on the methods of the U.S. Preventive Services Task Force: Methods for understanding certainty and net benefit when making recommendations. American Journal of Preventive Medicine 54(Suppl 1): S11–S18.

London, A. J. 2022. For the common good: Philosophical foundations of research ethics. New York: Oxford University Press.

Mabry-Hernandez, I. R., S. J. Curry, W. R. Phillips, F. A. García, K. W. Davidson, J. W. Epling, Jr., Q. Ngo-Metzger, and A. S. Bierman. 2018. U.S. Preventive Services Task Force priorities for prevention research. American Journal of Preventive Medicine 54(Suppl 1):S95–S103. https://doi.org/10.1016/j.amepre.2017.08.014.

Moberg, J., A. D. Oxman, S. Rosenbaum, H. J. Schünemann, G. Guyatt, S. Flottorp, C. Glenton, S. Lewin, A. Morelli, and G. Rada. 2018. The GRADE Evidence to Decision (EtD) framework for health system and public health decisions. Health Research Policy and Systems 16(1):1–15.

Morello-Frosch, R., J. G. Brody, P. Brown, R. G. Altman, R. A. Rudel, and C. Perez. 2009. Toxic ignorance and right-to-know in biomonitoring results communication: A survey of scientists and study participants. Environmental Health 8:6. https://doi.org/10.1186/1476-069X-8-6.

NASEM (National Academies of Sciences, Engineering, and Medicine). 2017. An evidence framework for genetic testing. Washington, DC: The National Academies Press.

Norris, S. L., M. T. Aung, N. Chartres, and T. J. Woodruff. 2021. Evidence-to-decision frameworks: A review and analysis to inform decision-making for environmental health interventions. Environmental Health 20(1):124. https://doi.org/10.1186/s12940-021-00794-z.

Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×

NRC (National Research Council). 2004. Adaptive management for water resources project planning. Washington, DC: The National Academies Press. https://doi.org/10.17226/10972.

Rehfuess, E. A., J. M. Stratil, I. B. Scheel, A. Portela, S. L. Norris, and R. Baltussen. 2019. The WHO-INTEGRATE evidence-to-decision framework version 1.0: Integrating WHO norms and values and a complexity perspective. BMJ Global Health 4(Suppl 1):e000844.

Resnik, D. B. 2021. Precautionary reasoning and the precautionary principle. In Precautionary reasoning in environmental and public health policy. Cham, Switzerland: Springer. Pp. 111–128.

Sawaya, G. F., J. Guirguis-Blake, M. LeFevre, R. Harris, and D. Petitti. 2007. Update on the methods of the U.S. Preventive Services Task Force: Estimating certainty and magnitude of net benefit. Annals of Internal Medicine 147(12):871–875. https://doi.org/10.7326/0003-4819-147-12200712180-00007.

Shrader-Frechette, K. 2002. Environmental justice: Creating equality, reclaiming democracy. New York: Oxford University Press.

Wandersman, A. 2003. Community science: Bridging the gap between science and practice with community-centered models. American Journal of Community Psychology 31(3–4):227–242. https://doi.org/10.1023/A:1023954503247.

Workman, M., K. Dooley, G. Lomax, J. Maltby, and G. Darch. 2020. Decision-making in contexts of deep uncertainty—An alternative approach for long-term climate policy. Environmental Science & Policy 103:77–84.

Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×
Page42
Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×
Page43
Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×
Page44
Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×
Page45
Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×
Page46
Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×
Page47
Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×
Page48
Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×
Page49
Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×
Page50
Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×
Page51
Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×
Page52
Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×
Page53
Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×
Page54
Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×
Page55
Suggested Citation:"2 Principles for Decision Making Under Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up. Washington, DC: The National Academies Press. doi: 10.17226/26156.
×
Page56
Next: 3 Potential Health Effects of PFAS »
Guidance on PFAS Exposure, Testing, and Clinical Follow-Up Get This Book
×
Buy Paperback | $45.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

In thousands of communities across the United States, drinking water is contaminated with chemicals known as perfluoroalkyl and polyfluoroalkyl substances (PFAS). PFAS are used in a wide range of products, such as non-stick cookware, water and stain repellent fabrics, and fire-fighting foam, because they have properties that repel oil and water, reduce friction, and resist temperature changes. PFAS can leak into the environment where they are made, used, disposed of, or spilled. PFAS exposure has been linked to a number of adverse health effects including certain cancers, thyroid dysfunction, changes in cholesterol, and small reductions in birth weight.

This report recommends that the Centers for Disease Control and Prevention (CDC) update its clinical guidance to advise clinicians to offer PFAS blood testing to patients who are likely to have a history of elevated exposure, such as those with occupational exposures or those who live in areas known to be contaminated. If testing reveals PFAS levels associated with an increased risk of adverse effects, patients should receive regular screenings and monitoring for these and other health impacts. Guidance on PFAS Exposure, Testing, and Clinical Follow-Up recommends that the CDC, Agency for Toxic Substances and Disease Registry (ATSDR), and public health departments support clinicians by creating educational materials on PFAS exposure, potential health effects, the limitations of testing, and the benefits and harms of testing.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!