National Academies Press: OpenBook
« Previous: Summary
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×

1

Introduction

Conducting biomedical research involving human participants often entails the generation of laboratory test results associated with individual research participants—results that in the past have not been routinely shared with the individuals participating in the research. In recent years, however, that has begun to change. The research enterprise has begun to take steps to become more participant-centric, acknowledging the importance of increasing the engagement of and transparency with research participants across all phases of research. And one particular aspect of the relationship between investigators and participants that increases transparency and engagement is the return of individual research results to research participants (Ohayon et al., 2017). Engaging participants more broadly in research has been shown to improve participants’ trust in the research enterprise and to encourage individuals to participate in future research (CTSA Community Engagement Key Function Committee Task Force on the Principles of Community Engagement, 2011; Domecq et al., 2014; Holzer et al., 2014). Thus, as part of the broader movement to make the research enterprise more participant-centric, there has been an increasing push to return individual research results to participants. This push is the product not only of transformation in the research enterprise but also of the changing expectations of the research participants themselves. There is a growing demand by research participants to gain access to their individual results—a demand that is driven not just by the potential benefit that individuals could gain by learning about clinically actionable information, but also by participants’ desire to learn about themselves from information that they would not otherwise obtain (Facio et al., 2013; Sanderson et al., 2016).

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×

However, the return of individual research results generated in research laboratories presents a number of challenges. Research by its very nature often produces results that are of uncertain value and, depending on the stage of research, may not be analytically or clinically valid.1 One overarching challenge is determining how to weigh the potential benefits and harms of returning results which may not be accurate or have clear meaning. Additionally, many research laboratories do not currently have the personnel or quality procedures in place to ensure result validity (Ambulos, 2013) or have the requisite knowledge and experience to effectively return individual research results (Rigby and Fernandez, 2005; Thorogood et al., 2014). Complicating matters is the fact that current interpretations of two federal regulations established to protect individuals’ health and health information—the Clinical Laboratory Improvement Amendments of 1988 (CLIA) and the Health Insurance Portability and Accountability Act of 1996 (HIPAA)—conflict in their requirements for and limitations on the return of test results to individuals.

Over the last several decades, consensus has been growing in certain research domains that some types of research results or secondary findings2—specifically, those that are clinically actionable and valid—can be and, when certain conditions apply (e.g., immediate clinical action is warranted), should be returned to participants (see the section “Past Expert Group Recommendations on the Return of Individual Research Results”). As a result, some research projects have begun to offer research results as part of their study plan (Brody et al., 2014; Fullerton et al., 2012; Jarvik et al., 2014; Kullo et al., 2014). Still, a number of questions remain, and these have generated considerable debate among investigators, research participants, research sponsors, and legal scholars. These questions include

  • Under what circumstances is it appropriate for investigators to offer individual research results to participants? Are there circumstances under which the return of individual-specific results should be obligated, encouraged, or discouraged?
  • Do participants have a right (either ethically or legally) to receive any or all of their results if they request them from the investigator?
  • If individual research results are returned, what should the expected standards be for investigators to adequately communicate the meaning and level of confidence or uncertainty in their results to participants? Do best practices for the return of individual research results exist as a guide for investigators?

___________________

1Analytic validity indicates how well a test measures the property or characteristic it was intended to measure, whereas clinical validity is a measure of how consistently and accurately a test detects or predicts the intermediate or final outcomes of interest (IOM, 2012).

2Secondary findings are results that are not the primary objective of the research (PCSBI, 2013). Such findings are referred to in the literature by a variety of terms, such as “additional,” “secondary,” “incidental,” “ancillary,” “supplemental,” etc., and these terms can be combined with additional clarifiers such as “unanticipated” and “anticipated” (Tan et al., 2017).

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
  • Are the current regulatory requirements adequate to address returning individual research results in an appropriate manner and, if not, what new, revised, or alternative policies or regulatory requirements might better address the appropriate return of individual research results?

To address these issues, the Centers for Medicare & Medicaid Services (CMS), the Food and Drug Administration (FDA), and the National Institutes of Health (NIH) requested that the National Academies of Sciences, Engineering, and Medicine (the National Academies) convene a committee to consider whether and under what circumstances individual research results generated in research laboratories ought to be returned to study participants, considering participant preferences and investigator obligations, current practices, and the available evidence on potential benefits and harms as well as the regulatory environment for returning individual research results to participants. The full Statement of Task for the committee is presented in Box 1-1. This report presents the findings, conclusions, and recommendations of the National Academies committee empaneled to respond to this request.

STUDY SCOPE AND KEY TERMINOLOGY

The topic of the return of research results is exceptionally broad in scope and encompasses all fields of human research, including biomedical, psychological, and behavioral research. During the committee’s first meeting on July 19, 2017, its members had an opportunity to clarify the scope of the study with representatives of the three sponsoring federal agencies, each of whom presented the charge to the committee and took part in a subsequent question-and-answer period. In the course of that discussion, the study sponsors clarified that the committee was intended to focus on research results that are generated from the analysis of human biospecimens, i.e., samples of material collected from the human body, such as urine, blood, tissue, cells, and protein (NCI, 2018). The committee was not to consider the return of results from imaging, behavioral, or cognitive tests, for example. However, the committee acknowledged that the recommendations in this report may have broader impact beyond their application to results generated from biospecimens.

Of note, the committee’s charge was not limited to the return of genetic test results, as many other kinds of research are performed on human biospecimens. Such research may include, for example, basic science studies using tumor biopsies to identify a new biomarker for colon cancer, clinical trials that evaluate blood samples for antibody levels induced by a new malaria vaccine, and epidemiological studies measuring the level of a suspected toxin in urine samples for an environmental exposure study; all of these types of research involve laboratory tests on human biospecimens and are in the scope of this report.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×

In recent years, the topic of the return of individual research results has generated immense interest and debate among bioethicists and scientists, particularly in the fields of genetics (Fabsitz et al., 2010; Green et al., 2013; Holm et al., 2014; Jarvik et al., 2014) and environmental exposure research (Brody et al., 2007; Haines et al., 2011; Haynes et al., 2016; Sly et al., 2009). In the genetics context, much of the debate has been focused on the return of clinically actionable secondary findings—results that are not the primary objective of the research (PCSBI, 2013).3 This is an important issue in the broader context of returning information generated in the course of research to individual participants, but the sponsors clarified that it was not intended to be a central focus of this committee’s report. Instead, in this report the committee uses the term “individual research results” to refer to results that are generated in the course of a study to help answer the research question or otherwise support the study objectives (e.g., to determine clinical trial inclusion/exclusion) and are specific to one participant

___________________

3 It is important to note that for some types of studies with narrow and clearly defined targets, it is very easy to distinguish individual research results from secondary findings, but for other studies (e.g., hypothesis-generating research) that have no primary target, there is no clear dividing line.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×

(PCSBI, 2013). There was, however, a recognition that secondary findings remain an important part of the discussion, given that many sequencing and other “omics” research studies have no primary target. Moreover, it should be noted that the issue of returning secondary findings has a long history (e.g., in the context of returning results from imaging tests), and the committee recognized that the lessons learned from those experiences might be relevant to the committee’s task. Distinctions can also be made between different types of individual research results according to the kind of information provided—i.e., uninterpreted versus interpreted findings. In the genetics context, there is also an ongoing discussion about the return of sequencing information which is generally referred to as raw data. For the purposes of this report, all these types of information are included in the term “individual research results.” Chapter 5 discusses ways to facilitate the understanding of different types of individual research results.

In addressing its charge, the committee considered three general scenarios in which consideration of the return of individual research results is relevant:

  1. the planned offer of anticipated individual research results by investigators to participants,
  2. the return of individual research results upon the request of participants, and
  3. the offer of unanticipated individual research results to participants.

For the purposes of this report, anticipated results are those results that are actively sought or are expected to arise when using a particular research test on human biospecimens. This includes results that are not the primary objective of the test or study. Unanticipated results are those that are unexpected either because they could not have been anticipated given the current state of scientific knowledge or because the research team did not consider the potential to generate them using a particular research test. In designing a study, investigators can anticipate several types of results and possible outcomes that may arise from the tests and analyses used over the course of investigation, and very few results should be unanticipated. However, despite investigators’ ability to predict the possible outcomes of their research, unanticipated results cannot be entirely avoided as the state of the science may change over the course of a study or a participant may have an unknown or undiagnosed condition that becomes apparent over the course of biospecimen analysis, thus generating unforeseen results.

Frequently, the considerations that stakeholders will need to take into account when deciding on the return of individual-specific research results to research participants will be the same for all of the three scenarios described above. Therefore, throughout the report, the committee uses the shorthand phrase “return of results” to refer to the practice of returning individual research results in any of the three scenarios described above when it is not important to make a distinction among them.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×

During the discussion of the charge at the committee’s first meeting, the following additional areas were identified by the study sponsors as falling outside the study scope, although it should be noted that only some of these are explicitly excluded in the Statement of Task:

  • Specific assays or test results (i.e., the committee was not asked to generate a list, for example, of specific genes associated with disease susceptibility that, when tested for, should or should not be returned);
  • The return of aggregate research data or study-level results;
  • The return of results from anonymized or de-identified specimens that investigators cannot link back to the contributing participant, as well as the role or obligation of biobanks that retain identifiers that would enable the return of individual research results generated by investigators using de-identified biobank specimens (e.g., for secondary research);
  • The infrastructure and policies needed for the implementation of a system to return results from secondary research; and
  • Laboratory developed tests (LDTs) and the associated LDT regulations.

In discussions with the sponsors, the committee also clarified the scope as it applies to CLIA. The sponsors indicated to the committee that it would be appropriate to include in its description of the current regulatory environment for the return of individual research results CMS’s current interpretation of the scope and applicability of CLIA, which is that “only those facilities performing research testing on human biospecimens that do not report patient-specific results may qualify to be excepted from CLIA certification” (CMS, 2014). Although CMS’s current interpretation has been questioned by some legal scholars (Burke et al., 2014; Evans, 2014; Prince et al., 2015), the committee was advised that making any comments, analysis, or conclusions regarding the appropriateness of that interpretation would be beyond what was intended in the Statement of Task. Furthermore, the committee was asked not to make recommendations to Congress regarding changes to the CLIA law. However, recommendations on changes to the CLIA regulations were within the study scope if the committee felt that such changes were needed to better align the regulatory environment with the risks and benefits of the return of research results. Chapter 6 addresses the committee’s recommendations on clarifying and revising federal regulations.

STUDY CONTEXT

A number of societal drivers have brought the issue of returning individual research results to participants to the forefront of public debate. First, participants have expressed the desire to receive their results, and in the growing movement toward participant-centered research, research sponsors and investigators are increasingly listening to them (Brody et al., 2014; Dunagan et al., 2013; Jarvik et al.,

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×

2014; Ohayon et al., 2017). Most notably, NIH made a commitment to return research results to participants enrolled in the All of Us Research Program (discussed in Box 1-3). Second, expert groups have come to some consensus concerning the general principles for returning certain types of results, although investigators, sponsors, and institutional review boards (IRBs) are given little concrete guidance on how to weigh the benefits and risks of returning research results that may not be accurate, clinically actionable, or have clear meaning, and, more broadly, there has been insufficient exploration of the benefits, burdens, and costs to the research enterprise of returning results. Third, the conflicting CLIA and HIPAA regulations regarding the circumstances under which laboratories are obligated to return research results—or prohibited from returning them—upon a participant’s request have created confusion among investigators and their institutions, resulting in calls for guidance from federal agencies on how to apply the law. The sections below further explore these drivers and the impetus for this study.

The Evolution of the Participant Role in Research and Implications for Return of Results

The issue of returning individual-specific research results is one facet of a growing movement to engage research participants more substantively in the design and conduct of biomedical research (see Box 1-2). The roles of research participants, patients, family members, advocacy organizations, and community representatives have undergone a significant transformation in the past few decades as representatives of these groups have voiced their desire to be engaged in all stages of the research process, from planning and design to execution, interpretation, and dissemination. The increased focus by the research enterprise on engagement has been in part motivated by the recognition that health is socially determined and that addressing the health needs of diverse communities requires the involvement of those most affected, enabling them to be the beneficiaries of the research.

In 21st-century research, patient groups have catalyzed a growing array of sophisticated, innovative initiatives, including launching patient registries and conducting natural history studies, funding translational and early phase clinical research programs, designing trials, developing novel trial infrastructure, using venture philanthropy to drive therapy development, and conducting policy advocacy aimed at the evolving regulatory environment (CTTI, 2015). Additionally, participant-centric technologies and digital health applications have changed the way consumers interact with their own data, creating a new system in which data are being both generated and controlled by participants. As a result, recent years have seen the establishment of a number of initiatives that are moving the research enterprise to a more participatory model of research. The Patient-Centered Outcomes Research Institute, for example, requires that investigators engage stakeholders representing the population of interest in the

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×

research it funds to ensure that studies address questions that are important to patients and other stakeholders and that the outcomes that are measured are those patients and other stakeholders find meaningful (Frank et al., 2015). In this new paradigm of participant-centered and community-partnered research, community members, patients, and their advocates have a seat at the table (for example, through membership on community advisory boards) and are increasingly included in the development of research protocols.

This increasingly common role of participants as research collaborators and partners, combined with the ethical principle of respect for participants, has led to calls for general study results (i.e., aggregate results) and lay summaries of the research to be shared with participants, regardless of whether the study conclusions are negative or the study is ultimately published. This practice, which may also bolster engagement and trust in the research enterprise (Beskow et al., 2012), has been endorsed and promoted by NIH, FDA, and other federal and nonfederal research sponsors, although its adoption is still in the early stages, and it has not yet become routine practice (Federal Register, 2016; IOM, 2015). Arguments similar to the ones for returning general study results to study participants have also

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×

been made for returning individual-specific research results. Furthermore, some advocates have argued that once participants have access to their results, they will be able to pool and share their data to help advance the science and guide more participant-centered research (Open Humans, 2018). Current literature shows support from participants and investigators for the return of results to individuals (as discussed further in Chapter 2), and, notably, maximizing participants’ access to information about themselves was adopted as a key goal of the Precision Medicine Initiative (NIH, 2018) (see Box 1-3).

Past Expert Group Recommendations on the Return of Individual Research Results

The question of whether and when to return individual-specific research results has been considered by several expert groups, and a number of recommendations and position statements have been released supporting the return of results and secondary findings under specific conditions (Bookman et al., 2006; Fabsitz et al., 2010; Green et al., 2013; Jarvik et al., 2014; MRCT Center, 2017; National Bioethics Advisory Commission, 1999; PCSBI, 2013; SACHRP, 2016; Wolf et al., 2008). If one examines the publications, certain themes emerge:

  • research participants have a right to refuse results that are offered to them;
  • the result, whether a research test result or a secondary finding, should be analytically and clinically valid if it is to be returned;
  • the result should be important to the individual’s health, although there is not universal agreement on whether results relevant to reproductive decisions should be returned;
  • the result should be “actionable,” in that a meaningful intervention is available that can prevent or at least ameliorate the disease course to an extent that would not likely otherwise occur;
  • investigators do not have a duty to use limited research funds to hunt for actionable results, such as genomic findings (Berg et al., 2013; Green et al., 2013; Kalia et al., 2016);4
  • IRBs should require investigators’ study proposals to include documentation of whether and how individual research results will be returned; and
  • the obligations of investigators to return results generally ends with the completion of the research project.

___________________

4 In contrast, the American College of Medical Genetics and Genomics has recommended that all clinical laboratories that conduct genetic sequencing should seek out and report pathogenic mutations for 56 specified genes (Green et al., 2013).

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×

Despite this general level of consensus, there is little evidence that sponsors, investigators, IRBs, or research institutions have taken active measures to routinely promote the return of individual research results. Beyond these general principles, more specific guidance is needed on how stakeholders should consider the benefits and risks of returning individual research results to participants in different circumstances and also on the infrastructure, expertise, and resources that would be needed to enable this endeavor.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×

The Current Legal and Regulatory Environment

Historically, the federal government has clearly separated regulations related to clinical care from those related to research. FDA, for example, regulates tests used for clinical purposes, but it largely delegates decisions on tests used for research purposes to IRBs, except for FDA oversight of investigational new drug and investigational device exemption research. However, the increasing focus on translational research and the emergence of the learning health system model (IOM, 2013) have highlighted how, in some contexts, the traditional distinctions between clinical and research activities break down.

The growing practice of returning individual-specific research results exemplifies the increasing interconnectedness of research and clinical care (Wolf et al., 2018). With the rise of next-generation or massively parallel sequencing, which allows the sequencing of most of an individual genome, along with other “omics” methods, biomedical investigators are increasingly finding themselves in positions where the analysis of biospecimens using cutting-edge methods and techniques—which may or may not yet be validated for clinical use—generates results with potential clinical significance, leaving some investigators with the desire or sense of obligation to share those results with the research participant. This poses various challenges, particularly in the current regulatory environment, which has been slow to adapt to the growing interconnectedness of clinical care and research, and this in turn has led to calls for modernizing the regulations to better support this interconnected world.

CLIA was put in place to help protect patients by requiring that any laboratory in the United States that performs tests for the purpose of providing information for the diagnosis, prevention, or treatment of a disease or for the assessment of the health of an individual be CLIA-certified through an accreditation process that is designed to ensure that certain quality-control assurances and standards are used by the testing laboratory. CLIA does not apply to laboratories that conduct tests on human biospecimens for research purposes (e.g., academic and industry laboratories) and that do not report patient-specific results. However, if laboratories do report patient-specific results, CMS (the agency that administers CLIA) has interpreted the regulations to mean that those laboratories must be CLIA certified, even if they otherwise only perform research functions. This creates a dilemma when clinically actionable research results, particularly those that are urgent and might not otherwise be discovered, are generated in research laboratories that are not CLIA certified (Burke et al., 2014; Dressler et al., 2012).

Moreover, HIPAA regulations give patients full right of access to their medical information in the designated record set maintained by any HIPAA-covered entity.5 The designated record set is defined as the group of records maintained by

___________________

5 HIPAA-covered entities include health plans, health care clearinghouses, and health care providers who transmit health information in electronic form in connection with covered financial or administrative transactions (e.g., billing transactions). HIPAA also extends to business associates of covered entities (45 C.F.R. § 160.103).

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×

or for an institution or other entity covered by HIPAA, including medical records, billing records, health plan enrollment payment, and claims adjudication, which are used to make decisions about individuals (45 C.F.R. § 164.501). This definition does not clarify whether research results are or are not part of the DRS.6 Prior to 2014, laboratories had a specific exception to this rule, which allowed them to refuse to provide test result information. However, a 2014 amendment to the regulation removed this exception. Consequently, laboratories that are not CLIA certified but are covered under HIPAA may be in the position of, on the one hand, being required under HIPAA to provide patient-specific results upon request and, on the other hand, being barred by CLIA from doing so (Barnes et al., 2015). Additional legal and regulatory challenges associated with the return of individual-specific research results are discussed further in later chapters, but the moral dilemma faced by investigators and the CLIA/HIPAA conflict highlighted here demonstrate the need for clarifying guidance from federal agencies on how investigators and research laboratories should proceed—as well as providing a critical analysis as to whether current regulations appropriately address the risks and benefits of participants having access to their research results.

STUDY APPROACH

To respond to its charge, the National Academies convened a 15-member committee composed of individuals with expertise in bioethics, legal and regulatory practice, research and laboratory practice, health communication, health literacy, decision science, and patient and community advocacy. Biographies of the committee members can be found in Appendix E.

The committee deliberated from July 2017 to May 2018. During this time period, the committee met in person five times (July, September, October, and December 2017, and February 2018), and the first four meetings included information-gathering sessions that were open to the public. A 2-day workshop was conducted in conjunction with the September 2017 meeting and covered participant and community perspectives on the return of individual research results, the perspectives of investigators and institutions, applicable laws and regulations, the institutional infrastructure and oversight needed to enable the appropriate return of results, and communication practices used in returning results. Additional perspectives from investigators and from participants were sought during the public sessions held during the October and December meetings, respectively. In addition, the committee held a Web-based meeting in December 2017 to further explore laboratory standards for regulated and non-regulated biomedical research laboratories. Open session agendas for all meetings are provided in Appendix A.

Members of the public were given opportunities to comment on the committee’s task at the September and October in-person meetings. The committee

___________________

6 45 C.F.R. § 164.501—Definitions.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×

also proactively solicited written comments from stakeholders and the public to ensure that a diverse sample of perspectives was captured and considered in its deliberations.

Throughout the study process, the committee reviewed publicly available peer-reviewed and gray literature to inform its findings, conclusions, and recommendations. The committee also drew on two commissioned papers (see Appendixes C and D) to obtain additional background information and supporting evidence. One paper described the legal and regulatory landscape relevant to the return of individual research results, and the second provided a critical analysis of the ethical principles commonly used to justify disclosure or non-disclosure as well as of the philosophic literature on the relationship between the research participant and the investigator or research institution and the implications of that analysis for the return of results.

Additional detailed information on the committee’s methodology, including its literature search strategies and processes for the solicitation of public comments, is provided in Appendix A.

ORGANIZATION OF THE REPORT

This report is organized into six chapters that collectively describe a path forward for the responsible return of individual research results. Following this introduction, Chapter 2 presents the relevant ethical principles and the societal considerations for the return of individual research results, including the potential risks and benefits to individuals and the research enterprise. The chapter ends by listing the committee’s guiding principles. Chapter 3 describes the quality management system and infrastructure needed to ensure the quality, reproducibility, and validity of test results produced in research laboratories. Chapters 4 and 5 address the “how” of returning individual research results appropriately. Chapter 4 provides a framework for weighing the competing considerations in study-specific decisions on whether and when individual research results should be returned and also describes the advance planning needed to minimize risks and maximize benefits. Chapter 5 describes best practices for communicating with participants in order to set appropriate expectations and to effectively deliver results in such a way that their meaning and limitations can be understood. Finally, Chapter 6 describes the changes to the regulatory landscape that will be needed to achieve the vision articulated by the committee in this report.

REFERENCES

Ambulos, N. P. 2013. The good, the bad and the ugly: Getting CLIA-certification for a basic research lab. Journal of Biomolecular Techniques 24(Suppl):S21.

American Psychological Association. 1973. Ethical principles in the conduct of research with human participants. American Psychologist 28(1):79–80.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×

Barnes, M., S. Stayin, D. Forster, M. Russell-Einhorn, D. Peloquin, and A. Medina-Jordan. 2015. The CLIA/HIPAA conundrum of returning test results to research participants. Medical Research Law & Policy Report, July 15. https://www.ropesgray.com/~/media/Files/articles/2015/July/2015-07-15-Bloomberg-BNA.ashx (accessed May 18, 2018).

Berg, J. S., L. M. Amendola, C. Eng, E. Van Allen, S. W. Gray, N. Wagle, H. L. Rehm, E. T. DeChene, M. C. Dulik, F. M. Hisama, W. Burke, N. B. Spinner, L. Garraway, R. C. Green, S. Plon, J. P. Evans, and G. P. Jarvik. 2013. Processes and preliminary outputs for identification of actionable genes as incidental findings in genomic sequence data in the Clinical Sequencing Exploratory Research consortium. Genetics in Medicine 15(11):860–867.

Beskow, L. M., W. Burke, S. M. Fullerton, and R. R. Sharp. 2012. Offering aggregate results to participants in genomic research: Opportunities and challenges. Genetics in Medicine 14(4):490–496.

Bookman, E. B., A. A. Langehorne, J. H. Eckfeldt, K. C. Glass, G. P. Jarvik, M. Klag, G. Koski, A. Motulsky, B. Wilfond, T. A. Manolio, R. R. Fabsitz, R. V. Luepker, and N. W. Group. 2006. Reporting genetic results in research studies: Summary and recommendations of an NHLBI working group. American Journal of Medical Genetics, Part A 140(10):1033–1040.

Boynton, P. M. 1998. People should participate in, not be subjects of, research. British Medical Journal 317(7171):1521.

Bresnick, J. 2015. NIH unveils precision medicine, genomics, big data analytics plan. Health IT Analytics, September 21. https://healthitanalytics.com/news/nih-unveils-precision-medicine-genomics-big-data-analytics-plan (accessed May 18, 2018).

Brody, J. G., R. Morello-Frosch, P. Brown, R. A. Rudel, R. G. Altman, M. Frye, C. A. Osimo, C. Pérez, and L. M. Seryak. 2007. “Is it safe?”: New ethics for reporting personal exposures to environmental chemicals. American Journal of Public Health 97(9):1547–1554.

Brody, J. G., S. C. Dunagan, R. Morello-Frosch, P. Brown, S. Patton, and R. A. Rudel. 2014. Reporting individual results for biomonitoring and environmental exposures: Lessons learned from environmental communication case studies. Environmental Health 13:40.

Bromley, E., L. Mikesell, F. Jones, and D. Khodyakov. 2015. From subject to participant: Ethics and the evolving role of community in health research. American Journal of Public Health 105(5):900–908.

Burke, W., B. J. Evans, and G. P. Jarvik. 2014. Return of results: Ethical and legal distinctions between research and clinical care. American Journal of Medical Genetics 166C(1):105–111.

CMS (Centers for Medicare & Medicaid Services). 2014. Research testing and Clinical Laboratory Improvement Amendments of 1988 (CLIA) regulations.https://www.cms.gov/Regulations-and-Guidance/Legislation/CLIA/Downloads/Research-Testing-and-CLIA.pdf (accessed May 18, 2018).

Colaianni, A., S. Chandrasekharan, and R. Cook-Deegan. 2010. Impact of gene patents and licensing practices on access to genetic testing and carrier screening for Tay-Sachs and Canavan disease. Genetics in Medicine 12(4 Suppl):S5–S14.

Corrigan, O., and R. Tutton. 2006. What’s in a name? Subjects, volunteers, participants, and activists in clinical research. Clinical Ethics 1(2):101–104.

CTSA (Clinical and Translational Science Awards) Community Engagement Key Function Committee Task Force on the Principles of Community Engagement. 2011. Principles of community engagement. Bethesda, MD: NIH.

CTTI (Clinical Trials Transformation Initiative). 2015. CTTI recommendations on effective engagement with patient groups around clinical trials.https://www.ctti-clinicaltrials.org/briefing-room/recommendations/ctti-recommendations-effective-engagement-patient-groups-around (accessed May 18, 2018).

Domecq, J. P., G. Prutsky, T. Elraiyah, Z. Wang, M. Nabhan, N. Shippee, J. P. Brito, K. Boehmer, R. Hasan, B. Firwana, P. Erwin, D. Eton, J. Sloan, V. Montori, N. Asi, A. M. A. Dabrh, and M. H. Murad. 2014. Patient engagement in research: A systematic review. BMC Health Services Research 14(89):1–9.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×

Dressler, L. G., S. Smolek, R. Ponsaran, J. M. Markey, H. Starks, N. Gerson, S. Lewis, N. Press, E. Juengst, G. L. Wiesner, and G. Consortium. 2012. IRB perspectives on the return of individual results from genomic research. Genetics in Medicine 14(2):215–222.

Dunagan, S. C., J. G. Brody, R. Morello-Frosch, P. Brown, S. Goho, J. Tovar, S. Patton, and R. Danford. 2013. When pollution is personal: Handbook for reporting results to participants in biomonitoring and personal exposure studies. Newton, MA: Silent Spring Institute.

Evans, B. J. 2014. The First Amendment right to speak about the human genome. Journal of Constitutional Law 16(3):549–646.

Fabsitz, R. R., A. McGuire, R. R. Sharp, M. Puggal, L. M. Beskow, L. G. Biesecker, E. Bookman, W. Burke, E. G. Burchard, G. Church, E. W. Clayton, J. H. Eckfeldt, C. V. Fernandez, R. Fisher, S. M. Fullerton, S. Gabriel, F. Gachupin, C. James, G. P. Jarvik, R. Kittles, J. R. Leib, C. O’Donnell, P. P. O’Rourke, L. L. Rodriguez, S. D. Schully, A. R. Shuldiner, R. K. Sze, J. V. Thakuria, S. M. Wolf, G. L. Burke, and National Heart, Lung, and Blood Institute Working Group. 2010. Ethical and practical guidelines for reporting genetic research results to study participants: Updated guidelines from a National Heart, Lung, and Blood Institute Working Group. Circulation: Cardiovascular Genetics 3(6):574–580.

Facio, F. M., H. Eidem, T. Fisher, S. Brooks, A. Linn, K. A. Kaphingst, L. G. Biesecker, and B. B. Biesecker. 2013. Intentions to receive individual results from whole-genome sequencing among participants in the ClinSeq study. European Journal of Human Genetics 21(3):261–265.

Federal Register. 2016. Clinical trials registration and results information submission. Federal Register 81(183):64982–65157.

Frank, L., L. Forsythe, L. Ellis, S. Schrandt, S. Sheridan, J. Gerson, K. Konopka, and S. Daugherty. 2015. Conceptual and practical foundations of patient engagement in research at the Patient-Centered Outcomes Research Institute. Quality of Life Research 24(5):1033–1041.

Fullerton, S. M., W. A. Wolf, K. B. Brothers, E. W. Clayton, D. C. Crawford, J. C. Denny, P. Greenland, B. A. Koenig, K. A. Leppig, N. M. Lindor, C. A. McCarty, A. L. McGuire, E. R. McPeek Hinz, D. B. Mirel, E. M. Ramos, M. D. Ritchie, M. E. Smith, C. J. Waudby, W. Burke, and G. P. Jarvik. 2012. Return of individual research results from genome-wide association studies: Experience of the Electronic Medical Records and Genomics (eMERGE) network. Genetics in Medicine 14(4):424–431.

Genetics Home Reference. 2018. What is the Precision Medicine Initiative?https://ghr.nlm.nih.gov/primer/precisionmedicine/initiative (accessed May 19, 2018).

Green, R. C., J. S. Berg, W. W. Grody, S. S. Kalia, B. R. Korf, C. L. Martin, A. L. McGuire, R. L. Nussbaum, J. M. O’Daniel, K. E. Ormond, H. L. Rehm, M. S. Watson, M. S. Williams, and L. G. Biesecker. 2013. ACMG recommendations for reporting of incidental findings in clinical exome and genome sequencing. Genetics in Medicine 15(7):565–574.

Haines, D. A., T. E. Arbuckle, E. Lye, M. Legrand, M. Fisher, R. Langlois, and W. Fraser. 2011. Reporting results of human biomonitoring of environmental chemicals to study participants: A comparison of approaches followed in two Canadian studies. Journal of Epidemiology and Community Health 65(3):191–198.

Haynes, E. N., S. Elam, R. Burns, A. Spencer, E. Yancey, P. Kuhnell, J. Alden, M. Walton, V. Reynolds, N. Newman, R. O. Wright, P. J. Parsons, M. L. Praamsma, C. D. Palmer, and K. N. Dietrich. 2016. Community engagement and data disclosure in environmental health research. Environmental Health Perspectives 124(2):A24–A27.

Holm, I. A., S. K. Savage, R. C. Green, E. Juengst, A. McGuire, S. Kornetsky, S. J. Brewster, S. Joffe, and P. Taylor. 2014. Guidelines for return of research results from pediatric genomic studies: Deliberations of the Boston Children’s Hospital Gene Partnership Informed Cohort Oversight Board. Genetics in Medicine 16(7):547–552.

Holzer, J. K., L. Ellis, and M. W. Merritt. 2014. Why we need community engagement in medical research. Journal of Investigative Medicine 62(6):851–855.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×

IOM (Institute of Medicine). 2012. Evolution of translational omics: Lessons learned and the path forward. Washington, DC: The National Academies Press.

IOM. 2013. Delivering high-quality cancer care: Charting a new course for a system in crisis. Washington, DC: The National Academies Press.

IOM. 2015. Sharing clinical trial data: Maximizing benefits, minimizing risk. Washington, DC: The National Academies Press.

Jarvik, G. P., L. M. Amendola, J. S. Berg, K. Brothers, E. W. Clayton, W. Chung, B. J. Evans, J. P. Evans, S. M. Fullerton, C. J. Gallego, N. A. Garrison, S. W. Gray, I. A. Holm, I. J. Kullo, L. S. Lehmann, C. McCarty, C. A. Prows, H. L. Rehm, R. R. Sharp, J. Salama, S. Sanderson, S. L. Van Driest, M. S. Williams, S. M. Wolf, W. A. Wolf, eMerge Act–ROR Committee, CERC Committee, CSER Act–ROR Working Group, and W. Burke. 2014. Return of genomic results to research participants: The floor, the ceiling, and the choices in between. American Journal of Human Genetics 94(6):818–826.

Kalia, S. S., K. Adelman, S. J. Bale, W. K. Chung, C. Eng, J. P. Evans, G. E. Herman, S. B. Hufnagel, T. E. Klein, B. R. Korf, K. D. McKelvey, K. E. Ormond, C. S. Richards, C. N. Vlangos, M. Watson, C. L. Martin, and D. T. Miller. 2016. Recommendations for reporting of secondary findings in clinical exome and genome sequencing, 2016 update: A policy statement of the American College of Medical Genetics and Genomics. Genetics in Medicine 19:249–255.

Kullo, I. J., R. Haddad, C. A. Prows, I. Holm, S. C. Sanderson, N. A. Garrison, R. R. Sharp, M. E. Smith, H. Kuivaniemi, E. P. Bottinger, J. Connolly, B. J. Keating, C. A. McCarty, M. S. Williams, and G. P. Jarvik. 2014. Return of genomic results in the genomic medicine projects of the eMERGE network. Frontiers in Genetics 5:50.

Miller, F. G., and A. Wertheimer. 2007. Facing up to paternalism in research ethics. The Hastings Center Report 37(3):24–34.

MRCT (Multi-Regional Clinical Trials) Center. 2017. Return of individual results to participants recommendations document. Boston, MA: MRCT Center.

National Bioethics Advisory Commission. 1999. Research involving human biological materials: Ethical issues and policy guidance, volume 1. Rockville, MD: U.S. Government Printing Office.

NCI (National Cancer Institute). 2018. NCI dictionary of cancer terms. https://www.cancer.gov/publications/dictionaries/cancer-terms/def/biospecimen (accessed March 1, 2018).

NIH (National Institutes of Health). 2018. About the All of Us research program. https://allofus.nih.gov/about/about-all-us-research-program (accessed March 1, 2018).

Ohayon, J. L., E. Cousins, P. Brown, R. Morello-Frosch, and J. G. Brody. 2017. Researcher and institutional review board perspectives on the benefits and challenges of reporting back biomonitoring and environmental exposure results. Environmental Research 153:140–149.

Open Humans. 2018. About Open Humans. https://www.openhumans.org/about (accessed March 1, 2018).

PCSBI (Presidential Commission for the Study of Bioethical Issues). 2013. Anticipate and communicate ethical management of incidental and secondary findings in the clinical, research, and direct to consumer contexts. Washington, DC: Presidential Commission for the Study of Bioethical Issues.

Prince, A. E. R., J. M. Conley, A. M. Davis, G. Lázaro-Muñoz, and R. J. Cadigan. 2015. Automatic placement of genomic research results in medical records: Do researchers have a duty? Should participants have a choice? Journal of Law, Medicine & Ethics 43(4):827–842.

Rigby, H., and C. V. Fernandez. 2005. Providing research results to study participants: Support versus practice of researchers presenting at the American Society of Hematology annual meeting. Blood 106(4):1199–1202.

SACHRP (Secretary’s Advisory Committee on Human Research Protections). 2016. Attachment B: Return of individual research results. https://www.hhs.gov/ohrp/sachrp-committee/recommendations/attachment-b-return-individual-research-results/index.html (accessed July 13, 2017).

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×

Sanderson, S. C., M. D. Linderman, S. A. Suckiel, G. A. Diaz, R. E. Zinberg, K. Ferryman, M. Wasserstein, A. Kasarskis, and E. E. Schadt. 2016. Motivations, concerns and preferences of personal genome sequencing research participants: Baseline findings from the HealthSeq project. European Journal of Human Genetics 24(1):14–20.

Skloot, R. 2010. The immortal life of Henrietta Lacks. New York: Crown Publishing Group.

Sly, P. D., B. Eskenazi, J. Pronczuk, R. Šrám, F. Diaz-Barriga, D. G. Machin, D. O. Carpenter, S. Surdu, and E. M. Meslin. 2009. Ethical issues in measuring biomarkers in children’s environmental health. Environmental Health Perspectives 117(8):1185–1190.

Tan, N., L. M. Amendola, J. M. O’Daniel, A. Burt, M. J. Horike-Pyne, L. Boshe, G. E. Henderson, C. Rini, M. I. Roche, F. M. Hisama, W. Burke, B. Wilfond, and G. P. Jarvik. 2017. Is incidental finding the best term? A study of patients’ preferences. Genetics in Medicine 19(2):176–181.

Thorogood, A., J. Yann, B. Knoppers, T. Nilsson, P. Metrakos, A. Lazaris, and A. Salman. 2014. An implementation framework for the feedback of individual research results and incidental findings in research. BMC Medical Ethics 15:88.

Wolf, S. M., F. P. Lawrenz, C. A. Nelson, J. P. Kahn, M. K. Cho, E. W. Clayton, J. G. Fletcher, M. K. Georgieff, D. Hammerschmidt, K. Hudson, J. Illes, V. Kapur, M. A. Keane, B. A. Koenig, B. S. Leroy, E. G. McFarland, J. Paradise, L. S. Parker, S. F. Terry, B. Van Ness, and B. S. Wilfond. 2008. Managing incidental findings in human subjects research: Analysis and recommendations. Journal of Law, Medicine & Ethics 36(2):219–248.

Wolf, S. M., L. M. Amendola, J. S. Berg, W. K. Chung, E. W. Clayton, R. C. Green, J. Harris-Wai, G. E. Henderson, G. P. Jarvik, B. A. Koenig, L. S. Lehmann, A. L. McGuire, P. O’Rourke, C. Somkin, B. S. Wilfond, and W. Burke. 2018. Navigating the research-clinical interface in genomic medicine: Analysis from the CSER consortium. Genetics in Medicine 20(5):545–553.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 39
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 40
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 41
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 42
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 43
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 44
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 45
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 46
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 47
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 48
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 49
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 50
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 51
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 52
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 53
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 54
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 55
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 56
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 57
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2018. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. Washington, DC: The National Academies Press. doi: 10.17226/25094.
×
Page 58
Next: 2 Principles for the Return of Individual Research Results: Ethical and Societal Considerations »
Returning Individual Research Results to Participants: Guidance for a New Research Paradigm Get This Book
×
Buy Paperback | $85.00 Buy Ebook | $69.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

When is it appropriate to return individual research results to participants? The immense interest in this question has been fostered by the growing movement toward greater transparency and participant engagement in the research enterprise. Yet, the risks of returning individual research results—such as results with unknown validity—and the associated burdens on the research enterprise are competing considerations.

Returning Individual Research Results to Participants reviews the current evidence on the benefits, harms, and costs of returning individual research results, while also considering the ethical, social, operational, and regulatory aspects of the practice. This report includes 12 recommendations directed to various stakeholders—investigators, sponsors, research institutions, institutional review boards (IRBs), regulators, and participants—and are designed to help (1) support decision making regarding the return of results on a study-by-study basis, (2) promote high-quality individual research results, (3) foster participant understanding of individual research results, and (4) revise and harmonize current regulations.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!