National Academies Press: OpenBook
« Previous: Summary
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press. doi: 10.17226/26289.
×

1

Introduction

The U.S. Environmental Protection Agency’s (EPA) Integrated Risk Information System (IRIS) Program is the part of the agency’s Office of Research and Development (ORD) that develop human health assessments that focus on hazard identification and dose-response analyses for chemicals in the environment. IRIS assessments cover cancer and noncancer outcomes and include toxicity values (reference values and unit risks) that can be combined with exposure estimates to develop quantitative estimates of risk. The assessments are highly important as they are used to inform risk assessments and risk management decisions throughout the agency, including EPA regulatory programs. The assessments are also used by federal, state and tribal agencies as well as community organizations and agencies in other countries to inform decisions concerning health risk assessment and management.1

Over the years, questions had been raised about the scientific basis of toxicity values reported in some IRIS assessments and the extensive amount of time taken to complete assessments. A 2011 report from the National Academies of Sciences, Engineering, and Medicine that reviewed a draft IRIS assessment of formaldehyde offered recommendations for improving the IRIS process (NRC, 2011). In 2014, the National Academies issued a report that reviewed the IRIS program process and expanded on those recommendations. Although many other recommendations were provided in the 2014 report, it concluded that “substantial improvements in the IRIS process have been made, and it is clear that EPA has embraced and is acting on the [National Academies] recommendations” (NRC, 2014, p. 9). In 2018, a National Academies report reviewed progress toward transforming the IRIS program and concluded that substantive progress has been made in transforming the IRIS program to date. It also found the program to be on track toward full implementation of systematic review and transparency in IRIS assessments (NASEM, 2018).

A major recommendation of the 2014 National Academies report was to adopt systematic review methods throughout the IRIS assessment process, including the steps on scoping, problem formulation, protocol development, study identification and evaluation, and evidence integration (NRC, 2014). Systematic review is “a scientific investigation that focuses on a specific question and uses explicit, prespecified scientific methods to identify, select, assess, and summarize the findings of similar but separate studies. The goal of systematic review methods is to ensure that the review is complete, unbiased, reproducible, and transparent” (IOM, 2011, p. 1).

The 2014 National Academies report recommended that EPA develop a handbook to “provide a single detailed guidance document for all those involved in the development

___________________

1 Additional information on the IRIS program is available at https://www.epa.gov/iris.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press. doi: 10.17226/26289.
×

of IRIS assessments” and make the IRIS process transparent to stakeholders (NRC, 2014, p. 23). The 2014 report acknowledged the challenges of having one guidance document to meet the needs of EPA staff and consultants who conduct IRIS assessments, as well as stakeholders.

A 2021 National Academies report that reviewed EPA’s Toxic Substances Control Act (TSCA) Risk Evaluations (NASEM, 2021) also recommended the development of a handbook for EPA’s TSCA review and evidence integration methodology. TSCA risk evaluations are conducted by EPA’s Office of Pollution Prevention and Toxics (OPPT), which is separate from the IRIS Program. The 2021 report also recommended that OPPT review its approach and incorporate components of the National Institute of Environmental Health Science’s Office of Health Assessment and Translation systematic review and evidence integration method (NTP, 2019), the University of California San Francisco’s Navigation Guide (Woodruff and Sutton, 2014), and IRIS methods.

The ORD Staff Handbook for Developing IRIS Assessments (the handbook), issued in November 2020, provides guidance to scientists who perform the IRIS assessments in order to foster consistency in the assessments and enhance transparency about the IRIS assessment process (EPA, 2020a; Thayer, 2021). The handbook does not supersede existing EPA guidance and does not serve as guidance for other EPA programs. It also provides an opportunity for stakeholder communities to become aware of the processes and policies guiding those who are drafting IRIS assessments.

THE COMMITTEE, ITS TASK, AND ITS APPROACH

EPA requested that the National Academies conduct a review of the handbook (EPA, 2020a). In response, the Committee to Review EPA’s IRIS Assessment Handbook was convened to evaluate the operating procedures, as described in the handbook, for hazard identification and dose-response assessment of potential human health effects from exposure to environmental contaminants. The committee was also asked to review the procedures and considerations for operationalizing the principles of systematic reviews, and the methods described in the handbook for determining the scope of the IRIS assessments (e.g., health outcomes, routes of exposure), evidence evaluation, evidence integration, and dose-response analyses. The committee’s verbatim statement of task is provided in Box 1-1. The committee included experts in epidemiology, pharmacology, toxicology, statistics, environmental health, dose-response modeling, pharmacokinetic and pharmacodynamic modeling, systematic review, and risk assessment. (See Appendix A for biographical information on the committee.)

EPA provided the committee with a list of detailed questions to further guide the committee’s review that, with one exception (see Chapter 6 of this report), fall within the general outline of the statement of task. Appendix B provides brief descriptions of the organization and content of the handbook chapters and a listing of EPA’s questions for the committee.

In carrying out its task, the committee held eight meetings, including information-gathering sessions on February 11, 2021, and April 16, 2021, to hear presentations from Jonathan Samet, Colorado School of Public Health, and Kris Thayer, EPA, respectively.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press. doi: 10.17226/26289.
×

The committee also heard public comments during each meeting (see Appendix C for meeting agendas). EPA’s presentation addressed questions from the committee regarding the handbook (Thayer, 2021). In addition, the committee received various written materials, including EPA’s written responses to the committee’s questions and the public comments on the handbook that were submitted to the EPA docket.2

The committee reviewed the handbook, recognizing that the primary audience is internal to EPA but also includes external stakeholders. It developed a list of considerations (Box 1-2) to guide its critique of each chapter of the handbook. The list includes transparency; clarity; organization; terminology; areas of controversy about the method; and whether the described method was accurate, complete, and could be applied by those responsible for conducting IRIS assessments. Not all considerations were relevant to each handbook chapter.

The committee’s review of the handbook is independent of the 2021 TSCA review, although the committee’s findings and recommendations could help inform OPPT’s responses to recommendations in NASEM (2021) and harmonize EPA’s approach to hazard evaluation.

ORGANIZATION OF THE REPORT

The committee structured its report chapters according to the major steps in the IRIS process for developing assessments. Chapter 2 focuses on an overview of the handbook’s organization and content. Chapter 3 considers the handbook’s coverage of planning IRIS assessments, including evidence mapping, scoping, protocol development, the refined evaluation plan, and organization of the IRIS assessment. Chapter 4 focuses on the handbook’s treatment of evaluation of individual studies within a data stream. Chapter 5 covers the handbook’s handling of evidence synthesis and judgments about the strength or cer-

___________________

2 A 90-day public comment period was associated with the release of the handbook. See Federal Register vol. 85, No. 230, Nov 30, 2020. 76566–76567.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press. doi: 10.17226/26289.
×

tainty of the synthesized evidence within a data stream. Chapter 6 focuses on the handbook’s treatment of evidence integration across data streams and the judgments used to describe the strength of conclusions regarding hazard identification. Chapter 7 covers the handbook’s steps of selecting studies for toxicity derivation and deriving toxicity values based upon dose-response analyses. Table 1-1 maps the chapters of the committee’s report with the questions from EPA and relevant handbook chapters.

As requested by EPA (see Appendix B), the committee organized its recommendations to indicate their relative importance during EPA’s revision of the handbook:

  • Tier 1: Recommended Revisions—Highest priority recommendations the committee believes are critical to improve the scientific rigor and/or clarity of the document.
  • Tier 2: Suggestions—Recommendations that EPA should consider to strengthen the document.
  • Tier 3: Future Considerations—Topic areas that may inform future developments.

These recommendations are outside the immediate scope and/or needs of the current document under review.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press. doi: 10.17226/26289.
×

TABLE 1-1 Mapping of Committee’s Report Chapters and EPA Questions

Committee Report Chapter Questions from EPAa Relevant Handbook Chapter
2. Overview of Organization and Content of the Handbook Q1. Comment on the overall organization of the handbook, in particular on whether key aspects of the assessment process are represented. Entire Document
3. Planning Assessments Q2. Are systematic review approaches used by the IRIS Program (outlined in Chs. 1-5) clearly described and consistent with methodologies considered to be state-of-the-science by experts in the field? 1-7, 10
Q4. Does the handbook clearly lay out a state-of-the-science approach for refinement of the scope and analyses of the IRIS assessment? Are there specific areas for improvement (recommended alternatives)?
Q5. Review and assess the handbook’s process for evaluating and integrating mechanistic data. [Sections 2.2, 4.3.3, and 6.6, and Ch. 10] Are there specific areas for improvement (recommended alternatives)?
4. Study Evaluation Q3. Are the study evaluation methods in Ch. 6 for individual human studies (epidemiology and controlled exposure), animal studies, mechanistic evidence (pilot testing approaches), and pharmacokinetic models adequate? If not, how can the methods be improved? 6
5. Evidence Synthesis Q6 (part a). Are the approaches to evidence synthesis described in Ch. 9 and Ch. 10 scientifically sound? 9-11
  • Are the considerations sufficiently broad to allow for application to the wide range of scenarios that will be encountered when applied to individual assessments?
  • Are the methods sufficiently clear in describing the intent to synthesize the relevant evidence, incorporating study evaluation conclusions, regardless of the study results?
    [NOTE: Ch. 11 discusses synthesis as part of integration]
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press. doi: 10.17226/26289.
×
6. Evidence Integration Q6 (part b). Are the approaches described in Ch. 11 scientifically sound and appropriate for integrating the various types of evidence relevant to investigating the potential for human health effects from exposure to environmental chemicals? 11
Q7. Comment on approaches using five categories versus three categories for drawing evidence integration conclusions. Which approach is recommended and why? Are there any specific refinements for improvement?
7. Hazard Considerations and Study Selection for Deriving Toxicity Values Q8. Does the handbook provide appropriate considerations in Ch. 12 and Ch. 13 for identifying data sets for dose-response analysis based on systematic review conclusions? Are the basic methods for dose-response modeling and deriving toxicity values consistent with the current state of the science, and presented with sufficient clarity? 12, 13

a Chapter and section numbers listed in the questions refer to those of the ORD Staff Handbook for Developing IRIS Assessments (EPA, 2020a). See Appendix B of this report for additional information.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press. doi: 10.17226/26289.
×
Page 13
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press. doi: 10.17226/26289.
×
Page 14
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press. doi: 10.17226/26289.
×
Page 15
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press. doi: 10.17226/26289.
×
Page 16
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press. doi: 10.17226/26289.
×
Page 17
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press. doi: 10.17226/26289.
×
Page 18
Next: 2 Overview of Organization and Content of the Handbook »
Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version Get This Book
×
Buy Paperback | $25.00 Buy Ebook | $20.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The U.S. Environmental Protection Agency's (EPA) Integrated Risk Information System (IRIS) program develops human health assessments that focus on hazard identification and dose-response analyses for chemicals in the environment. The ORD Staff Handbook for Developing IRIS Assessments (the handbook) provides guidance to scientists who perform the IRIS assessments in order to foster consistency in the assessments and enhance transparency about the IRIS assessment process. At the request of the EPA, this report reviews the procedures and considerations for operationalizing the principles of systematic reviews and the methods described in the handbook for determining the scope of the IRIS assessments, evidence integration, extrapolation techniques, dose-response analyses, and characterization of uncertainties.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!