Consensus Study Report

NATIONAL ACADEMIES PRESS 500 Fifth Street, NW, Washington, DC 20001

This activity was supported by contracts between the National Academy of Sciences and the U.S. Environmental Protection Agency. Any opinions, findings, conclusions, or recommendations expressed in this publication do not necessarily reflect the views of any organization or agency that provided support for the project.

International Standard Book Number-13: 978-0-309-70077-1
International Standard Book Number-10: 0-309-70077-9
Digital Object Identifier: https://doi.org/10.17226/26906

This publication is available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.

Copyright 2023 by the National Academy of Sciences. National Academies of Sciences, Engineering, and Medicine and National Academies Press and the graphical logos for each are all trademarks of the National Academy of Sciences. All rights reserved.

Printed in the United States of America.

Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2023. Building Confidence in New Evidence Streams for Human Health Risk Assessment: Lessons Learned from Laboratory Mammalian Toxicity Tests. Washington, DC: The National Academies Press. https://doi.org/10.17226/26906.

The National Academy of Sciences was established in 1863 by an act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president.

The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. John L. Anderson is president.

The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president.

The three academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine.

Learn more about the National Academies of Sciences, Engineering, and Medicine at www.nationalacademies.org.

Consensus Study Reports published by the National Academies of Sciences, Engineering, and Medicine document the evidence-based consensus on the study’s statement of task by an authoring committee of experts. Reports typically include findings, conclusions, and recommendations based on information gathered by the committee and the committee’s deliberations. Each report has been subjected to a rigorous and independent peer-review process, and it represents the position of the National Academies on the statement of task.

Proceedings published by the National Academies of Sciences, Engineering, and Medicine chronicle the presentations and discussions at a workshop, symposium, or other event convened by the National Academies. The statements and opinions contained in proceedings are those of the participants and are not endorsed by other participants, the planning committee, or the National Academies.

Rapid Expert Consultations published by the National Academies of Sciences, Engineering, and Medicine are authored by subject-matter experts on narrowly focused topics that can be supported by a body of evidence. The discussions contained in rapid expert consultations are considered those of the authors and do not contain policy recommendations. Rapid expert consultations are reviewed by the institution before release.

For information about other products and activities of the National Academies, please visit www.nationalacademies.org/about/whatwedo.

COMMITTEE ON VARIABILITY AND RELEVANCE OF CURRENT LABORATORY MAMMALIAN TOXICITY TESTS AND EXPECTATIONS FOR NEW APPROACH METHODS (NAMS) FOR USE IN HUMAN HEALTH RISK ASSESSMENT

WEIHSUEH A. CHIU (Chair), Texas A&M University

KIM BOEKELHEIDE, Brown University School of Medicine

PATIENCE BROWNE, Organisation of Economic Co-operation and Development (until September 2022)

HOLLY DAVIES, Washington State Department of Health

CORIE A. ELLISON, Procter and Gamble

MARIE C. FORTIN, Jazz Pharmaceuticals

NICOLE KLEINSTREUER, National Institute of Environmental Health Sciences (until September 2022)

NANCY E. LANE, University of California Davis

HEATHER B. PATISAUL, North Carolina State University

ELIJAH J. PETERSEN, National Institute of Standards and Technology

KRISTI PULLEN FEDINICK, Center for Earth, Energy, and Democracy

MARTYN T. SMITH, University of California Berkeley

ROBYN L. TANGUAY, Oregon State University

CHRISTOPHER VULPE, University of Florida Gainesville

TRACEY J. WOODRUFF, University of California San Francisco

JOSEPH C. WU, Stanford University

Consultants to the Committee

A. JOHN BAILER, Miami University

MALCOLM MACLEOD, University of Edinburgh

Other Consultants

GRACE COONEY, ICF Resources LLC

CARY HAVER, ICF Resources LLC

JENNIFER SAUNDERS

JENNA SPROWLES, ICF Resources LLC

Staff

KATHRYN Z. GUYTON, Project Director

NATALIE ARMSTRONG, Associate Program Officer

LESLIE BEAUCHAMP, Senior Program Assistant

ELIZABETH BOYLE, Senior Program Officer

TAMARA DAWSON, Program Coordinator (until August 2022)

CORRINE LUTZ, Senior Program Officer (until July 2022)

THOMASINA LYLES, Senior Program Assistant

Sponsor

U.S. ENVIRONMENTAL PROTECTION AGENCY

BOARD ON ENVIRONMENTAL STUDIES AND TOXICOLOGY

Members

FRANK W. DAVIS (Chair), University of California Santa Barbara

ANN M. BARTUSKA, U.S. Department of Agriculture

DANA BOYD BARR, Emory University

GERMAINE M. BUCK LOUIS, George Mason University

FRANCESCA DOMINICI, Harvard University

R. J. LEWIS, ExxonMobil Biomedical Sciences, Inc.

MARIE L. MIRANDA, Children’s Environmental Health Initiative

REZA J. RASOULPOUR, Corteva Agriscience

JOSHUA TEWKSBURY, Smithsonian Tropical Research Institute

SACOBY M. WILSON, University of Maryland, College Park

TRACEY J. WOODRUFF, University of California San Francisco

Staff

CLIFFORD DUKE, Director

RAYMOND A. WASSEL, Scholar and Director of Environmental Studies

KATHRYN Z. GUYTON, Senior Program Officer

NATALIE ARMSTRONG, Associate Program Officer

ANTHONY DEPINTO, Associate Program Officer

LAURA LLANOS, Finance Business Partner

LESLIE BEAUCHAMP, Senior Program Assistant

KATHERINE KANE, Senior Program Assistant

THOMASINA LYLES, Senior Program Assistant

INSTITUTE FOR LABORATORY ANIMAL RESEARCH^1,2

Council Members

ROBERT C. DYSKO (Chair), University of Michigan

CORY BRAYTON (Ex Officio Member), Johns Hopkins University

SONNET S. JONKER, Oregon Health & Science University

ANNE MAGLIA, University of Massachusetts Lowell

SUZAN MURRAY, Smithsonian Conservation Biology Institute

BARBARA J. NATTERSON-HOROWITZ, University of California Los Angeles, and Harvard Medical School

GUY H. PALMER (NAM), Washington State University

CHRISTINE A. PETERSEN, University of Iowa

ROSALIND ROLLAND, Anderson Cabot Center for Ocean Life, New England Aquarium (Emeritus)

CAROLINE ZEISS, Yale School of Medicine

Staff

TERESA J. SYLVINA, Director, Strategic Initiatives on Animal Research; Director, ILAR; Director, Roundtable on Science and Welfare in Laboratory Animal Use; Director, Collaborating Centre World Organisation for Animal Health (WOAH; OIE) (until February 2023)

SUSANA RODRIGUEZ, Program Officer

JEANNE AQUILINO, Finance Business Partner

KYLE CAVAGNINI, Associate Program Officer

RENEE DALY, Program Assistant

___________________

¹ Collaborating Centre, World Organisation for Animal Health (WOAH; originally established as OIE).

² Consistent with broadening its scope to include wildlife, non-model animal species, and biodiversity, ILAR will be known as the Board on Animal Health Sciences, Conservation, and Research (BAHSCR).

This page intentionally left blank.

Reviewers

This Consensus Study Report was reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise. The purpose of this independent review is to provide candid and critical comments that will assist the National Academies of Sciences, Engineering, and Medicine in making each published report as sound as possible and to ensure that it meets the institutional standards for quality, objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process.

We thank the following individuals for their review of this report:

Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations of this report, nor did they see the final draft before its release. The review of this report was overseen by DAVID DORMAN, North Carolina State University, and JOEL KAUFMAN (NAM), University of Washington. They were responsible for making certain that an independent examination of this report was carried out in accordance with the standards of the National Academies and that all review comments were carefully considered. Responsibility for the final content rests entirely with the authoring committee and the National Academies.

This page intentionally left blank.

Acknowledgements

Many people were critical in helping the committee accomplish its charge. The committee gratefully acknowledges the participants at the committee’s information-gathering sessions, who provided insights and viewpoints pertinent to the committee’s task (see Appendix B). In particular, we thank David Allen, Thomas Burke, Vincent Cogliano, David Dorman, Suzanne Fenton, Katie Paul Friedman, Helen Goeden, Thomas Hartung, Rashmi Joglekar, Sharon Munn, Reza Rasoulpour, and Joshua Robinson, who contributed their expertise to the first virtual workshop; and Carl-Gustaf Bornehag, Beate Escher, Elaine Faustman, Chris Gennings, Helena Hogberg, Richard Judson, Paul Mermelstein, Cynthia Rider, Ruthann Rudel, Tim Shafer, Jason Stein, Shirlee Tan, Laura Vandenberg, Tom Webster, and Fred Wright, who contributed their expertise to the second virtual workshop.

In addition, we are also grateful to Environmental Protection Agency (EPA) staff for their presentations to the committee: Maureen Gwinn and Russell Thomas, Office of Research and Development; and Tala Henry and Anna Lowit, Office of Chemical Safety and Pollution Prevention.

Importantly, the committee heard from a number of individuals who shared their perspectives during the public comment periods of the information-gathering sessions, and provided written input for the committee’s consideration. The committee is very grateful for these valuable contributions to its work.

Gratitude is also extended to Malcolm Macleod and A. John Bailer, who served as consultants to the committee. In addition, the committee thanks Jennifer Saunders, who served as the Rapporteur for the first virtual workshop and developed the Proceedings of a Workshop—in Brief. The committee is also grateful to Vincent Cogliano and Suzanne Fenton, as well as David Butler and Jennifer Cohen from the National Academies, who served as reviewers for this publication. The committee’s work was also enhanced by data visualization and extraction support, and the committee is grateful to Grace Cooney, Cary Haver, and Jenna Sprowles who served as consultants.

The committee is grateful to the staff of the National Academies of Sciences, Engineering, and Medicine who contributed to producing this report, especially the outstanding and tireless study staff: Natalie Armstrong, Leslie Beauchamp, Elizabeth Boyle, Tamara Dawson, Kathryn Guyton, Corrine Lutz, and Thomasina Lyles. Thanks also go to the staff of the Division on Earth and Life Studies who provided additional support, including Clifford Duke, Elizabeth Eide, Lauren Everett, Nancy Huddleston, Radiah Rose, and Maggie Walser. This project also received important assistance from Laura Llanos (Office of Financial Administration). Valuable research assistance was provided by Christopher Lao-Scott, senior research librarian in the National Academies Research Center.

This page intentionally left blank.

Preface

A number of seminal reports from the National Academies of Sciences, Engineering, and Medicine (NASEM) have provided recommendations for advancing the ways in which the exposures and effects of environmental agents are characterized and the associated hazards and risks are assessed. As stand-alone reports of different ad hoc committees, these consensus reports nonetheless provide some common themes on which the current committee’s work was built. The landmark 2007 report, Toxicity Testing in the 21^st Century: A Vision and a Strategy, motivated a shift in focus from adverse apical endpoints to mechanistic processes and other biomarkers of homeostatic perturbations. The 2009 Science and Decisions: Advancing Risk Assessment report highlighted the need to provide hazard and dose-response information in the absence of epidemiologic or chronic experimental animal toxicology data, suggesting approaches such as quantitative structure–activity relationships (QSAR) or use of shorter-term toxicity tests to generate “actionable” predictions for risk assessment and as input into subsequent risk management decisions. The 2015 report Application of Modern Toxicology Approaches for Predicting Acute Toxicity for Chemical Defense recommended an approach that incorporates information from a suite of applicable databases, assays, models, and tools that also balances the need for accuracy and timeliness. The subsequent 2017 report Using 21^st Century Science to Improve Risk-Related Evaluations emphasized the importance of advancing the science and regulatory acceptance of using data from a wider variety of testing systems, such as fish and alternative animal models, including alternative mammalian models.

A number of other NASEM reports have provided concrete recommendations for advancing the science and practice of risk assessment, focusing on systematic review methods that increase transparency and build confidence in the resulting decision. These reports build from the 2011 review of the assessment of formaldehyde by Environmental Protection Agency’s (EPA) Integrated Risk Information System (IRIS) and include subsequent recommendations for the IRIS program (in 2014, 2018, and 2022), regarding Toxic Substances Control Act (TSCA) evaluations (in 2021), and to the Department of Defense (in 2019). For instance, NASEM’s reviews of the IRIS program have noted considerable progress in use of systematic reviews for chemical assessments. In addition, numerous NASEM reports (e.g., the 2017 report, Application of Systematic Review Methods in an Overall Strategy for Evaluating Low-Dose Toxicity from Endocrine Active Chemicals) have demonstrated application of systematic review methods to evaluate agents of concern.

Many recommendations from these prior NASEM reports remain valid today. In addition, these reports highlight the salient and long-standing challenges faced in risk assessment and the opportunities for addressing them. Particularly relevant today is the potential to improve assessment of the effects of chemicals or other stressors using novel testing strategies. For instance, these methods and approaches afford the potential for better protecting human health by moving away from frank effects to examine perturbations that may be both more subtle as well as more relevant and important to public health. Another promising application of such novel data streams is to inform timely decision-making when no data are available from existing laboratory mammalian toxicity tests or epidemiological studies. In addition, the advent of population-based models encompassing genetic diversity, both in vitro and in vivo, as well as the exploration of experimental approaches to incorporate nonchemical stressors provide potential avenues to ensure protection of susceptible and vulnerable populations. Finally, recommendations on the application of systematic review methodologies provide the basis for state-of-the-art methods to evaluate the human health hazards of environmental contaminants in a transparent, reproducible manner to reach sound and defensible conclusions.

Because the committee’s charge focuses on human health risk assessment applications of data from “new approach methods” or NAMs, the prior recommendations, challenges identified, and opportunities highlighted serve as an important background to the present work. However, despite recommendations that date to 2007, there are few examples of the application of NAMs to inform risk assessment decision-making. For instance, most evidence integration paradigms do not formally permit a hazard identification or classification that is solely based on mechanistic evidence or other data streams besides human epidemiological or laboratory mammalian toxicity studies. A notable exception is the 2019 International Agency for Research on Cancer (IARC) Monographs Preamble, under which human, experimental animal, and mechanistic evidence are integrated simultaneously, and hazard conclusions are possible in the absence of human and experimental animal evidence. In addition, although dose- or concentration-response information from NAMs have been proposed for use in prioritization, laboratory mammalian toxicity studies still form the basis of most toxicity values that are used for setting human health-based exposure limits or other risk management actions.

Therefore, the committee aimed to provide recommendations for bridging this notable gap between the potential of NAMs and their practical application in human health risk assessment. The committee sought to take advantage of the experiences already gained since the landmark 2007 report, both in development of novel testing methods and strategies as well as in the development of systematic review-based approaches to evidence evaluation, synthesis, and integration. Together, these experiences highlight the opportunity to design up front those approaches for incorporating NAMs data into risk assessment and decision-making based on current best practices. As such, the recommendations aim to provide a path that builds confidence in such data and approaches from start to finish. These recommendations also aim to prepare for a future when these data may be the sole source of information on which to base human health risk assessment and risk management decisions.

Ultimately, the committee envisions that implementing these recommendations, along with those of previous NASEM committees, will address many long-standing challenges in human health risk assessment—from lack of data for most chemicals in the environment to better coverage of susceptible and vulnerable populations—and thereby lead to better overall protection of public health.

Weihsueh Chiu, Chair
Committee on Variability and Relevance of Current
Laboratory Mammalian Toxicity Tests and Expectations for New Approach Methods (NAMs) for Use in Human Health Risk Assessment

Contents

ACRONYMS AND ABBREVIATIONS

SUMMARY

1 INTRODUCTION

The Committee and Its Task

The Committee’s Approach to Its Task

Organization of This Report

References

2 BACKGROUND AND DEFINITIONS

Background on the Use of Laboratory Mammalian Toxicity Tests in Human Health Risk Assessment

Background on the Future of Toxicity Testing

Findings and Recommendations

References

3 VARIABILITY

Experimental Variability in Laboratory Mammalian Toxicity Studies

Biological Variability in Laboratory Mammalian Toxicity Studies

Insights from Workshops

Insights from the Literature Review

Findings and Recommendations

References

4 CONCORDANCE WITH HUMAN DATA

Levels of Concordance

Toxicokinetics

Insights from Workshops

Insights from the Literature Review

Literature Presented to the Committee

Findings and Recommendations

References

5 ISSUES IN DEVELOPING A SCIENTIFIC CONFIDENCE FRAMEWORK FOR NAMS

NASEM 2017 Findings Related to Model/Assay Validation and Acceptance

Systematic Evaluation of Scientific Confidence of NAMs

Evaluating Scientific Confidence of NAM-Based Testing Strategies

Evaluation of NAMs Data in Human Health Hazard or Risk Assessment for Particular Chemical(s)

References

APPENDIXES

A COMMITTEE, STAFF, AND CONSULTANT BIOGRAPHIES

B OPEN SESSION AGENDAS

C EVIDENCE REVIEW: APPROACH, METHODS, AND RESULTS

D SELECTED EXAMPLES OF SCIENTIFIC CONFIDENCE FRAMEWORKS FOR NAMs

BOXES, FIGURES, AND TABLES

BOXES

S-1 Key Components of Scientific Confidence Framework for NAMs

1-1 Statement of Task

2-1 Perspectives from Workshop 1 on Use of Mammalian Toxicity Tests in Human Health Risk Assessment

2-2 Explanation of Terms and Definitions

3-1 Assessment of Variability Among Studies

5-1 Key Components of Scientific Confidence Framework for NAMs

5-2 Key Terminology

C-1 AMSTAR 2 Critical Domains

FIGURES

S-1 Overview of committee’s approach depicting the roadmap of information sources, approaches, and report chapters

S-2 A continuum of computational models and biological assays can provide information for human health risk assessment

S-3 The multiple sources of variability in laboratory mammalian toxicity tests

S-4 Relationship between purpose and context of use, parallel PECO statements, and external validity

S-5 Interface between components of scientific confidence for a toxicity testing method and human health hazard and risk assessment

1-1 Overview of committee’s approach depicting the roadmap of information sources, approaches, and report chapters

2-1 Exposure-to-outcome continuum and methods in toxicology

3-1 Multiple sources of variability in laboratory mammalian toxicity tests

5-1 Relationship between purpose and context of use, parallel PECO statements, and external validity

5-2 International Agency for Research on Cancer Monographs framework for integration of streams of evidence in reaching overall classifications for carcinogenicity

5-3 Interface between components of scientific confidence for a toxicity testing method and human health hazard and risk assessment

C-1 PRISMA diagram for variability studies

C-2 PRISMA diagram for concordance studies

C-3 Evidence map showing summary information on the studies included in the review

C-4 PRISMA diagram for the supplemental literature on variability

C-5 PRISMA diagram for the supplemental literature on concordance

TABLES

3-1 Summary of Higher Quality Variability Studies Identified from the Committee’s Literature Review

3-2 Reported Values for τ and for I² for the Effect of Six Phthalates on Two Endpoints, Anogenital Distance and Testosterone Level

3-3 Summary of Variability Studies Identified from Literature Presented to the Committee That Are Not Systematic Reviews

4-1 Summary of Higher Quality Concordance Studies Identified from the Committee’s Literature Review

4-2 Summary of Concordance Studies Identified from Literature Presented to the Committee That Are Not Systematic Reviews

5-1 Examples of Describing Toxicity Testing Methods Using Parallel Test Method and Target Human PECO Statements

5-2 Examples of Study Evaluation Domains for Epidemiological, Experimental Animal, and In Vitro Studies

5-3 Example Domain-Specific Approach to Evaluating External Validity Related to Directness/Indirectness

5-4 Example Approach for Structured Evaluation of External Validity for Rodent Cancer Bioassays for Predicting Cancers in Humans

5-5 Example Approach for Structured Evaluation of External Validity for Human iPSC-Derived Cardiomyocytes for Predicting Arrhythmic Cardiotoxicity in Humans

5-6 Example Approach for Structured Evaluation of External Validity for Zebrafish for Predicting Developmental Outcomes in Humans

5-7 Example Approach for Structured Evaluation of External Validity for the OECD Adverse Outcome Pathway/Key Event–Based Guideline for the Testing of Chemicals for Skin Sensitization

5-8 Example Approach for Structured Evaluation of External Validity for the Molecular Docking Prediction of Endocrine Disruption by Altering the Bioactivity of the Androgen Receptor

5-9 Interface Between Components of Scientific Confidence for a Toxicity Testing Method and Human Health Hazard and Risk Assessment

C-1 Rating Overall Confidence in the Results of the Review

C-2 Overview of Charge Questions and How They Map onto the Literature Review and Scoping Questions

D-1 Selected Examples of Scientific Confidence Frameworks for NAMs

This page intentionally left blank.

Acronyms and Abbreviations

___________________

¹ EPA defines “NAMs” as follows in its December 2021 New Approach Methods Work Plan: “NAMs are defined as any technology, methodology, approach, or combination that can provide information on chemical hazard and risk assessment to avoid the use of animal testing.”

This page intentionally left blank.