Proceedings of a Workshop

New Approach Methods (NAMs) for Human Health Risk Assessment

Proceedings of a Workshop—in Brief

Animal testing is often used to assess the potential risks, uses, and environmental impacts of chemicals. New Approach Methods (NAMs) are technologies and approaches (including computational modeling, in vitro assays, and testing using alternative animal species) that can inform hazard and risk assessment decisions without the use of animal testing. Two landmark publications from the National Research Council (NRC)¹ and the National Academies of Sciences, Engineering, and Medicine² provided recommendations for developing, improving, and validating NAMs and outlined opportunities to best integrate and use the emerging results in evaluating chemical risk. An ad hoc National Academies committee³ will build on these efforts by reviewing the variability and relevance of existing mammalian toxicity tests for human health risk assessment to inform the development of approaches for validation and establishing scientific confidence in using NAMs. As part of its work, the committee organized a 1-day virtual public workshop, held on December 9, 2021,⁴ to address the potential utility and expectations for the future use of NAMs in risk assessment and to reflect on the challenges to their implementation. The workshop addressed the following critical questions:

• How are traditional toxicity studies used in informing chemical safety decisions?
• What do we know about the variability and concordance of traditional mammalian toxicity studies?
• What are the needs and expectations of different stakeholders?

During the workshop, experts from academia, industry, government, and other organizations discussed current scientific knowledge with regard to traditional toxicity studies and NAMs. Recorded presentations were made available via the event webpage⁵ before the workshop and were briefly summarized during each of four panel discussions.

Kate Guyton, National Academies Project Director, and Weihsueh Chiu, Committee Chair, introduced the workshop topics, broader study task, and committee members. They explained that the workshop would provide important information gathering opportunities and inform the committee’s work as it develops a consensus study report.

__________________

¹ NRC (National Research Council). 2007. Toxicity testing in the 21st century: A vision and a strategy. Washington, DC: The National Academies Press. https://doi.org/10.17226/11970.

² NASEM (National Academies of Sciences, Engineering, and Medicine). 2017. Using 21st century science to improve risk-related evaluations. Washington, DC: The National Academies Press. https://doi.org/10.17226/24635.

³ See https://www.nationalacademies.org/our-work/variability-and-relevance-of-current-laboratory-mammalian-toxicity-tests-and-expectations-for-new-approach-methods--nams--for-use-in-human-health-risk-assessment.

⁴ See https://www.nationalacademies.org/event/12-09-2021/new-approach-methods-nams-for-human-health-risk-assessment-workshop-1.

⁵ Ibid.

THE USE OF TRADITIONAL MAMMALIAN TOXICITY STUDIES IN INFORMING CHEMICAL SAFETY DECISIONS

Chiu and Tracey Woodruff, University of California, San Francisco, moderated the first panel addressing the use of traditional mammalian toxicity studies in informing chemical safety decisions.

Thomas Burke, Johns Hopkins University, provided a brief overview of risk assessment, including referencing the 1983 NRC report Risk Assessment in the Federal Government: Managing the Process⁶ and the 2008 NRC report Science and Decisions: Advancing Risk Assessment.⁷ Burke noted that risk assessment serves as an essential public policy tool to inform decisions about human health. There are multiple types of risk assessment, ranging from screening assessments to comprehensive high-level assessments that present a synthesis of the evidence including human, animal, and other toxicity testing, including NAMs.

Laboratory mammalian toxicity testing has been the cornerstone of risk science, providing a strong correlation with human disease, including cancer. However, animal studies have limitations. For example, animal studies are typically conducted at high doses, which may not be relevant to humans. Although animal studies can examine mixtures, they do not consider other exposures present in the environment that may contribute to adverse effects.

To protect public health, Burke noted, future risk assessments will need to use the full range of available data, draw on innovative methods to integrate diverse data streams, and consider health endpoints that reflect the range of subtle effects and morbidities observed in human populations. Given these factors, there is a need to reframe chemical risk assessment to be more clearly aligned with the public health goal of minimizing exposures associated with disease, he said.

The demand for chemical-specific assessments remains and whole animal testing will likely continue to play an essential role in the support and validation of other evidence streams, in dose–response modeling, and in the development of points of departure (PODs; the dose–response point from which human health risk calculations are made). There is interest in moving away from “bright line” estimates of population risk and shifting toward a public health approach and broader consideration of environmental health risks such as environmental justice, climate change, and cumulative impacts. It is time for a unified approach, Burke added, that considers population vulnerabilities, background exposures, and impacts, allowing for a more complete understanding of the variability of the population risk, as discussed in the 2008 NRC report Science and Decisions. New methods can address gaps and strengthen the tools for mode of action, population variability, exposure assessment, and risk characterization. Burke closed by noting the committee’s tremendous opportunity to advance science and refine the practice and application of risk assessment.

Sharon Munn, Joint Research Centre, discussed hazard identification of endocrine disruptors (i.e., exogenous substance or mixture that alters functions of the endocrine system, causing adverse health effects), focusing on efforts in the European Union. The European Union developed criteria and guidance for endocrine disruptors in 2018 and a Chemicals Strategy for Sustainability⁸ in 2020, which bans endocrine disrupting chemicals from consumer products. The Chemicals Strategy also strengthens information requirements and accelerates the need for data on endocrine disrupting chemicals, including requiring assessment of more chemicals for critical hazards. Munn noted that this will be a significant challenge given the thousands of chemicals to assess within a tight timeframe. The International Programme on Chemical Safety⁹ has also developed criteria around endocrine disruptors, including relevance to humans, specificity, data generated according to international standards, and systematic literature review.

To accelerate the assessment of chemicals for endocrine disrupting properties, Munn shared, there is a need for more methods to investigate mechanisms of action and NAMs can play a role in this capacity. Currently, NAMs, in combination with in vivo data, can indicate adverse effects in the intact organism; however, these approaches are not a replacement for animal studies. Munn added that there is a need for data on more sensitive endocrine endpoints. There are opportunities to move forward in advancing NAMs, including ongoing work to develop complex NAMs with 3D tissues,¹⁰ which may contribute to the understanding of adversity in the context of in vitro effects.

Vincent Cogliano, California Environmental Protection Agency, discussed hazard identification and the dose–response of carcinogens, noting that animal studies have served as the backbone of cancer risk assessment. There is usually insufficient human evidence available to support the risk assessment of suspected carcinogens. In the vast majority of cases, animal studies have been used in cancer hazard identification and cancer dose–response assessment.

__________________

⁶ NRC. 1983. Risk assessment in the federal government: Managing the process. Washington, DC: National Academy Press. https://www.nap.edu/catalog/366.

⁷ NRC. 2008. Science and decisions: Advancing risk assessment. Washington, DC: The National Academies Press. https://www.nap.edu/catalog/12209.

⁸ EC (European Commission). 2020. Chemicals strategy for sustainability towards a toxic-free environment. https://ec.europa.eu/environment/pdf/chemicals/2020/10/Strategy.pdf.

⁹ See https://inchem.org.

¹⁰ NASEM. 2017. Using 21st century science to improve risk-related evaluations. Washington, DC: The National Academies Press. https://www.nap.edu/catalog/24635.

Animal studies thus provide a basis to identify a chemical as a carcinogen and to take regulatory action to reduce human exposures. Additionally, they can be used to investigate agents not covered by human studies, identify life stage windows of sensitivity and susceptible populations, and quantify differences in risk in evaluating complex exposures.

Cogliano added that the current methods for conducting linear and nonlinear extrapolation to low doses could inform a framework for NAMs. It is also important to utilize NAMs to investigate the effects of chemicals not covered by human studies. While there are high expectations that NAMs will allow for the prediction of human environmental risk, Cogliano noted the importance of not losing the current capability afforded by animal studies, which he termed “actionable evidence.”

David Dorman, North Carolina State University, discussed two National Academies’ reports: A Class Approach to Hazard Assessment of Organohalogen Flame Retardants (2019)¹¹ and Application of Modern Toxicology Approaches to Predicting Acute Toxicity for Chemical Defense (2015).¹² The authoring committees faced similar risk assessment challenges in these studies: the need to evaluate a large number of chemicals with limited traditional toxicology data or human data to support hazard identification. In both cases, the committees recognized that sole reliance on mammalian studies would limit the ability to screen or classify large numbers of chemicals. The committees viewed NAMs and read-across approaches (i.e., approaches that extrapolate data from one chemical to another) as important tools for predicting toxicity. The committees both recommended a tiered approach in which mammalian studies would be included but conducted for fewer chemicals.

In the 2019 report, the committee was tasked with examining approaches to evaluating 150 different chemicals used as flame retardants; these chemicals are among >20,000 halogenated compounds. To address this challenge, the committee examined the problem through a class and subclass approach, developing 14 defined subclasses of these chemicals based on physiological properties and biology. The literature was surveyed to identify the availability and extent of relevant data to inform the hazard assessment of the subclasses. The committee recommended a tiered approach to testing to fill gaps in data, relying on NAMs (computational modeling, in vitro assays, and testing using alternative animal species) to identify endpoints of interest and anchor chemical(s) for targeted mammalian toxicity studies. In the 2015 report, the Department of Defense (DoD) asked the committee to develop a framework to examine a variety of chemicals to identify “bad actors.” DoD wanted to screen a range of potential chemical-warfare agents to identify chemicals of interest with either a high degree of toxicity or those with low toxicity. The committee also developed a framework to aid in this process, recognizing that the one chemical at a time approach was not feasible. The framework recommended in the 2015 report proposes a variety of data and models as tools to support a prioritization strategy to address the acute toxicity of chemicals, noting that traditional mammalian toxicity studies would be preferred when other approaches were deemed inadequate.

Dorman discussed the inherent policy decisions embedded in the conceptual frameworks offered in both reports, for example, assigning chemicals to categories, the level of evidence required to make reliable decisions, and a reliance on surrogate outcomes.¹³

Panel Discussion

Several panelists noted that animal studies are the cornerstone of the current practice of risk assessment. The overwhelming predominance of evidence has come from animal studies and there is confidence within the scientific community in using them to support decision-making. The panelists addressed the factors and challenges reinforcing the continued use of traditional animal toxicity tests in risk assessment and the limitations of traditional animal toxicity tests, including for predicting risks for all populations. Munn noted that traditional mammalian studies offer data about the intact organism and provide toxicokinetic context. In contrast, Dorman added that NAMs do not allow for an examination into injury and repair in a holistic way. With NAMs, there will be a general leap of faith that the results will show more clinical relevance to humans. “It will be important to develop the confidence in NAMs that we have with animal studies, despite their limitations,” he said. Also, studies of interindividual variability in humans indicate that there may be more variability than previously understood, a complicating factor for NAMs, Munn noted.

The panelists commented that NAMs will offer different data than animal studies; researchers will both gain and lose information using these approaches. With NAMs, Dorman noted, cellular-level responses will be examined, and these data can be used to make public health decisions. As such, Dorman shared that it is also important to create policies around NAMs to support the science. Cogliano was hopeful that NAMs could provide information on human

__________________

¹¹ NASEM. 2019. A class approach to hazard assessment of organohalogen flame retardants. Washington, DC: The National Academies Press. https://www.nap.edu/catalog/25412.

¹² NASEM. 2015. Application of modern toxicology approaches to predicting acute toxicity for chemical defense. Washington, DC: The National Academies Press. https://www.nap.edu/catalog/21775.

¹³ An outcome that can be observed sooner, at lower cost, or less invasively than the true outcome, and that enables valid inferences about the effect of intervention on the true outcome. Staner, L. 2006. Surrogate outcomes in neurology, psychiatry, and psychopharmacology. Dialogues in Clinical Neuroscience 8(3):345–352. https://doi.org/10.31887/DCNS.2006.8.3/lstaner.

variation and may serve as the backbone of future regulations. As Burke stated, NAMs offer a tremendous opportunity to reduce the use of animal testing and better utilize the resources currently available.

Burke added that current approaches focus on developing a point of departure to drive a single estimate to define acceptable risk, which can ignore other aspects of co-exposures and population susceptibility. COVID-19 highlighted the importance of considering exposures in vulnerable populations, and of lowering the impact of exposures on vulnerable populations, rather than having a single acceptable risk number or point of departure. In the future, NAMs could provide an opportunity to increase the understanding of diversity, he added, “to get to the idea of complex exposures in a complex population and design experiments to get at those questions.”

To make NAMs part of actionable evidence, Cogliano noted that there is a need for policy that will reflect a consensus that certain data are indicative of potential health hazards. By integrating NAMs into the decision-making process, there is an opportunity to improve public health, Munn noted. Burke added that regarding NAMs, “once we get to that actionable evidence, we have to be prepared to defend it. And to be very transparent about the inherent limitations of all types of data.”

UNDERSTANDING THE VARIABILITY OF TRADITIONAL MAMMALIAN TOXICITY STUDIES WITH DIFFERENT LEVELS OF COMPLEXITY

Nicole Kleinstreuer, NTP Interagency Center for the Evaluation of Alternative Toxicological Methods, and Holly Davies, Washington State Department of Health, moderated a panel on the variability of traditional mammalian toxicity studies.

David Allen, Integrated Laboratory Systems, LLC, discussed ways to evaluate variability within traditional mammalian toxicity studies and how that information can be used in the development of NAMs. Historically, stakeholders placed a lot of confidence in in vivo toxicity testing. In order to establish confidence in NAMs, they are typically compared to in vivo test methods, Allen noted. However, there is a need to characterize the usefulness and limitations of in vivo methods. Studies examining the variability of acute endpoints have used both qualitative and quantitative approaches. Allen described the results of analyses of in vivo variability within three guideline test methods, for eye and skin irritation in rabbits and acute oral toxicity in rats. Researchers examined the conditional probability of finding the same hazard category with the same chemical tested multiple times. Regarding eye and skin irritation, variability was the greatest when testing mild and moderate irritants, while some corrosive substances were found to be non-irritants, if retested. For acute oral toxicity, the results were variable across categories, while mild toxicity was the most reproducible.

In vivo data have been used to derive thresholds for hazard categorization, precautionary labeling, and performing quantitative risk assessments, Allen noted. Establishing confidence in NAMs could be informed by the consideration of variability in in vivo test methods. In vivo variability is another factor to consider for determining whether NAM concordance with animal data is an appropriate comparison.

Katie Paul Friedman, U.S. Environmental Protection Agency (EPA), discussed qualitative and quantitative analyses of variability among traditional mammalian toxicity studies of the same and different design. EPA, as required by the Toxic Substances Control Act and amendments, currently relies on data from animal tests but is moving to replace the current approaches with NAMs that are validated and shown to be equivalent to, or better than, the replaced animal tests. She noted that, quantitatively, variability in traditional animal toxicity tests is a measure of how far values are spread from the average. Qualitatively, variability concerns whether a specific effect is observed or not (i.e., are there false positives or negatives). Friedman discussed efforts using a range of different modeling approaches and the ToxRefDB v2.0¹⁴ dataset to approximate total variance in systemic effect levels. She noted that the estimate of variance (root mean square error) in curated lowest effect levels (LELs) and/or lowest observed adverse effect level (LOAEL) approaches 0.5 log₁₀-mg/kg/day. The work published by Pham et al. (2020),¹⁵ and Pradeep et al. (2020)¹⁶ supported Friedman’s conclusion that variability in in vivo toxicity studies limits the predictive accuracy of NAMs. The maximal R-squared for a NAM-based predictive model of systemic effect levels may be 55%–73% (i.e., as much as one-third of the variance in these data may not be explainable using study descriptors). Understanding that a prediction of an animal systemic effect level within +/– 1 log₁₀-mg/kg/day demonstrates a strong NAM is important for acceptance of these approaches for chemical safety assessment. Friedman noted that the construction of NAM-based effect level estimates that offer an equivalent level of public health protection as effect levels derived from animal tests may lead to a reduction in the use of animal testing. In addition, this may support the identification of cases in which animals may provide scientific value. Friedman commented that existing QSAR (quantitative structure-activity relationship) methods for repeat dose PODs may be informative if the intent is to evaluate a large number of chemicals in a short period of time,

__________________

¹⁴ EPA (U.S. Environmental Protection Agency). 2020. ToxRefDB version 2.0: Improved utility for predictive and retrospective toxicology analyses. https://catalog.data.gov/dataset/toxrefdb-version-2-0-improved-utility-for-predictive-and-retrospective-toxicology-analyses.

¹⁵ Pham, L. L., S. Watford, P. Pradeep, M. T. Martin, R. Thomas, R. Judson, R. W. Setzer, and K. P. Friedman. 2020. Variability in in vivo studies: Defining the upper limit of performance for predictions of systemic effect levels. Computational Toxicology 15:100126. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7787987.

¹⁶ Pradeep, P., K. P. Friedman, and R. Judson. 2020. Structure-based QSAR models to predict repeat does toxicity points of departure. Computational Toxicology 16:100139. https://doi.org/10.1016/j.comtox.2020.100139.

and that work is in progress to support best practices for predicting in vivo PODs at the organ level.

Suzanne Fenton, National Institute of Environmental Health Sciences, discussed variability within and across animal species in traditional mammalian toxicity studies. In addressing environmental contributors to public health issues, healthy rodent models are used in test guideline (TG) studies to identify dose–response, sex- or diet-dependence, developmental time sensitivity, and mechanistic relationships with relevant disease outcomes. Traditional TG studies that are similar in design may use different strains of rats or multiple species, use a dose range that is often much higher than what is relevant to humans, use varied diets, and may not include windows of sensitivity for most developing organ systems. Fenton discussed the sources of variability in animal tests, including those related to diet and water source; sex-specific differences; route of administration; timing of administration; collection of female tissue in various phases of estrous cycle; strain; and species, which can be determinants of the sensitivity to the substance being tested and the propensity to manifest the disease outcome of interest.

Moving forward, Fenton discussed areas for improvement to address sources of variability, for example, controlling for possible environmental contamination, evaluating both sexes equally, and using the most appropriate species to evaluate a particular outcome. Adding endocrine sensitive endpoints to current TG studies to reduce the number of additional studies is also a potential area for improvement. The testing of unhealthy or stressed animals would be beneficial as most testing is done with healthy animals. Bioaccumulation in the offspring through placental and lactational transfer of test compounds is also a variable to be considered. TG studies currently fall short in a number of areas that are of critical public health interest, including breast development and functional assessment, obesity and metabolic diseases, assessment of placental or pregnancy complications, thyroid disease, hypertension, and allergy, asthma, and autoimmune studies. To generate actionable data specific to issues of public health concern, Fenton noted that non-traditional or non-TG toxicity studies may be necessary. Shorter, more predictive in vivo assays along with genetic data could help to predict disease outcomes. Early life exposures and early life endpoints could also be used to predict later life endpoints.

Malcolm MacLeod, University of Edinburgh, summarized the use of systematic reviews and meta-analyses in examining in vivo data. He noted that, compared with human studies, there are important differences in the data structures of animal studies: the number of subjects is typically smaller, and thus variance in sample does not always reflect variance in population. In addition, the number of studies is usually large, and therefore there may be heterogeneity expected between studies rather than within studies (e.g., because of the sex or age of the animals, dose used, or timing of assessments). Between-study heterogeneity can be estimated in different ways, including tau squared (the measure of the dispersion of the various true effect sizes). Thus, if the effect of a chemical is adequately described in a corpus of work, further studies will increase the precision of tau squared but not change its value. MacLeod noted that these approaches can examine whether there is a biological effect, how big the effect may be, among a number of other areas, including whether there is publication bias (i.e., when publication depends on the nature and direction of study results¹⁷). A meta-analysis can assess issues such as whether a chemical is reliably hazardous across subjects, ages, and ethnicities. These analyses can examine whether there is a subgroup of subjects who are at greater risk of adverse effects from exposure.

MacLeod discussed the challenges to conducting systematic reviews of animal studies. For example, it can be challenging to conduct these studies, given the size of the available literature and data and that systematic reviews can be time consuming and cumbersome, particularly as they require significant time to extract the data, conduct the analysis, and develop conclusions. However, there are some automation tools (e.g., machine learning, artificial intelligence), that can increase the speed with which these analyses can be conducted, MacLeod noted. Through these new tools, there is the possibility of creating systematic online living evidence summaries.

Panel Discussion

Kleinstreuer and Davies led a discussion around the variability of traditional mammalian toxicity studies. Allen observed that regulators have indicated that they have strong confidence in in vivo studies to inform specific hazard identification or comprehensive risk assessments. Thus, new approaches to develop these data have been held to a tremendously high standard, including the requirement to repeat the identical outcome of an in vivo method.

Friedman, agreeing with Allen’s comments, discussed her own efforts to predict the point of departure in repeat dose studies, noting that a best-case scenario for replacing repeat dose studies may be to consider what the scientific community wants to accomplish from these studies. Friedman noted that a quantitative evaluation that determines a point of departure, LD₅₀, or dose estimate may be needed. She added that if researchers have a specific hazard to predict, there is a need to carefully consider what is identified as the reference chemical.

__________________

¹⁷ Royle, P., and N. Waugh. 2003. Literature searching for clinical and cost-effectiveness studies used in health technology assessment reports carried out for the National Institute for Clinical Excellence appraisal system. Health Technology Assessment 7(34):iii, ix–x, 1–51. https://pubmed.ncbi.nlm.nih.gov/14609481.

MacLeod added to the discussion of challenges around variability, specifically, highlighting the experimental replication efforts of results in animal studies. In the context of NAMs, he noted that there is a need to be careful about the current state of in vitro research, as systematic reviews have highlighted significant bias and other issues in the animal studies.

Fenton added that when discussing communities of color, variability between individuals and groups of people is relevant; variability in animal studies based on genetic differences is also expected, just as would be expected in human communities. This translates to how variability is considered in NAMs, especially when results are derived from a single cell type. Fenton noted that, currently, in vitro and in vivo NAMs do a poor job of addressing the complexity of issues faced by communities of color and consideration across a broader dataset may be informative of key issues around race, ethnicity, and diversity. It will be necessary for NAMs to be as good as animal studies, Fenton added.

MacLeod noted that the consistency of effect and heterogeneity between studies are important aspects of variability. The latter is a measure of the difference in the potential biological responses; it is necessary to have a way of embracing heterogeneity, he said.

Davies asked the panelists to discuss the strengths and limitations of traditional animal toxicity tests with regard to reliability and quantitative reproducibility. Allen noted that the largest area of variability is in the subjective qualitative endpoints that are scored by individual technicians and the differences in diet, husbandry, or strain that might limit the analysis. Friedman added that similar limitations in repeat dose studies include pathological scoring and the study design. In these studies, there is a need to approximate the extent of variability. Additionally, many findings underscore observable adverse effects, and it is not clear what pathway is involved. Fenton added that limitations of animal toxicity tests include a lack of best practices around protocols and in reporting the data, problems that will hinder progress in this area.

Davies asked the panelists to comment on how NAMs can help assess chemicals with relatively little or no data. Friedman said that it may be particularly useful to utilize NAMs for data poor substances with little to no in vivo data; in these cases, one can approximate the uncertainty in a NAM-based point of departure. She also noted that using the estimated variability in in vivo data can help to estimate NAMs-based approaches as ranges of values that are reasonable, allowing a movement away from the point estimate. Fenton added that data from NAMs can be compared with what is already known using a rich dataset. This could lead to the ability to prioritize emerging compounds in a class, for example.

The moderators also asked about the extent to which the use of LOAEL or LEL contributes to observed variability in results, instead of using a measure such as the benchmark dose. The panelists discussed the role of NAMs in informing uncertainty factors. As Friedman noted, NAMs can further support an examination of vulnerability in the human population, thus informing uncertainty factors. NAMs allow scientists to inform multiple parts of the uncertainty factor, which is critical, she noted.

Kleinstreuer asked the panelists to discuss the challenges of using traditional mammalian toxicity studies for evaluating NAMs. Allen noted that a significant challenge is demonstrating and providing the information for which adequate confidence can be developed in NAMs; it may be appropriate to examine a combination of NAMs instead. Friedman added that NAMs will offer an opportunity to articulate what variability is to a much greater extent, including how much variability is leading to uncertainty. Another issue is the level of confidence in the methodology. If there is strong confidence that the methodology is not highly variable and is reproducible, and that the cellular-level outcomes are relevant to humans, NAMs could help researchers shift away from animal studies. Fenton noted that integrating the results from different approaches (shorter-term animal studies, human studies, and one or more NAMs) would enhance the representation and understanding of variability and increase the confidence and policy application of NAMs. MacLeod noted that one approach is to conduct experiments with different NAMs across various laboratories to examine when heterogeneity is saturated (by calculating the tau squared). From this effort, one can identify a core set of data from a collection of NAMs for further evaluation.

EXAMINING THE CONCORDANCE OF TRADITIONAL MAMMALIAN TOXICITY STUDIES WITH HUMANS

Patience Browne, Organisation for Economic Co-operation and Development, and Nancy Lane, University of California, Davis, moderated a session on understanding the concordance of traditional mammalian toxicity studies with humans.

Thomas Hartung, Johns Hopkins University, discussed the concordance of traditional mammalian toxicity studies with clinical human outcomes. He began by describing studies of pharmaceuticals, which are considered to be the best toxicological assessments. Yet, many fail; in fact, as described in his presentation,¹⁸ the vast majority will fail at the clinical trials phase of testing. This poses a significant challenge for drug development, both in terms of the time and also the cost.

__________________

¹⁸ See https://www.nationalacademies.org/event/12-09-2021/new-approach-methods-nams-for-human-health-risk-assessmentworkshop-1.

Hartung discussed the comparison of human versus animal bioavailability of drugs in studies; the results vary significantly and do not seem to correlate with species, highlighting a key challenge in conducting quantitative risk assessments. Regarding the validity of animal tests, Hartung discussed how difficult it has been for researchers to face criticism about these long-standing methods, limiting the incentive to question and enhance current tools and approaches. The validation of these tests is also expensive and time consuming.

Hartung discussed the process for the formal validation of tests, which can provide regulators the evidence they need to determine if they can trust a new or alternative method. He and other researchers developed a database of more than 10,000 chemicals and through a search of the literature also identified 800,000 related studies of these chemicals. The results indicated that the nine most frequent toxicity tests consume 57% of animals in toxicology.¹⁹ In an examination of studies of chemicals considered reproductive toxicants there was 60% inter-species correlation. Hartung said that he strongly feels that “by acknowledging and quantifying the limitation of animal tests, we can open up the door for new methods.” Hartung noted, for example, a review of studies of side effects in clinical trials conducted by the pharmaceutical industry indicated that rodent studies alone predict about only about 43% of human side effects.²⁰

Joshua Robinson, University of California, San Francisco, presented on the concordance between animal and human toxicology. Studies of concordance provide a valuable system to study many aspects of human development. In fact, he noted that there is high similarity between rat and human embryos during early organogenesis, whereby the morphological and underlying molecular changes are conserved across the neurulation/early organogenesis period.

Robinson also discussed the challenges in examining animal toxicity studies, including the need to carefully consider the choice of animal model, differences in sensitivity and metabolism among species, and selection of the appropriate exposure route and vehicle. Other considerations include whether the chemical target is present within the test species, high to low dose extrapolation approaches, and uncertainty factors. Regarding studies in humans, while there are many advantages, particularly relevance, there are challenges, including difficulty addressing cause and effect, assessing genetically diverse populations, ethical challenges, the need to interpret low dose effects, and the cost of these studies.

As discussed in Olson et al. (2000),²¹ while there are many human toxicants with concordance between animal and human data, this is dependent on the species tested. Also, not all endpoints are equally concordant, as this can vary according to the human target organ. The best concordance identified by Olson et al. (2000)²² was for hematological, gastrointestinal, and cardiovascular endpoints, while musculoskeletal, respiratory, and other endpoints were found to have poor concordance.²³ Environmental chemicals usually require several studies to determine concordance and causation, which can take years to evaluate.

While there are numerous examples of known toxic agents in animals and humans, direct comparisons are difficult and take years to establish causal evidence, Robinson added. Studies examining concordance between animal and human toxicity are lacking. However, the few studies that have been conducted suggest positive concordance (around 70%). Questions remain regarding overall predictive value, species selection, and appropriate endpoints.

Dorman presented the findings from the 2017 National Academies report Application of Systematic Review Methods in an Overall Strategy for Evaluating Low-Dose Toxicity from Endocrine Active Chemicals.²⁴ The committee responsible for the report was tasked with developing a strategy to evaluate the evidence of adverse human health effects from low doses of exposure to chemicals that can disrupt the endocrine system. It was also tasked with conducting systematic reviews of animal and human toxicology data for phthalates and polybrominated diphenyl ethers (PBDEs).

Dorman provided a brief overview of the committee’s examination of phthalates, which are present in a wide range of consumer products and ubiquitous in the environment. The committee focused on male reproductive effects related to phthalates based on in utero exposure, including changes in anogenital distance (AGD), incidence of hypospadias, and lower fetal testosterone concentrations. A systematic review examined in utero exposure and included 13 human studies and 70 animal studies. In its examination of animal studies of di(2-ethyhexyl) phthalate (DEHP) exposure, the committee was able to identify an association between exposure in utero and changes in AGD, but noted discordant results based on the types of rats used in the study.

__________________

¹⁹ Smirnova, L., N. Kleinstreuer, R. Corvi, A. Levchenko, S. C. Fitzpatrick, and T. Hartung. 2018. 3S - Systematic, systemic, and systems biology and toxicology. ALTEX: Alternatives to Animal Experimentation 35(2):139–162. doi: 10.14573/altex.1804051.

²⁰ Olson, H., G. Betton, D. Robinson, K. Thomas, A. Monro, G. Kolaja, P. Lilly, J. Sanders, G. Sipes, W. Bracken, M. Dorato, K. Van Deun, P. Smith, B. Berger, and A. Heller. 2000. Concordance of the toxicity of pharmaceuticals in humans and in animals. Regulatory Toxicology and Pharmacology 32(1):56–67.

²¹ Ibid.

²² Ibid.

²³ Tamaki, C., T. Nagayama, M. Hashiba, M. Fujiyoshi, M. Hizue, H. Kodaira, M. Nishida, K. Suzuki, Y. Takashima, Y. Ogino, D. Yasugi, Y. Yoneta, S. Hisada, T. Ohkura, and K. Nakamura. 2013. Potentials and limitations of nonclinical safety assessment for predicting clinical adverse drug reactions: Correlation analysis of 142 approved drugs in Japan. The Journal of Toxicological Sciences 38:581–598.

²⁴ NASEM. 2017. Application of systematic review methods in an overall strategy for evaluating low-dose toxicity from endocrine active chemicals. Washington, DC: The National Academies Press. https://www.nap.edu/catalog/24758.

A wide range of endpoints for DEHP were considered. The committee had moderate to high confidence in the studies examining AGD, enabling it to make a final hazard call from this endpoint. For the other outcomes, incidence of hypospadias, and lower fetal testosterone, the human data were found to be inadequate to draw conclusions. Regarding concordance between phthalates and AGD, Dorman noted that current testing methods can identify a hazard that is presumed to be of concern to humans but might not be able to accurately predict exposures at which humans are affected.

The 2017 report committee’s evaluation of PBDEs focused on outcomes related to changes in spontaneous motor activity and impaired performance in learning and memory tests in rodents and reduced IQ in children. A systematic review of the literature examined nonhuman mammals and humans. Across studies in rats and mice, the committee noted a significant relationship between PBDE exposure and changes in the latency to complete the last trial of a Morris water maze. Human studies provide a moderate level of evidence that PBDEs are associated with decrements in human IQ, but there was limited evidence to support an association between PBDEs and attention-deficit/hyperactivity disorder (ADHD) in children.

Regarding concordance, Dorman noted that the integration of human data that evaluated measures of IQ and ADHD with animal studies that examined learning, memory, and attention was challenging given the varying endpoints. Additionally, the animal studies used different tests of learning and memory. The test methods and data analyses also often differed between studies and exposures were lower in the human studies. Dorman noted that ultimately the committee found that current mammalian-based testing paradigms could detect a hazard (change in learning and memory) that is presumed to be a concern in humans; however, a comparison of doses between the animal and human studies was challenging and imprecise.

Panel Discussion

Lane and Browne moderated a panel discussion following the presentations. Browne asked the panelists to discuss the challenges and limitations of the use of animal models. Hartung highlighted several challenges, including the fact that the animals used in these studies are often healthy, pathogen free, and are thus not representative of the human patient population. Also, animal studies are often small, with 10–15 animals per study, which is another concern.

Dorman added that the heterogeneity among mammals is another issue, along with genetics and the complexity of diets fed to animals under study. From a regulatory perspective, there is a focus on using uncertainty factors to address variability. While this approach has been useful, as researchers learn more about variability, there is a need to revisit how they consider uncertainty. Another challenge Dorman noted is the disconnect between regulatory studies that follow stringent guidelines, with pre-prescribed outcomes compared to studies in academic labs where outcomes can vary significantly. Reproducing these studies in different labs is a challenge.

Browne asked the panelists to discuss differences that arise between health effects observed in animal versus human studies. There are differences that have been observed, Fenton said, citing the example of per- and polyfluoroalkyl substances (PFAS), where cholesterol and thyroid hormone outcomes associated with exposure differ between humans and animals. In fact, the observed effects may go in opposite directions in humans versus animals. Fenton noted that these differences may not necessarily reflect species differences in toxic response, rather, they may highlight that a sensitive pathway is being affected.

Several endpoints are not often observed in rodents, Dorman said, for example, more subtle neurological endpoints. However, it is important to consider how to explain that concordance in terms of model exposure and its difference from humans. Robinson also echoed these challenges, noting that there are studies looking at rodent versus human stem cells that show drastic differences in terms of sensitivity on a cellular level. If researchers consider these different factors and look at the whole animal, screen the species, and compare with humans, they are going to have difficulty in making comparisons, he added.

Dorman noted that toxicologic studies were historically designed with the goal of preventing catastrophic events and have demonstrated some success. However, their use has become more challenging as researchers develop a more nuanced set of outcomes. He posed the question, are NAMs to be used to prevent a catastrophe or examine a nuanced outcome? Robinson and others discussed the opportunities to incorporate NAMs into technologies that can examine global molecular changes. Dorman and Fenton noted that NAMs could be helpful in improving predictions, especially in the context of analyses across and within large datasets.

Lane asked the panelists to reflect on what biologic factors would be helpful to consider in the qualitative or quantitative extrapolation of results in rodents and animal models. Fenton responded that NAMs could support the analyses of specific chemicals within a larger group of chemicals (e.g., PFAS), informing health effects and risk assessments going forward. Dorman added that the current committee will likely consider the decision context with which NAMs is being used; is it to be used to replace or augment mammalian data? The decision context will drive the answer to that question. The panelists suggested that there is also a need to consider new and better ways to analyze data from experiments; for example, much has been learned in the past three decades about the extent to which variance can impact different endpoints.

FINAL PANEL REFLECTIONS

Kristi Pullen Fedinick, Natural Resources Defense Council, and Corie Ellison, The Procter & Gamble Company, moderated a closing panel discussion focusing on the key reflections from the workshop.

Strengths and Limitations of Traditional Animal Studies

The panelists discussed the strengths and limitations for using animal studies as a gold standard for toxicity testing. Helen Goeden, Minnesota Department of Health, noted that animal data are essential for her work; if the animal data are not adequate to support the derivation of risk-based criteria, it is not possible to take regulatory action. She added that considering multiple durations of exposure is important as historically, there has been a focus on chronic exposure.

Reza Rasoulpour, Corteva, added that the animal models offer strengths for discovering adverse effects but his research has identified positive outcomes in animal species for a metabolite that would never be found in humans. This highlights the importance of toxicokinetic information, a need also supported by Munn. Goeden added that toxicokinetics can change over life stage and by gender and race, a significant gap in the current knowledge.

Utilizing NAMs for Human Health Risk Assessment

The panelists also discussed the role NAMs can play in strengthening risk assessment and identified the key challenges to advancing these approaches. For example, Goeden noted that there is a dearth of data for many chemicals and the hope is to leverage NAMs to address this gap, including around toxicokinetic data. Munn supported this point noting that animal data cannot be collected on all chemicals of interest so there is a need to exploit NAMs to this end. Rashmi Joglekar, Earth Justice, added that NAMs will be a critical tool in prioritizing chemicals, moving chemicals up higher on the list to be evaluated for risk later under other statutes. Goeden agreed that while NAMs will help with prioritization, a quantitative assessment is often still needed and has typically relied on animal data.

Rasoulpour discussed the use of animal studies and NAMs in novel product development, noting that his company has a predictive safety center that uses NAMs to support data on metabolomic endpoints, toxicogenomic endpoints, and toxicokinetics to identify biological points of departure. The center is also able to conduct a full battery of regulatory assays. He has observed through his work that there is significant value in being able to share data used in regulatory decisions more broadly. If NAMs are used to inform future decision-making, analyses using big data techniques would be a key focus.

The panelists also discussed concerns with the use of NAMs. Goeden noted that NAMs may not be able to assess sensitive endpoints. There are also concerns about the variability and instability of cell lines over time, which may affect the comparability of results. Issues regarding model tissue sources and cultures are considerations to be included along with limitations for evaluating certain types of chemicals and particular outcomes (e.g., those related to the thyroid and neurological effects). Goeden added that there is a need for guidance about how to interpret the results of NAMs and how to apply that information. Joglekar noted that animal studies play a critical role in examining complex health outcomes and it will be difficult to replace this with NAMs. The current NAMs tests in the neurotoxicity space can be used to complement current animal tests, but not replace gold standard testing protocols or to establish the safety of chemicals. However, a challenge as researchers incorporate NAMs is to determine if they are defensible and actionable. This may be achieved through demonstration studies. Rasoulpour noted the need to consider differing levels of sensitivity, understanding, and confidence in some of the new science used to support decisions in countries around the world. Bringing different types of datasets into these decisions can be challenging and will benefit from harmonization from the scientific community.

Understanding Susceptibility and Protecting Underserved and Disadvantaged Populations

Ellison asked the panelists whether current toxicity testing and biomonitoring studies are falling short in protecting communities of color and underserved and disadvantaged populations. The panelists noted that the cumulative impacts of mixtures is a key concern in understanding the risks in communities of color; however, little progress has been made in this area with respect to toxicity testing. Munn and others agreed that NAMs may be useful in strengthening the approaches toward the evaluation of multiple chemicals. Joglekar noted the shortcomings of current animal models in protecting susceptible populations and supported the need to broaden testing to examine mixtures of chemicals and address non-chemical stressors. There are opportunities to determine the relative contributions for each of these stressors for adverse outcomes. Addressing human variability is another critical aspect highlighted by Joglekar and others, which cannot be addressed through cell lines derived from single individuals. Rasoulpour added that studies of exposures that affect underserved and disadvantaged populations are complicated by the environmental and socioeconomic stressors faced by these communities. Understanding this complexity is critical to inform interventions, he said. Open science, transparency, and big data can be utilized to make progress on this issue. Dorman noted that toxicology

researchers currently do not design studies with biomonitoring in mind, a critical gap that can limit the applicability of the findings in real-world settings. Fedinick also queried the panelists on the role of uncertainty factors in the discussions around protecting susceptible populations. One opportunity noted by the panelists is to shift risk assessment to probabilistic methods to quantify risk.

Following the final panel discussion, the public offered its comments to the committee and workshop panelists. These comments are not summarized here, but a recording of this public comment session can be found online.²⁵

__________________

²⁵ See https://www.nationalacademies.org/event/12-09-2021/new-approach-methods-nams-for-human-health-risk-assessment-workshop-1.

DISCLAIMER: This Proceedings of a Workshop—in Brief was prepared by Jennifer Saunders as a factual summary of what occurred at the workshop. The statements made are those of the rapporteur or individual workshop participants and do not necessarily represent the views of all workshop participants; the planning committee; the workshop participants’ institutions; or the National Academies of Sciences, Engineering, and Medicine.

COMMITTEE ON VARIABILITY AND RELEVANCE OF CURRENT LABORATORY MAMMALIAN TOXICITY TESTS AND EXPECTATIONS FOR NEW APPROACH METHODS (NAMs) FOR USE IN HUMAN HEALTH RISK ASSESSMENT

WEIHSUEH A. CHIU (Chair), Professor, Department of Veterinary Integrative Biosciences, Texas A&M University; KIM BOEKELHEIDE, Professor (Research) and Professor (Emeritus), Department of Pathology and Laboratory Medicine, Brown University School of Medicine; PATIENCE BROWNE, Hazard Assessment and Pesticide Programmes, Environmental, Health, and Safety Division, Organisation for Economic Co-operation and Development; HOLLY DAVIES, Senior Toxicologist, Washington State Department of Health; CORIE A. ELLISON, Group Scientist, The Procter & Gamble Company; MARIE C. FORTIN, Associate Director of Toxicology, Jazz Pharmaceuticals; NICOLE C. KLEINSTREUER, Acting Director, NTP Interagency Center for the Evaluation of Alternative Toxicological Methods; NANCY E. LANE, Endowed Professor of Medicine, Rheumatology, and Aging Research, Director for the Center for Musculoskeletal Health, University of California, Davis; HEATHER B. PATISAUL, Associate Dean for Research, College of Sciences, North Carolina State University; ELIJAH J. PETERSEN, Staff Scientist, National Institute of Standards and Technology; KRISTI PULLEN FEDINICK, Chief Science Officer, Natural Resources Defense Council; MARTYN T. SMITH, Professor of Toxicology, Kaiser Professor of Cancer Epidemiology, School of Public Health, University of California, Berkeley; ROBYN L. TANGUAY, University Distinguished Professor, Oregon State University; CHRISTOPHER VULPE, Professor, University of Florida, Gainesville; TRACEY J. WOODRUFF, Alison S. Carlson Endowed Professor, Department of Obstetrics, Gynecology, and Reproductive Sciences, University of California, San Francisco; and JOSEPH C. WU, Director, Stanford Cardiovascular Institute, Simon H. Stertzer, MD, Professor of Medicine and Radiology, Stanford University.

STAFF: KATHRYN GUYTON, Study Director, Board on Environmental Studies and Toxicology (BEST); CORRINE LUTZ, Senior Program Officer, Institute for Laboratory Animal Research; and TAMARA N. DAWSON, Program Coordinator, BEST.

REVIEWERS: To ensure that it meets institutional standards for quality and objectivity, this Proceedings of a Workshop—in Brief was reviewed by Vincent Cogliano, California Environmental Protection Agency, and Suzanne Fenton, National Toxicology Program. We also thank staff members David Butler and Jennifer Cohen for reading and providing helpful comments on this manuscript.

SPONSORS: This workshop was supported by the U.S. Environmental Protection Agency.

Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2022. New Approach Methods (NAMs) for Human Health Risk Assessment: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. https://doi.org/10.17226/26496.

Division on Earth and Life Studies

Copyright 2022 by the National Academy of Sciences. All rights reserved.