The Use of Systematic Review in EPA's Toxic Substances Control Act Risk Evaluations (2021) / Chapter Skim
Currently Skimming:
2 Evaluation of the TSCA Systematic Review Approach
2 Evaluation of the TSCA Systematic Review Approach
Pages 14-51
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 14... ...
. Methods have also been proposed for applying systematic review methods to risk evaluations for ecological receptors in a framework integrated with human health risk evaluations (Suter et al.
|
From page 15... ...
To determine the approach to systematic review being used within the Toxic Substances Control Act (TSCA) risk evaluations, the committee reviewed the Draft Risk Evaluation for Trichloroethylene (TCE)
|
From page 16... ...
The committee also considered whether OPPT followed the appropriate agency guidelines for these evidence streams (i.e., Guidelines for Human Exposure Assessment, the Guidelines for Ecological Risk Assessment, and the operating procedures for the use of the ECOTOX knowledgebase [EPA 1998, 2019b, 2020b]
|
From page 17... ...
Another crosscutting finding, discussed in detail in the evidence integration section of this chapter, relates to terminology and specifically to the interchangeable use of the terms "weight of evidence" and "systematic review." The Procedures for Chemical Risk Evaluation under the Amended Toxic Substances Control Act, referred to as the "Risk Evaluation Rule" (40 CFR Part 702) , specify that weight of the scientific evidence "means a systematic review method." However, this definition may not be intended to mean a systematic review as defined by the IOM.
|
From page 18... ...
. The risk evaluation process is typically much broader than that of systematic review alone, and generally not every stream of evidence is evaluated using systematic review, while other evidenced-based approaches may be used.
|
From page 19... ...
Scoping reviews can be extremely useful in identifying the reach and breadth of a systematic review prior to developing a research question. Assessing the breadth of the body of literature before defining the research question can allow for selection of specific endpoints, evidence streams, routes of exposure, or developmental stages.
|
From page 20... ...
The protocol is a detailed plan or set of steps that should describe the methods that will be used to conduct all the steps of the systematic review from evidence identification through evidence synthesis. Protocols are critical in de novo systematic reviews and can also be used in other evidenced-based literature searching methods, such as scoping reviews and narrative reviews.
|
From page 21... ...
. Committee Description of the Approach in TSCA Risk Evaluations OPPT is using a variety of software tools and approaches to conduct broad searching and to map the available evidence.
|
From page 22... ...
As with the problem formulations for hazard assessment, the problem formulation, scoping, and data collection processes are also merged for these streams. The determination of what exposures are relevant to the risk evaluation is dictated by the "conditions of use." "For purposes of prioritization, the Administrator may determine that certain activities fall outside the definition of ‘conditions of use.' During the risk evaluation scoping process, EPA may decide to narrow the scope of the risk evaluation further, potentially excluding conditions of use that present low risk."4 For example in the TCE scope documents, all indoor studies of exposure are included and evaluated, but in the final risk evaluation, measurements of indoor home exposures were determined not to be related to a specific condition of use and thus were not included in determining exposures.
|
From page 23... ...
TABLE 2-1 Health Hazard Assessment PECO Statement from the 1-BP Risk Evaluation PECO Element Evidence Stream Features Included Population Human Any population All life stages Study designs: – Controlled exposure, cohort, case-control, cross-sectional, case crossover – Case studies and case series that are related to deaths from acute exposure Animal All non-human, whole-organism mammalian species All life stages Exposure Human Exposure based on administered dose or concentration of 1-BP, biomonitoring data (e.g., urine, blood or other specimens) , environmental or occupational-setting monitoring data (e.g., air, water levels)
|
From page 24... ...
Critique of the TSCA Approach Looking at the core review elements of the Statement of Task, which is to address whether the TSCA approach to systematic review is "comprehensive, workable, objective, and transparent," the committee finds that the approach to problem formulation and protocol development could be improved broadly to better meet these characteristics. BOX 2-2 PECO Statement for General Exposures in the TCE Risk Evaluation Population • Human: Consumers (i.e., receptors who use a product directly)
|
From page 25... ...
The ill-defined questions within TSCA risk evaluations hinder the necessary prespecification of systematic review methods, notably the eligibility criteria for studies. Failing to adequately refine the focus of a systematic review leads to overly broad questions, in turn leading to the identification of heterogeneous studies, to more complicated analysis, and to challenges in integrating across evidence streams to draw conclusions.
|
From page 26... ...
It is not well documented in any of the risk evaluations or related scope documents reviewed for this report, and procedures for problem formulation are not included in Application of Systematic Review in TSCA Risk Evaluations (herein 2018 guidance document)
|
From page 27... ...
A systematic review may not be required for every stream of evidence that is part of a risk evaluation. The full problem formulation and understanding of the litera ture base for an evaluation should allow OPPT to determine which research questions may be evaluated with a systematic review and which questions should be evaluated with a different evidenced-based approach.
|
From page 28... ...
, and the approach described for searching for evidence for other streams is similar to the hazard assessment. COMMITTEE DESCRIPTION OF THE APPROACH IN TSCA RISK EVALUATIONS Searching for the Evidence OPPT uses exhaustive search strategies that include major scientific databases, backward searching for studies in previous chemical risk assessments, additional gray literature sources, studies submit 5 See www.prisma-statement.org, accessed November 13, 2020.
|
From page 29... ...
Addi TABLE 2-2 Search Strategies and Terms in TSCA Risk Evaluations NOTE: ATSDR, Agency for Toxic Substances and Disease Registry; CAS RN, CAS Registry Number; ECOTOX, ECOTOXicology; EPA, U.S. Environmental Protection Agency; NHANES, National Health and Nutrition Examination Survey; NIH, National Institutes of Health; NIOSH, National Institute for Occupational Safety and Health.
|
From page 30... ...
In addition, EPA ORD has advanced adverse outcome pathway (AOP) conceptual models to support mechanistic ecotoxicology data integration within risk evaluations.
|
From page 31... ...
FIGURE 2-4 Committee's interpretation of the OPPT approach to identifying and selecting evidence in the TCE and 1-BP risk evaluations.
|
From page 32... ...
Comprehensive TSCA risk evaluations include searches for evidence in most major scientific databases, backward searching for studies in previous chemical risk assessments, additional gray literature sources, studies submitted under TSCA, and studies identified in peer review (see Figure 2-3)
|
From page 33... ...
incidence, prevalence report) a Some of the studies that are excluded based on the PECO statement may be considered later during the systematic review process.
|
From page 34... ...
Recommendations In order to improve these issues with OPPT's approach to evidence identification, the committee recommends the following: • Registering the protocol for each risk evaluation is important: That protocol should include an explicit search strategy, and search strategies for each database should be consistently listed in the appendix to the risk evaluation. • OPPT could improve the evidence identification process by requesting information from man ufacturers, such as ingredients for products, and from organizations that have provided data previously during the peer-review stage.
|
From page 35... ...
The most appropriate method to exclude studies from evidence synthesis is based on predefined exclusion criteria that should preclude an irrelevant study from being evaluated. Although there is not a specific standard of practice for evaluating exposure data, the agency Guidelines for Human Exposure Assessment discuss the importance of critically reviewing data for use in an exposure assessment.
|
From page 36... ...
Ring test participants perceived it to be less dependent on expert judgment, more accurate and consistent, and practical regarding the use of criteria and time needed for performing an evaluation. Committee Description of the Approach in TSCA Risk Evaluations OPPT has developed an extensive de novo critical appraisal tool, termed TSCA's "fit-for-purpose evaluation framework," which is applied to human, animal, ecological receptors, mechanistic, exposure, fate, and physical chemical property studies.
|
From page 37... ...
; organism, outcome assessment, toxicity to algae, Cyanobacteria, and other confounding/variable control, and microorganisms; toxicity to terrestrial data presentation and analysis invertebrates; acute oral toxicity to birds; toxicity to reproduction of birds; toxicity to terrestrial plants; toxicity to mammalian wildlife Animal and in vitro toxicity studies Animal: oral, dermal, and inhalation routes; Test substance, test design, lethality, irritation, sensitization, exposure characterization, test reproduction, fertility, developmental, organism/test model, outcome neurotoxicity, carcinogenicity, systemic assessment, confounding/variable toxicity, metabolism, pharmacokinetics, control, and data presentation and absorption, immunotoxicity, genotoxicity, analysis mutagenicity, endocrine disruption In vitro: irritation, corrosion, sensitization, genotoxicity, dermal absorption, phototoxicity, ligand binding, steroidogenesis, developmental, organ toxicity, mechanisms, high throughput, immunotoxicity Epidemiological studies Controlled exposure, cohort, case-control, Study participation, exposure cross-sectional, case-crossover characterization, outcome assessment, potential confounding/variability control, analysis SOURCE: Derived from EPA 2018a.
|
From page 38... ...
Yet, the committee found that although the 2018 guidance document discusses the use of PBPK models in risk assessments, and that OPPT will use evaluation strategies for animal and in vitro toxicity data to assess the quality of the data supporting the model, the document does not give guidance as to how these models will be evaluated. The committee could not find evidence of this practice being followed in the draft TCE risk evaluation document and supplemental materials.
|
From page 39... ...
The committee notes that many public comments also discussed these problems with using numeric scores to evaluate studies. The committee notes that completing the detailed evaluations of each study that may be included with risk evaluation is time consuming.
|
From page 40... ...
Following evidence synthesis, evidence integration across multiple evidence streams -- an essential step in the TSCA risk evaluation process -- is done but is not a part of the traditional systematic review process. There is much research and a growing consensus on how certainty in a body of evidence should be determined in the field of toxicology.
|
From page 41... ...
in risk evaluations is the SSD Toolbox.8 Committee Description of the Approach in TSCA Risk Evaluations Section 3.4 of the 2018 guidance document does not separate evidence synthesis from evidence integration and instead combines the two into a single step of data integration (EPA 2018a, p.
|
From page 42... ...
It finds that, in the absence of an explicit definition of an evidence stream, it is difficult to assess whether OPPT should follow a systematic review approach for evidence synthesis and, if so, how evidence synthesis should be conducted. Making that judgment is further complicated by OPPT's merger of evidence synthesis within a stream and the step of evidence integration across streams.
|
From page 43... ...
For cancer endpoints, the TCE risk evaluation included a meta-analysis to synthesize evidence, combining three cancers -- non-Hodgkin lymphoma, kidney cancer, and liver cancer -- in human studies. Although OPPT followed a systematic review approach for evidence from mechanistic models, the number of mechanistic studies was limited and the evidence synthesis was narrative, concluding that a genotoxic mode of action is highly plausible for kidney cancer.
|
From page 44... ...
Workable Without a clear definition of evidence streams or documented approaches to evidence synthesis within each evidence stream, it is difficult to assess and reproduce the results of evidence synthesis. Concurrent implementation of evidence synthesis and evidence integration further makes evidence synthesis approaches of TSCA risk evaluations less workable.
|
From page 45... ...
Transparent The absence of a well-documented protocol reduces transparency and consistency of evidence synthesis. Lack of documentation and justification for the use of average modeling or composite risk modeling for cancer dose-response assessment in the evaluation of 1,4-Dioxane and TCE raises concern about consistency in TSCA risk evaluations.
|
From page 46... ...
. Committee Description of the Approach in TSCA Risk Evaluations According to Table 3-1 in the 2018 guidance document, the evidence integration step has three phases.
|
From page 47... ...
The presentation of Drs. Barone and Wong, titled "Evidence Integration Supporting Exposure and Hazard Assessments for TSCA Risk Evaluations" (Stanley Barone and Eva Wong, presentation to the committee, July 23, 2020)
|
From page 48... ...
(B) Considerations for evidence integration within TSCA risk evaluations.
|
From page 49... ...
All exposures were separately evaluated in the risk assessment. Critique of the TSCA Approach The committee has not found the 2018 TSCA systematic review document, the several presentations made to the committee by OPPT, or the 1-BP and TCE evaluations sufficiently detailed to provide the information needed to assess the methodology and appropriateness of the framework for evidence integration.
|
From page 50... ...
The term "data integration" is mentioned in several of the other evidence streams as well, including exposure and environmental fate, but it is not clear as to how the data were selected or integrated into the assessment. Significant improvement in transparency and consistency is needed to fully understand EPA's process of evidence integration within this assessment.
|
From page 51... ...
• As has been noted in the committee's reviews of the completed TCE and 1-BP risk evaluations, there is significant lack of clarity in the language used to describe the integration process, for both hazard and exposure, and the lack of clarity is not limited to the evidence descriptors. It is difficult to understand why and how certain steps were taken, particularly regarding the inclu sions and exclusions of certain data.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.