Review of EPA's Integrated Risk Information System (IRIS) Process (2014) / Chapter Skim
Currently Skimming:
5 Evidence Evaluation
5 Evidence Evaluation
Pages 63-84
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 63... ...
Drawing on approaches developed for systematic reviews in clinical practice and public health, the committee emphasizes the need for EPA to assess the "risk of bias" in individual studies. Accordingly, the best-practice section defines the terms, notes the possibility that bias could arise throughout the conduct and reporting of a study, and discusses how to review studies within a risk-of-bias framework.
|
From page 64... ...
Features that would be assessed for quality and potential risk of bias include study design, study population and targetpopulation setting, participation rate and follow-up, comparability between exposed and control populations, exposure-assessment methods, outcome measures, and data presentation, such as statistical analyses. The handbook also develops similar criteria for assessing the quality of animal studies.
|
From page 65... ...
Overall, EPA (2013a) has identified relevant study attributes to consider in evaluating study quality and indicates its intention to adopt risk-of-bias analyses.
|
From page 66... ...
Here, following the general recommendation of the NRC formaldehyde report and the direction that EPA is taking with its revisions, the committee focuses on the best practices as developed for systematic reviews in clinical medicine and public health. As described above, although EPA has identified and is assessing important characteristics of the quality of human and animal studies, it has not historically conducted the assessments in a consistent and standardized way for studies included in IRIS assessments.
|
From page 67... ...
In observational studies, the potential for confounding is a critical aspect of study design, data collection, and data analysis that needs to be assessed; in randomized clinical trials, confounding is addressed through randomization. Selection bias reflects differences in participant characteristics at baseline or that arise during follow-up and reflects patterns of participation that distort the results from those that would be found if the full 1 The committee defines randomization, allocation concealment, and blinding as follows: randomization is a process that ensures that test subjects are randomly assigned to treatment groups, allocation concealment is a process that ensures that the person allocating subjects to treatment groups is unaware of the treatment groups to which the subjects are being assigned, and blinding (or masking)
|
From page 68... ...
Cochrane Approach The Cochrane Handbook for Systematic Reviews of Interventions includes a tool for systematically assessing the risk of bias in individual studies of the causal efficacy of a health intervention (Higgins and Green 2011)
|
From page 69... ...
It provides little help in systematic review of observational studies, which predominate in human research on the risks posed by chemicals. The Cochrane Handbook (Higgins and Green 2011, Section 8.3.1)
|
From page 70... ...
Evaluation of Tools for Assessing Risk of Bias in Human Studies All the existing tools applicable to assessing risk of bias in a study of human risks require a substantial amount of expert judgment to rate a study's effectiveness in controlling or adjusting for potential bias with methods other than randomization, allocation concealment, or blinding. One challenge in adapting systematic-review methods for environmental-epidemiology studies is the formal consideration of potential confounding and its consequences.
|
From page 71... ...
. Experience gained from randomized clinical trials in human and veterinary medicine suggests that systematic reviews that assess animal toxicology studies for quality and risk of bias would improve the quality of IRIS reviews.
|
From page 72... ...
Moreover, a recent systematic review of tools for assessing animal toxicology studies identified 30 distinct tools for which the number of assessed criteria ranged from two to 25 (Krauth et al.
|
From page 73... ...
. Similarly, several systematic reviews related to disease in animals have noted "a lack of reporting of group-allocation methods, blinding, and details related to intervention protocols, outcome assessments, and statistical analysis methods in some published veterinary clinical trials" (O'Connor et al.
|
From page 74... ...
Invertebrate models, including such flies as Drosophila melanogaster and such nematodes as Caenorhabditis elegans, are increasingly used, especially for mechanistic toxicology studies. Not all animal models are relevant for assessing human health effects, so understanding of the toxicologic end point of interest, concordance between animal and human responses, pharmacokinetic data, and mechanistic information is often key to evaluating the importance of results of an animal study for human health.
|
From page 75... ...
Like bioassays, pharmacokinetic studies need to specify test-chemical stability, chemical exposure route, and exposure-measurement methods. A recent review of mechanistic studies used a systematic approach to identify studies, extract information, and summarize study findings but fell short of assessing risk of bias in the studies (Kushman et al.
|
From page 76... ...
They found that reporting of those aspects in the in vitro studies occurred at much lower rate than that of clinical studies and thus hindered interpretation of reported tissue-culture and cell-culture studies. EVALUATING EVIDENCE FROM INDIVIDUAL STUDIES Assessment of the risk of bias and other methodologic characteristics of relevant studies is a critical part of the systematic-review process.
|
From page 77... ...
FINDINGS AND RECOMMENDATIONS Finding: The checklist developed by EPA that is presented in the preamble and detailed in the draft handbook addresses many of the concerns raised by the NRC formaldehyde report. EPA has also developed broad guidance for the assessment of the quality of observational studies of exposed human populations and, to a smaller extent, animal toxicology studies.
|
From page 78... ...
Likewise, EPA should consider a tool for assessing risk of bias in in vitro studies. Finding: The development of standards for evaluating individual studies for risk of bias is most advanced in human clinical research.
|
From page 79... ...
Finding: Although reviews of human clinical studies have shown that study funding sources and financial ties of investigators are associated with research outcomes that are favorable for the sponsors, less is known about the extent of funding bias in animal research. Recommendation: Funding sources should be considered in the risk-of-bias assessment conducted for systematic reviews that are part of an IRIS assessment.
|
From page 80... ...
Finding: EPA has not developed procedures that describe how the evidence evaluation for individual studies will be incorporated, either qualitatively or quantitatively, into an overall assessment. Recommendation: The risk-of-bias assessment of individual studies should be carried forward and incorporated into the evaluation of evidence among data streams.
|
From page 81... ...
2011. Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0.
|
From page 82... ...
2008. Systematic reviews of animal experiments demonstrate poor contributions toward human healthcare.
|
From page 83... ...
2006. A review of randomized clinical trials reporting antibiotic treatment of infectious bovine keratoconjunctivitis in cattle.
|
From page 84... ...
2010. Can animal models of disease reliably inform human studies?
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.