Skip to main content

Currently Skimming:

3 Planning Assessments
Pages 33-51

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 33...
... assessment: • Chapter 1––Scoping of IRIS Assessments • Chapter 2––Problem Formulation and Development of an Assessment Plan • Chapter 3––Protocol Development for IRIS Systematic Reviews • Chapter 4––Literature Search, Screening, and Inventory • Chapter 5––Refined Evaluation Plan • Chapter 7––Organizing the Hazard Review: Approach to Synthesis of Evidence • Chapter 10––Analysis and Synthesis of Mechanistic Information In reviewing those chapters, the committee considered whether the systematic review approaches used by the IRIS Program (outlined in Chapters 1-5 of the handbook) clearly describe and are consistent with methodologies considered to be state of the science (Question 2 in Appendix B)
From page 34...
... Chapter 3 "Protocol Development for IRIS Systematic Reviews" briefly describes how the IRIS program interprets the concept of a protocol for the conduct of a systematic review. Breaking with convention in systematic review terminology, the term "protocol," as used here, is a strategic document outlining an overall process for developing a planned approach to an overall IRIS assessment, rather than a detailed methodology for conducting a particular systematic review.
From page 35...
... While the handbook describes several steps for focusing IRIS assessments on specific health outcomes, as described further below, the process is not fully transparent in a manner consistent with conventional systematic reviews. Additionally, while the process outlined in the handbook is responsive to the NRC (2014, p.
From page 36...
... The outcomes may be particularly broad, typically encompassing "all cancer and noncancer health outcomes." This contrasts with the types of PECO statements used in systematic reviews (including those developed by the National Institute of Environmental Health Science's Office of Health Assessment and Translation and using the Navigation Guide [Woodruff and Sutton, 2014] for environmental health)
From page 37...
... Study evaluation is shown in a dashed box because it is not covered in this chapter but is positioned before "Organizing the Hazard Review: Approach to Synthesis of Evidence" in the handbook.
From page 38...
... It also positions the literature search and inventory development as occurring after the development of the systematic review protocol, when in fact these two elements have a fundamental role in the development of the protocol. As discussed later in this chapter, the protocol itself seems ambiguously characterized as a product of the planning process of an assessment, and in a manner at odds with conventional understanding of its use in systematic reviews.
From page 39...
... Protocol Development This section discusses the processes described in the handbook for developing and making publicly available, in the form of a protocol, the planned objectives and methods for conducting an assessment. In a systematic review, the protocol is a complete account of planned methods that should be registered (i.e., publicly released in a time-stamped, read-only format)
From page 40...
... , the steps subsequent to the literature inventory are at this stage fairly generic. While this protocol includes a broad PECO statement, the summary box "Organization of the Protocol" on page 3-3 of the handbook presents a level of detail more readily associated with a formal systematic review protocol and seems to better match the intent of the final protocol.
From page 41...
... The development of refined PECO statements for each health effect category at the assessment protocol stage would therefore provide clarity as to how specific pieces of evidence are included or excluded, as well as transparency as to any subsequent changes to those criteria. The committee recognizes that documenting the planning process for IRIS assessments is complex.
From page 42...
... Specifically, a broad PECO statement that was used to develop a literature inventory was included in the Vanadium planning documents (EPA, 2020b) released on July 1, 2020, for public comment, and the results of this inventory were subsequently described in the Systematic Review Protocol (EPA, 2021a)
From page 43...
... FIGURE 3-2 Illustration of an assessment plan, literature inventory, and assessment protocol from the IRIS planning process. Note: The figure illustrates which of the three products might be updated or registered, and where the products would feed into the IRIS assessment process.
From page 44...
... Literature Study database and categorization Initial inventory • Updated periodically Inventory • Health effects made publicly (not for public comment) • Susceptible populations available alongside • Inventory for public assessment protocol comment available with • Mechanistic and TK draft for public public comment/peer information comment review draft of IRIS assessment • Final inventory published alongside final IRIS assessment Assessment • Systematic review protocols Draft for public • Revised version, Protocol (including refined PECOs comment, published incorporating changes specifying units of analysis; alongside initial based on public methods for study evaluation literature inventory comments, registered and analysis)
From page 45...
... It is also unclear when and how additional mechanistic data would enter the data organization process. For example, page 7-3 of the handbook states that "additional targeted searches for mechanistic information specific to those health effects and/or organ systems may need to be performed.
From page 46...
... In other cases, mechanistic data may warrant their own PECO statement (or set of PECO statements) because it is appropriate and possible to evaluate their strength of evidence using systematic review methods.
From page 47...
... Thus, a key benefit of organizing mechanistic data using KCs is that they can be readily used in evidence integration to transparently address biological plausibility. As discussed in NASEM (2017b)
From page 48...
... Having this information identified in the literature inventory could support the refinement of the assessment plan and later steps in the process, including dose-response assessment. Recommendation 3.3: The literature inventory process should consider tagging studies for the presence of data pertinent to identification of susceptible populations.
From page 49...
... . PECO statements in clinical systematic reviews often employ tiering of outcomes, for instance using hierarchies of broader to narrower categories, distinguishing between "critical" and "important" outcomes, and flexibility to group outcomes (e.g., see Higgins and Thomas, 2019, Section 3.2.4.3)
From page 50...
... Finding: This step deviates from best practices in conventional systematic reviews, where all outcomes are prespecified and not subject to change after study evaluation. The handbook does not provide sufficient justification for revisiting the design of the systematic review after study evaluation.
From page 51...
... At a minimum, all endpoints that may be used for toxicity values, including so-called "precursor" endpoints that might be viewed as "mechanistic," should require separate PECO statements; however, ap plication of systematic review methods to other mechanistic endpoints, such as mu tagenicity, may depend on the needs of the assessment. The key mechanistic and TK questions should be identified to the extent possible in the IRIS assessment plan and IRIS assessment protocol.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.