Review of EPA's Updated Problem Formulation and Protocol for the Inorganic Arsenic IRIS Assessment (2019)

• The modeling approaches to investigate dose-response relationships for cancer and noncancer end points, and
• Other aspects of the protocol and associated documents.

The committee’s deliberations on these topics were informed by a public data-gathering session on July 16, 2019. EPA gave presentations on the IRIS iAs assessment plan (Davis and Gift 2019; Lee et al. 2019a,b; Thayer 2019) and presented posters (https://www.epa.gov/iris/inorganic-arsenic-meetings-webinars) that provide supplemental information. The committee asked questions of EPA during the presentations, and had probing discussions with EPA staff during the poster sessions. The committee also received comments from stakeholders during the open microphone sessions of the public meeting, reviewed written submissions¹ provided during the course of the study, and reviewed a summary of comments that were received by EPA (2019b).

COMMITTEE’S FINDINGS

Overall, the majority of the committee found that the approaches outlined in EPA’s IRIS iAs assessment plan and supplemental materials address many of the recommendations made by the 2013 NRC committee. EPA has implemented systematic review approaches to evaluate the scientific literature; has prioritized the health outcomes for dose-response assessment; has presented an approach to assessing multiple cancer and noncancer end points; has begun to implement several analytical approaches to utilize human data in the low-dose range; and has introduced more advanced statistical approaches in its dose-response evaluations that take into consideration multiple studies. The committee was unable to reach consensus about the adequacy of the dose-response methods; the majority of the committee supported EPA’s approach and one member objected to this conclusion (see Appendix A for dissenting statement and committee rebuttal). The committee offers some observations and recommendations for completing the iAs IRIS assessment.

Transparency and Communication

The EPA (2019a) document is described as a “protocol,” a term the committee found confusing. The term “protocol” has certain connotations, especially with respect to systematic review, namely, that the methods and criteria are specified in advance of performing the review. Because EPA’s document outlines methods for completed work and approaches still in development, it would be better characterized as an “assessment plan.” This nomenclature will help alleviate confusion and public concerns about the conduct of the assessment. Henceforth, this report will refer to the EPA (2019a) document as the IRIS iAs assessment plan.

The committee found that if the IRIS iAs assessment plan was reviewed in isolation, without the benefit of the posters and interactions with EPA staff at the public meeting, the members would have had many more questions and concerns about the methods. This problem was also evident in the public comments, which were submitted prior to the public meeting. EPA indicated that more detail about the methods will be available in other documents that it is preparing for publication, as well as in the IRIS Handbook. These materials were not available for the committee’s review. It will be important that they be in the public domain as soon as possible or otherwise included in the documentation when the iAs IRIS assessment is completed.

EPA’s Approach to Prioritizing Health Outcomes

The committee’s understanding was that EPA used a stepwise process for prioritizing health outcomes associated with iAs. EPA started with the NRC (2013) report for identifying priority outcomes, and then

___________________

¹ Comments are available upon request from the National Academies Public Access Records Office at paro@nas.edu or (202) 334-3543.

made judgments about the strength of the evidence of each outcome based on additional information. For four outcomes (lung cancer, bladder cancer, skin cancer, and skin lesions), EPA relied on its prior IRIS assessment and authoritative reviews by other organizations to judge that these end points had robust strength of evidence of a causal association. For the other 14 outcomes, the agency performed systematic reviews of the evidence to make updated judgments (robust, moderate, or slight) about priorities. The committee agrees with EPA’s approach to prioritizing the health outcomes and its decision to focus dose-response analyses on the outcomes with robust or moderate evidence of a causal association. The committee notes that scoping-review approaches appear to have been conducted to support the systematic reviews and encourages EPA to provide more clarity about those approaches and generally to use scoping reviews as a first step in problem formulation, refining of research questions, and selection of outcomes. For some categories of noncancer outcomes, the committee recommends that EPA consider specifying the end points more precisely. For example, lower respiratory infections could be a specific immune outcome and IQ could be a specific end point of neurodevelopmental toxicity. Furthermore, EPA should consider pregnancy-related hypertension and diabetes within the category of pregnancy outcomes.

EPA’s IRIS iAs assessment plan also included a description of how it determined relative risk to background exposure (RRB) to make decisions about selecting outcomes or end points for dose-response analysis. The committee encourages consideration of exposure in guiding decisions within the risk assessment. However, it was unclear how the RRB evaluation was ultimately used in selecting outcomes or end points for dose-response analysis. Clearer documentation of how the RRB was used in the decision-making process should be provided.

The committee also offers some editorial comments on the presentation of the prioritization approach in the tables presented on pp. 9 and 46 (EPA 2019a, Table 2-2, Table 3-3²). First, one table would suffice for presenting information about the priority end points and whether dose-response analyses were performed. The committee suggests that “health outcomes of concern” be referred to as “health outcomes” because the systematic review approach depends on the robustness of the iAs literature and is not necessarily reflective of agency rankings of importance. Thus, “priorities of concern” gives the impression that the health end points are being weighed by importance, rather than on the robustness of the literature. The column referring to the NRC tiers is unnecessary. Finally, because health outcomes that are classified as having slight strength of evidence are automatically excluded from dose-response analysis, explanations for exclusion at this stage are only necessary for those with stronger evidence bases.

EPA’s Systematic Review Methods

Overall, the committee commends EPA’s integration of systematic review in the IRIS iAs assessment plan to identify evidence and to evaluate evidence for hazard identification and subsequent use in dose-response analyses. Such methods offer greater transparency, rigor, and better use of the overall evidence base. EPA, however, did not create a protocol a priori for each of the systematic reviews that it performed, which is a standard practice (IOM 2011; NRC 2014; NASEM 2017). The reasons for not doing so should be addressed in the iAs IRIS assessment, as well as specific details regarding which elements were pre-specified (i.e., eligibility criteria for systematic reviews). Specific comments related to the appropriateness of the other elements of systematic review are discussed below.

Populations, Exposures, Comparators, and Outcomes (PECO) Criteria

EPA used a PECO-structured approach to problem formulation and the literature search. This approach directly translates into increased transparency that is carried throughout the assessment. The differentiation as to how the PECO criteria were applied throughout the process was helpful. The following considerations will increase clarity in the iAs IRIS assessment:

___________________

² The table number in the IRIS iAs assessment plan should be corrected to be Table 5-3.

• Provide a research question for each systematic review.
• Clarify whether literature searches were restricted by outcomes.
• Clarify apparent discrepancies in eligibility criteria. For example, it is unclear how life stages were considered in the inclusion criteria. The PECO statement (EPA 2019a, p. 15) specifies for the post-2017 assessment that all human life stages are included for consideration, but life stages are subsequently described as “potentially relevant supplemental materials” (EPA 2019a, pp. 19-20). Another example is the categorization and inclusion of meta-analyses; it is unclear how existing meta-analyses were considered.

Literature Search and Screening

The screening process described in the IRIS iAs assessment plan is generally consistent with standard systematic review methods. The following are recommended to increase clarity in the iAs IRIS assessment:

• A more complete diagram (consistent with PRISMA [Preferred Reporting Items for Systematic Reviews and Meta-analyses] [Moher et al. 2009]) that represents the process for how the entire evidence base was identified should be included. For example, the final number of studies included for each outcome should be specified.
• EPA should ensure consistency between reporting platforms (HERO [Health & Environmental Research Online], the IRIS iAs assessment plan, and the iAs IRIS assessment) as it relates to categories (tags) of studies (e.g., included/excluded, potentially relevant supplemental categories). It is currently difficult to utilize HERO to identify those studies that were included, excluded, or categorized as potentially relevant. This will address apparent inconsistencies in the number of studies reported across the IRIS iAs assessment plan, the presentations, and in HERO at the time of the committee’s review.
• EPA should identify the excluded studies and the reasons for exclusion. It should also be clear how data from studies of the same population or cohort were handled in the systematic review.
• Clarify when literature searches are updated and how new data are handled for both hazard identification and dose-response analysis.

Data Extraction Procedures

The IRIS iAs assessment plan provided details on the procedures for extracting data from each included study, such as the tool used, the use of sequential review, and how missing data were handled. The appendix of the plan provided details on specific fields/items extracted. The committee supports the approach taken and the reporting of these methods.

Study Evaluation

EPA conducted risk-of-bias assessments following standard systematic review methodology (i.e., two independent assessments) and adapted a risk-of-bias tool developed by the National Toxicology Program for environmental studies. Provision of the risk-of-bias considerations (Table 3-2 in the IRIS iAs assessment plan; see EPA 2019a) and the intent to have assessment-specific adaptations and clarifications are appropriate. The committee notes, however, that Appendix C of the IRIS iAs assessment plan seems to have been copied from an assessment of bisphenol A and is not specific to iAs. The correct set of rating guidelines and clarifications and risk-of-bias judgments for each study should be made available in support of the iAs IRIS assessment.

The IRIS iAs assessment plan explains that the risk-of-bias judgment for an individual study is combined with “conclusions about sensitivity” to rate confidence in a study as high, medium, low, or uninformative. How sensitivity, the ability of a study to detect a true effect, is appraised for each study was

unclear. Also unclear was how the risk-of-bias assessments and sensitivity judgments were combined to rate confidence in each study. In particular, more detail is needed to understand how studies were judged as “uninformative,” as these studies were excluded from further analyses. A full record of the confidence ratings for each study and how they were determined should be available to support the iAs IRIS assessment. Any topic-specific refinements to the risk-of-bias or sensitivity criteria should also be made clear.

Studies with a low confidence rating (called low quality elsewhere in the IRIS iAs assessment plan; see EPA 2019a, p. 13) were excluded from any further consideration “unless they are the only studies available.” Generally, a study should be considered for inclusion based on its merits and relevancy, not based on what is available. Any fatal flaws in the methodology or conduct that precludes including a study would ideally be used as eligibility criteria during the screening process. Once a study is determined to be eligible, the study could be included in the synthesis and the risk-of-bias assessment and its limitations accounted for in any qualitative or quantitative synthesis. It is recognized, however, that additional elements of study design and reporting were important to the dose-response analysis, thus emphasizing the need to both clearly state appraisal criteria applied throughout the assessment and provide documentation for such criteria as applied to all eligible studies.

Evidence Synthesis and Integration

The iAs IRIS assessment should have supporting information that describes how the epidemiological information was synthesized to draw conclusions about hazard. For example, whether EPA considered performing meta-analyses for any of the outcomes to support hazard identification is unclear.

Although the term “evidence integration” is used in the IRIS iAs assessment plan, it does not appear that separate lines of evidence (human, animal, and mechanistic) were integrated to draw conclusions about hazard. A clear definition of integration and the methods for it should be included.

The scheme for assigning strength of evidence was described as based on the Bradford Hill recommendations for making judgments about causal association and GRADE (Grading of Recommendations Assessment, Development and Evaluation). The considerations in Table 3-4 were used to inform judgments about causality and the considerations in Table 3-6 were used to inform judgments about the strength of the human evidence. Whether a causality determination is made before assessing the strength of evidence or whether all factors (i.e., causality and strength of evidence) are considered simultaneously was not clear. More information on this process is needed, and the committee recommends the following elements to improve the clarity of the judgments:

• Separate the findings and conclusion about causality from the judgments about strength of evidence or confidence in those findings.
• Separate the different concepts of magnitude of effect and precision (i.e., the summary estimate or range of effects and the confidence interval).
• Do not use p-value as a measure of precision (described in Table 3-4).
• Clarify the apparent overlapping domains in Table 3-6. For instance, the risk-of-bias domain (first row) considers the “confidence” rating that earlier was described as combining risk-of-bias judgments with study “sensitivity judgments” but the next row specifically includes a separate domain for study sensitivity.

EPA’s Consideration of Potential Health Effects from Early Life Exposures

There are two primary issues related to early life exposure to iAs. The first is in utero exposure to iAs and children’s health end points. The second is windows of vulnerability during any stage of development that lead to adverse health effects later in life. Regarding the first issue, the IRIS iAs assessment plan includes neurodevelopmental toxicity and birth outcomes, which were judged by EPA to have moderate or robust evidence for causation, respectively, in the hazard assessment. The majority of childhood studies

rely on continuous outcome measures to assess chemical toxicity (e.g., IQ for developmental toxicity, birthweight for gestational age), because such studies can be more readily conducted prospectively in children than in adults (e.g., fetal/infancy/childhood exposure can be linked to IQ or other developmental tests in a relatively short time span). EPA is currently in the process of developing meta-analytic approaches to model data on continuous health outcomes.

The second issue is on the role of early life exposures as a period of vulnerability to iAs and adult health end points. Life stage susceptibility is addressed in the MOA sections but a plan to assess it and consider it in the hazard assessment and dose-response analysis was not clearly presented. A large body of literature supports the concept of developmental origins of child and adult diseases (e.g., NRC 2013; Steinmaus et al. 2014), thus early life iAs exposure might predict adult health outcomes more strongly than later childhood or adult iAs exposures. The IRIS iAs assessment plan does not indicate whether or how dose-response modeling of early life exposure to iAs and adult health end points will be performed.

Mode-of-Action Information to Inform Dose-Response Analyses

On the basis of the information presented in the IRIS iAs assessment plan, EPA’s posters, and discussions with EPA staff, the committee’s understanding is that primarily epidemiological data will be used in dose-response modeling in the iAs IRIS assessment. EPA has found that the epidemiological studies published since the NRC (2013) report have increased the amount of low-dose data on iAs health effects, including exposures at or approaching background levels in the United States. The agency concluded that MOA data were of limited utility for quantitatively characterizing the low-dose response relationship for cancer and noncancer end points when such data are available. Dose-response models that combine data from multiple epidemiological studies will be used by EPA to determine the shape of the dose-response curve. The models will include the full range of exposures, including lower exposures, and a wide variety of dose-response shapes will be considered through the use of fractional polynomials. The majority of the committee agrees that MOA information will not contribute directly to determining the shape of the dose-response curve based on epidemiological data (see Appendix A for dissenting statement from one member and committee rebuttal).

The committee offers the following observations and recommendations that should be considered in completing the IRIS assessment of iAs:

• Confusion and concern about EPA’s rationale for not using MOA data were clear from the public comments. There appears to be a misperception that this decision means that EPA would not consider the possibility of threshold effects, and will assume a linear dose response in the low-dose region. Thus, the committee recommends that the iAs toxicological assessment clarify that the statistical approaches used will make no assumptions about the shape of the curve.
• Appendix A in the IRIS iAs assessment plan presents information about multiple MOAs that might be involved in cancer and noncancer outcomes that was used for EPA’s preliminary analyses as summarized in the main text. How this MOA information will be used in other aspects of the iAs assessment is not clear. MOA information would be useful for evaluating biological plausibility, identifying sensitive populations or life stages, evidence integration to support hazard conclusions, and in situations where the data on a particular end point do not include exposure to iAs near background levels and it will therefore be necessary to perform dose-response extrapolations. It will be important for EPA to describe how MOA information is used by presenting the data in a better organized and synthesized manner for each context in which the information is used, given the complex and large amount of mechanistic studies with diverse in vitro and in vivo animal methodologies. If EPA will not be applying low-dose extrapolation methods, presentation of MOA information that is not directly used in the iAs IRIS assessment may not be necessary or helpful to the dose-response analysis section (but may be better utilized in the discussion of biological plausibility, for example).

Hierarchical, Bayesian Meta-Regression Approach

The majority of the committee supports EPA’s innovative application of advanced statistical methodology to incorporate the full range of epidemiological data on iAs and health outcomes (see Appendix A for dissenting statement from one member and committee rebuttal). On the basis of the IRIS iAs assessment plan and information provided during the public meeting with EPA, the dose-response meta-analysis is the preferred method of analysis when sufficient data are available. When there are insufficient studies to support a meta-analysis, EPA will analyze dose-response data from individual studies and use Bayesian model averaging (BMA) to derive a benchmark dose estimate. If BMA cannot be performed, EPA will default to the traditional approach of selecting a single dose-response model based on fit (e.g., through a comparison of Akaike information criteria [AIC]). The committee also commends EPA’s plan to perform sensitivity analyses of its prior distribution on the slope in the dose-response meta-analysis, which addresses concerns raised in some of the public comments.

Although the majority of the committee agrees with the IRIS iAs assessment plans for performing dose-response analyses, the following concerns should be addressed in the iAs IRIS assessment:

• EPA should clarify its methods and terminology. It appeared to the committee that EPA performed a dose-response meta-analysis and not a meta-regression. A meta-analysis is the statistical combination of results from separate studies, and a meta-regression is a method for performing subgroup analyses to evaluate heterogeneity among the studies in the meta-analysis (Greenland and Longnecker 1992; Egger et al. 2001).
• The specific criteria for excluding a study from the meta-analysis were not provided. It is important to distinguish between studies not meeting technical criteria for inclusion (e.g., insufficient number of categories, lack of data) versus other criteria related to risk of bias.
• The number of datasets needed to implement the dose-response meta-analysis for a specific health outcome as opposed to modeling data from a single study should be specified.
• EPA’s explanation of the methodology used to generate the effective counts (e.g., number of people affected) for their dose-response meta-analysis was incomplete. The committee understood from the public presentations and conversations with EPA staff that these values were estimated from the adjusted odds ratios and relative risks reported in different studies but the specific details about how this was done were not provided.
• EPA mentioned that, if data allow, additional analyses will be performed to consider the impact of latency, potentially sensitive groups, high intake of water among children, and smoking, but a specific plan for these analyses was not provided.
• Although EPA clearly explained the proposed methods for binary outcome data, a plan for analysis of continuous outcomes, such as birthweight and neurodevelopmental end points, was not provided.
• The committee recommends that EPA conduct sensitivity analyses if there are sufficient data. These should include separate analyses for studies based on exposure biomarkers versus studies of water iAs, analyses restricted to the lower end of the dose-response curve, studies that include information on early life exposures and latent effects, and analyses that consider the impact of including studies with a high risk of bias. If sensitivity analyses are not conducted, EPA should provide justification in the iAs IRIS assessment.

Common Intake Approach

The committee agrees with EPA’s conversion of a variety of exposure metrics to a common intake value to facilitate including as many studies as possible in the dose-response analysis for each of the end points considered. The development of the PBPK model and its validation using two large datasets with information on both urine and water arsenic, a wide range of exposure levels, and characterized by different

arsenic metabolism patterns, nutritional factors, and other factors supports that the method developed is appropriate and generalizable for converting urinary arsenic measures to water arsenic concentrations in many populations in the United States and in other countries. Because the PBPK model was developed for adults, it will limit the use of the common intake metric to studies in adult populations. If EPA wishes to potentially increase the number of end points and studies in the dose-response meta-analysis, studies that used other exposure biomarkers (e.g., toenails, blood) could be added by using alternative approaches to convert these biomarker measurements to water arsenic measures (e.g., Moon et al. 2017).

The committee offers some suggestions for additional clarification about the approach. The common intake dose metric (µg/kg-day) is presented in Section 5.3, including the formula to estimate dose in relation to well-water concentrations. It would be helpful for EPA to identify the sources or references for the different elements of the formula. For instance, well-water concentrations of iAs can come directly from the study or be estimated from urinary concentrations using the PBPK model. For other elements in the formula that are unlikely to be available in the original studies (e.g., body weight, dietary intake, fraction or time spent consuming well water, or water consumption rate), it will be important to provide information on the data sources used to support the values selected, and to select values relevant to the different study populations (e.g., Bangladesh, United States).

Modeling Approaches to Investigate Dose-Response Relationships

The majority of the committee agrees with EPA’s plans for performing dose-response investigations for cancer and noncancer end points (see Appendix A for dissenting statement from one member and committee rebuttal). On the basis of the IRIS iAs assessment plan and information provided at the public meeting, the committee understands that EPA plans to use the dose-response meta-analysis approach to generate relative risk estimates that will then be fed into a life table to generate estimates of lifetime extra risk. The life table analysis will be repeated for several doses to select a dose corresponding to a certain level of lifetime extra risk. The committee recommends that EPA also consider the life table methodology when analyzing data from a single study (i.e., when there are not enough data to support a meta-analysis). EPA should provide clear documentation of each step of this procedure, including how studies were selected for the dose-response analysis.

Several members of the public raised concerns that the proposed methodology does not accommodate a threshold effect for cancer outcomes, which some analyses of previous studies suggest. The committee does not share this concern because the logistic regression model inherently assumes a threshold in the dose response. As explained in Piegorsch and Bailer (1997), the logistic model is a member of a class of dose-response models known as tolerance distribution models. Tolerance distribution models assume that each organism has a latent tolerance to a chemical and an adverse response only occurs once this threshold is exceeded. The logistic regression model assumes that the distribution of these latent tolerances is logistic and uses the cumulative distribution function to calculate the probability that tolerance is less than a given dose. Thus, assuming a positive dose-response and a low background rate, the logistic regression model will produce an S-shaped curve with the left-hand tail approaching a threshold of zero extra response, if not exactly so. EPA is also addressing concerns about the behavior of the logistic model at low doses by considering fractional polynomial models, for example, which would allow for J-shaped curves. EPA explained during the public meeting that a pre-specified set of fractional polynomial models will be fit and the final model will be selected using an AIC. EPA is commended for considering models that are more flexible when there is considerable variability in dose (e.g., more than four distinct values). The committee suggests that EPA consider a method that more adequately accounts for model uncertainty, such as a stochastic search selection procedure (e.g., Sabanés Bové and Held 2011). Also, if there are sufficient data, the committee recommends that EPA consider a nonparametric regression procedure (i.e., splines) that avoids the model selection problem presented by the fractional polynomial approach. The dose-response methods should be completely described and justified in the iAs IRIS assessment. Some additional considerations include: