Advancing the Framework for
Assessing Causality of Health
and Welfare Effects to Inform
National Ambient Air Quality
Standard Reviews

_____

Committee on Assessing Causality from a
Multidisciplinary Evidence Base for National
Ambient Air Quality Standards

Board on Environmental Studies and Toxicology

Division on Earth and Life Studies

Board on Mathematical Sciences and Analytics

Division on Engineering and Physical Sciences

Consensus Study Report

NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001

This activity was supported by a contract between the National Academy of Sciences and the U.S. Environmental Protection Agency. Any opinions, findings, conclusions, or recommendations expressed in this publication do not necessarily reflect the views of any organization or agency that provided support for the project.

International Standard Book Number-13: 978-0-309-69011-9
International Standard Book Number-10: 0-309-69011-0
Digital Object Identifier: https://doi.org/10.17226/26612

This publication is available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.

Copyright 2022 by the National Academy of Sciences. National Academies of Sciences, Engineering, and Medicine and National Academies Press and the graphical logos for each are all trademarks of the National Academy of Sciences. All rights reserved.

Printed in the United States of America.

Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2022. Advancing the Framework for Assessing Causality of Health and Welfare Effects to Inform National Ambient Air Quality Standard Reviews. Washington, DC: The National Academies Press. https://doi.org/10.17226/26612.

The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president.

The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. John L. Anderson is president.

The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president.

The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine.

Learn more about the National Academies of Sciences, Engineering, and Medicine at www.nationalacademies.org.

Consensus Study Reports published by the National Academies of Sciences, Engineering, and Medicine document the evidence-based consensus on the study’s statement of task by an authoring committee of experts. Reports typically include findings, conclusions, and recommendations based on information gathered by the committee and the committee’s deliberations. Each report has been subjected to a rigorous and independent peer-review process and it represents the position of the National Academies on the statement of task.

Proceedings published by the National Academies of Sciences, Engineering, and Medicine chronicle the presentations and discussions at a workshop, symposium, or other event convened by the National Academies. The statements and opinions contained in proceedings are those of the participants and are not endorsed by other participants, the planning committee, or the National Academies.

Rapid Expert Consultations published by the National Academies of Sciences, Engineering, and Medicine are authored by subject-matter experts on narrowly focused topics that can be supported by a body of evidence. The discussions contained in rapid expert consultations are considered those of the authors and do not contain policy recommendations. Rapid expert consultations are reviewed by the institution before release.

For information about other products and activities of the National Academies, please visit www.nationalacademies.org/about/whatwedo.

COMMITTEE ON ASSESSING CAUSALITY FROM A MULTIDISCIPLINARY EVIDENCE BASE FOR NATIONAL AMBIENT AIR QUALITY STANDARDS

Members

ARMISTEAD G. RUSSELL (Co-Chair), Georgia Institute of Technology

ELIZABETH A. STUART (Co-Chair), Johns Hopkins Bloomberg School of Public Health

JILL S. BARON, U.S. Geological Survey and Colorado State University

JOAN WENNSTROM BENNETT, Rutgers University (until November 2021)

KIROS T. BERHANE, Columbia University Mailman School of Public Health

EDMUND A. C. CROUCH, Green Toxicology, LLC

IVÁN DÍAZ, New York University Langone Health

FRANCESCA DOMINICI, Harvard T.H. Chan School of Public Health

CHARLES T. DRISCOLL, Syracuse University

JOEL KAUFMAN, University of Washington

JANA B. MILFORD, University of Colorado Boulder

RICHARD L. POIROT, Private Air Quality/Climate Consultant

RICHARD L. SMITH, University of North Carolina

JUSTIN G. TEEGUARDEN, Pacific Northwest National Laboratory

Staff

SAMMANTHA MAGSINO, Project Director

LYLY LUHACHECK, Associate Program Officer

LESLIE BEAUCHAMP, Senior Program Assistant

KALEY BEINS, Program Officer (until July 2021)

TYLER KLOEFKORN, Senior Program Officer (until September 2021)

CARL-GUSTAV ANDERSON, Program Officer (until February 2022)

CHRISTOPHER REA, Program Officer (until February 2021)

TAMARA DAWSON, Program Coordinator (until August 2021)

MELISSA FRANKS, Senior Program Assistant (until May 2021)

Sponsor

U.S. ENVIRONMENTAL PROTECTION AGENCY

BOARD ON ENVIRONMENTAL STUDIES AND TOXICOLOGY

Members

FRANK W. DAVIS (Chair), University of California, Santa Barbara

ANN M. BARTUSKA, U.S. Department of Agriculture

DANA BOYD BARR, Emory University

GERMAINE M. BUCK LOUIS, George Mason University

FRANCESCA DOMINICI (NAM), Harvard University

GEORGE M. GRAY, The George Washington University

R. J. LEWIS, ExxonMobil Biomedical Sciences, Inc.

LINSEY C. MARR, Virginia Polytechnic Institute and State

MARIE L. MIRANDA, Children’s Environmental Health Initiative

R. CRAIG POSTLEWAITE, U.S. Department of Defense

REZA J. RASOULPOUR, Corteva Agriscience

JOSHUA TEWKSBURY, Smithsonian Tropical Research Institute

SACOBY M. WILSON, University of Maryland, College Park

TRACEY J. WOODRUFF, University of California, San Francisco

Staff

CLIFFORD DUKE, Director

RAYMOND A. WASSEL, Scholar and Director of Environmental Studies

KATHRYN GUYTON, Senior Program Officer

NATALIE ARMSTRONG, Associate Program Officer

LAURA LLANOS, Finance Business Partner

LESLIE BEAUCHAMP, Senior Program Assistant

THOMASINA LYLES, Senior Program Assistant

BOARD ON MATHEMATICAL SCIENCES AND ANALYTICS

Members

MARK L. GREEN (Chair), University of California, Los Angeles

HELENE BARCELO, Mathematical Sciences Research Institute

BONNIE BERGER, Massachusetts Institute of Technology

RUSSEL E. CAFLISCH, New York University

DAVID S. CHU, Institute for Defense Analyses

DUANE COOPER, Morehouse College

JAMES H. CURRY, University of Colorado Boulder

RONALD D. FRICKER, JR., Virginia Polytechnic Institute and State University

LYDIA E. KAVRAKI, Rice University

TAMARA G. KOLDA, MathSci.ai

PETROS KOUMOUTSAKOS, Harvard University

RACHEL KUSKE, Georgia Institute of Technology

YANN A. LECUN, Facebook and New York University

JILL C. PIPHER, Brown University

YORAM SINGER, Princeton University

TATIANA TORO, University of Washington, Seattle

LANCE A. WALLER, Emory University

AMIE WILKINSON, University of Chicago

KAREN WILLCOX, Oden Institute for Computational Engineering and Sciences and The University of Texas at Austin

Staff

MICHELLE K. SCHWALBE, Director

BRITTANY SEGUNDO, Program Officer

SAMANTHA KORETSKY, Research Assistant

HEATHER LOZOWSKI, Senior Finance Business Partner

JOSEPH PALMER, Senior Project Assistant

This page intentionally left blank.

Reviewers

This Consensus Study Report was reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise. The purpose of this independent review is to provide candid and critical comments that will assist the National Academies of Sciences, Engineering, and Medicine in making each published report as sound as possible and to ensure that it meets the institutional standards for quality, objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process.

We thank the following individuals for their review of this report:

Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations of this report nor did they see the final draft before its release. The review of this report was overseen by DAVID EATON, University of Washington, and MARK UTELL, University of Rochester School of Medicine and Dentistry. They were responsible for making certain that an independent examination of this report was carried out in accordance with the standards of the National Academies and that all review comments were carefully considered. Responsibility for the final content rests entirely with the authoring committee and the National Academies.

This page intentionally left blank.

Contents

SUMMARY

1 INTRODUCTION

Statement of Task

Committee Composition

Interpretation of Task and Study Boundaries

Causal Determinations

Committee Information Gathering

Report Organization

2 HISTORICAL AND LEGAL PERSPECTIVES ON THE NAAQS FRAMEWORK FOR ASSESSING CAUSALITY

Legal Background for the NAAQS

Historical Perspective on the 2015 Preamble Framework

3 DEFINITION OF CAUSALITY

Defining Causal Effects: Potential Outcomes

Definition of “Cause” from the Preamble

Evaluating Causality Through Mechanistic Understanding

Causal Models

Manipulative Causation

4 INFLUENCE OF THE INDIVIDUAL STUDY ON THE BODY OF EVIDENCE

Study Selection and Evaluation in EPA’s Causal Determination Framework

Types of Evidence Used in Integrated Science Assessments to Support Causal Determinations

Transparency, Reproducibility, and Replicability in Individual Studies

5 EVIDENCE SYNTHESIS IN INTEGRATED SCIENCE ASSESSMENTS

Current Approach to Evidence Synthesis and Integration for Health Effects

Integration and Synthesis Examples for Health Effects from the 2019 ISA for Particulate Matter and the 2020 ISA for Ozone

Integration and Synthesis Examples for Health Effects from the 2020 Ozone ISA

Current Approach to Evidence Synthesis and Integration for Welfare Effects

Integration and Synthesis Examples for Welfare from the 2020 Ozone ISA

Integration and Synthesis Examples from the 2020 NO_x-SO_x-PM ISA

Evolution of the ISA Process Since Publication of the Preamble

6 EXAMPLE CRITIQUES OF THE ISA PROCESS AND CAUSAL DETERMINATION FRAMEWORK

CASAC Comments from Recent ISA Reviews

Critiques Related to the Weight of Evidence Approach

Critiques Related to Systematic Review

Critiques Related to the Causal Categories

Critiques Related to Populations at Increased Risk and Selection of Endpoints

Critiques Related to External Review

Synthesis of Critiques and Reviews of the ISA Framework

7 ILLUSTRATIVE FRAMEWORKS FOR CAUSAL DETERMINATIONS

Preamble to the Integrated Science Assessments (ISAs) (2015a)

Recent ISAs (2016–2020)

Toxic Substances Control Act (TSCA)

World Health Organization (WHO)

Integrated Risk Information System (IRIS)

Office of Health Assessment and Translation (OHAT)

International Agency for Research on Cancer (IARC)

E-Cigarettes

United Nations Economic Commission for Europe (UNECE)

Vaccines

Canadian Smog Science Assessment (CSSA)

Synthesis: What Might Be Useful in the ISA Causal Determination Framework

8 EMERGING METHODS FOR OBSERVATIONAL STUDIES CONTRIBUTING TO CAUSAL DETERMINATIONS

Advances in Methods for Exposure Assessment

Clarifying and Assessing the Validity of Causal Assumptions in Observational Studies

Assessing the Quality of Individual Studies with Respect to Handling Measured Confounders

Methods for Handling Unmeasured Confounders and Post-Treatment Variables

Consideration of Multiple Exposures

Examination of Variation in Effects

9 USE OF THE FRAMEWORK TO ASSESS BOTH HEALTH AND WELFARE EFFECTS

Accounting for Different Characteristics of Causal Chains

Adequacy and Effectiveness of the Current EPA Causal Determination Framework for Both Health and Welfare Standards

10 CONCLUSIONS AND RECOMMENDATIONS

Improving the Single Causal Determination Framework for Health and Welfare Effects

Study Quality Evaluation

Integrating Evidence

Expertise

Concluding Thoughts

REFERENCES

APPENDIXES

A COMMITTEE MEMBER BIOGRAPHIES

B OPEN SESSION AGENDAS

C INTRODUCTION TO CAUSAL INFERENCE PRINCIPLES

This page intentionally left blank.