Characterizing and Communicating Uncertainty
in the Assessment of Benefits and Risks
of Pharmaceutical Products
Workshop Summary
Denise Caruso, Rebecca A. English, and Anne B. Claiborne, Rapporteurs
Forum on Drug Discovery, Development, and Translation
Board on Health Sciences Policy
INSTITUTE OF MEDICINE
OF THE NATIONAL ACADEMIES
THE NATIONAL ACADEMIES PRESS
Washington, D.C.
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
NOTICE: The workshop that is the subject of this workshop summary was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.
This activity was supported by contracts between the National Academy of Sciences and the U.S. Department of Health and Human Services (HHSN26300023 [Under Base #HHSN263201200074I] and HHSF22301026T [Under Base #HHSF223200810020I]), AbbVie Inc., American Diabetes Association, American Society for Microbiology, Amgen Inc., Association of American Medical Colleges, AstraZeneca, Burroughs Wellcome Fund, Critical Path Institute, Doris Duke Charitable Foundation, Eli Lilly and Company, FasterCures, Friends of Cancer Research, GlaxoSmithKline, Johnson & Johnson, March of Dimes Foundation, Merck & Co., Inc., Novartis Pharmaceuticals Corporation, Pfizer Inc., and Sanofi. The views presented in this publication do not necessarily reflect the views of the organizations or agencies that provided support for the activity.
International Standard Book Number-13: 978-0-309-31000-0
International Standard Book Number-10: 0-309-31000-8
Additional copies of this workshop summary are available for sale from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
For more information about the Institute of Medicine, visit the IOM home page at: www.iom.edu.
Copyright 2014 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
The serpent has been a symbol of long life, healing, and knowledge among almost all cultures and religions since the beginning of recorded history. The serpent adopted as a logotype by the Institute of Medicine is a relief carving from ancient Greece, now held by the Staatliche Museen in Berlin.
Suggested citation: IOM (Institute of Medicine). 2014. Characterizing and communicating uncertainty in the assessment of benefits and risks of pharmaceutical products: Workshop summary. Washington, DC: The National Academies Press.
“Knowing is not enough; we must apply.
Willing is not enough; we must do.”
—Goethe
INSTITUTE OF MEDICINE
OF THE NATIONAL ACADEMIES
Advising the Nation. Improving Health.
THE NATIONAL ACADEMIES
Advisers to the Nation on Science, Engineering, and Medicine
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. C. D. Mote, Jr., is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Victor J. Dzau is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. C. D. Mote, Jr., are chair and vice chair, respectively, of the National Research Council.
PLANNING COMMITTEE ON CHARACTERIZING AND COMMUNICATING UNCERTAINTY IN THE ASSESSMENT OF BENEFITS AND RISKS OF PHARMACEUTICAL PRODUCTS1
BARUCH FISCHHOFF (Co-Chair), Carnegie Mellon University, Pittsburgh, PA
ROBERT E. RATNER (Co-Chair), American Diabetes Association, Alexandria, VA
MARGARET ANDERSON, FasterCures, Washington, DC
CHRISTOPHER P. AUSTIN, National Center for Advancing Translational Sciences, Bethesda, MD
PATRICK J. FREY, Center for Drug Evaluation and Research, Silver Spring, MD
TAREK A. HAMMAD, Merck & Co., Inc., North Wales, PA
GAVIN HUNTLEY-FENNER, Huntley-Fenner Advisors, Irvine, CA
CHARLES F. MANSKI, Northwestern University, Evanston, IL
PAUL J. SELIGMAN, Amgen Inc., Washington, DC
LANA R. SKIRBOLL, Sanofi, Washington, DC
BRIAN L. STROM, Rutgers, the State University of New Jersey, Newark
MYRL WEINBERG, National Health Council, Washington, DC
STEVEN WOLOSHIN, Dartmouth Medical School, Hanover, NH
IOM Staff
ANNE B. CLAIBORNE, Forum Director
REBECCA A. ENGLISH, Program Officer
ELIZABETH F. C. TYSON, Associate Program Officer
BARRET ZIMMERMANN, Senior Program Assistant (until July 2014)
ANDREW M. POPE, Director, Board on Health Sciences Policy
__________________
1 Institute of Medicine planning committees are solely responsible for organizing the workshop, identifying topics, and choosing speakers. The responsibility for the published workshop summary rests with the workshop rapporteurs and the institution.
This page intentionally left blank.
FORUM ON DRUG DISCOVERY, DEVELOPMENT, AND TRANSLATION1
JEFFREY M. DRAZEN (Co-Chair), New England Journal of Medicine, Boston, MA
STEVEN K. GALSON (Co-Chair), Amgen Inc., Thousand Oaks, CA
RUSS BIAGIO ALTMAN, Stanford University, CA
MARGARET ANDERSON, FasterCures, Washington, DC
HUGH AUCHINCLOSS, National Institute of Allergy and Infectious Diseases, Bethesda, MD
CHRISTOPHER P. AUSTIN, National Center for Advancing Translational Sciences, Bethesda, MD
ANN C. BONHAM, Association of American Medical Colleges, Washington, DC
LINDA BRADY, National Institute of Mental Health, Bethesda, MD
GAIL H. CASSELL, Harvard Medical School (Visiting), Carmel, IN
ANDREW M. DAHLEM, Eli Lilly and Company, Indianapolis, IN
JAMES H. DOROSHOW, National Cancer Institute, Bethesda, MD
GARY L. FILERMAN, Atlas Health Foundation, McLean, VA
MARK J. GOLDBERGER, AbbVie Inc., Rockville, MD
HARRY B. GREENBERG, Stanford University School of Medicine, CA
KATHY L. HUDSON, National Institutes of Health, Bethesda, MD
LYNN D. HUDSON, Critical Path Institute, Tucson, AZ
S. CLAIBORNE JOHNSTON, Dell Medical School, University of Texas, Austin
MICHAEL KATZ, March of Dimes Foundation, White Plains, NY
JACK D. KEENE, Duke University Medical Center, Durham, NC
RUSTY KELLEY, Burroughs Wellcome Fund, Research Triangle Park, NC
RONALD L. KRALL, University of Pittsburgh, Steamboat Springs, CO
FREDA C. LEWIS-HALL, Pfizer Inc., New York, NY
CAROL MIMURA, University of California, Berkeley
BRIGGS W. MORRISON, AstraZeneca, Summit, NJ
BERNARD H. MUNOS, InnoThink Center for Research in Biomedical Innovation, Indianapolis, IN
ELIZABETH (BETSY) MYERS, Doris Duke Charitable Foundation, New York, NY
JOHN J. ORLOFF, Baxter BioScience, Deerfield, IL
RAJESH RANGANATHAN, National Institute of Neurological Disorders and Stroke, Bethesda, MD
__________________
1 Institute of Medicine forums and roundtables do not issue, review, or approve individual documents. The responsibility for the published workshop summary rests with the workshop rapporteurs and the institution.
ROBERT E. RATNER, American Diabetes Association, Alexandria, VA
MICHAEL ROSENBLATT, Merck & Co., Inc., Whitehouse Station, NJ
JAMES S. SHANNON, GlaxoSmithKline, Brentford, Middlesex, UK
ELLEN V. SIGAL, Friends of Cancer Research, Washington, DC
LANA R. SKIRBOLL, Sanofi, Washington, DC
BRIAN L. STROM, Rutgers, the State University of New Jersey, Newark
JANET TOBIAS, Ikana Media, New York, NY
JOANNE WALDSTREICHER, Johnson & Johnson, New Brunswick, NJ
JANET WOODCOCK, U.S. Food and Drug Administration, Rockville, MD
IOM Staff
ANNE B. CLAIBORNE, Forum Director
REBECCA A. ENGLISH, Program Officer
CHRISTOPHER J. DEFEO, Program Officer
ELIZABETH F. C. TYSON, Associate Program Officer
BARRET ZIMMERMANN, Senior Program Assistant (until July 2014)
KATHRYN HOWELL, Senior Program Assistant (as of August 2014)
ANDREW M. POPE, Director, Board on Health Sciences Policy
Reviewers
This workshop summary has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published workshop summary as sound as possible and to ensure that the workshop summary meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the process. We wish to thank the following individuals for their review of this workshop summary:
Patricia Furlong, Parent Project Muscular Dystrophy
Elaine K. Gallin, QE Philanthropic Advisors
Robert J. Meyer, University of Virginia School of Medicine
Michael Rosenblatt, Merck & Co., Inc.
Although the reviewers listed above have provided many constructive comments and suggestions, they did not see the final draft of the workshop summary before its release. The review of this workshop summary was overseen by Eli Y. Adashi, The Warren Alpert Medical School, Brown University. Appointed by the Institute of Medicine, he was responsible for making certain that an independent examination of this workshop
summary was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this workshop summary rests entirely with the rapporteurs and the institution.
Contents
The Impact of Uncertainty on Regulatory Decision Making
2 IDENTIFYING AND CHARACTERIZING UNCERTAINTY
Key Sources of Uncertainty in Benefit–Risk Assessment and Associated Challenges
Reducing Uncertainty Through Maximizing the Value of Evidence
Methods to Address Uncertainty in the Postmarket Phase
Approaches to Assessing Internal and External Validity of RCTs
Unique Challenges of the Pharmaceutical Regulatory Setting
Communicating with the Public About Benefit, Harm, and Uncertainty
4 BASIC METHODOLOGIES AND APPLICATIONS FOR UNDERSTANDING AND EVALUATING UNCERTAINTY
Applying Decision Science in the Drug Review Process
Applying Decision Theory Approaches to Regulatory Decision Making
Overview of Risk Communication
Increasing Public Awareness and Understanding of Uncertainty
Communicating Uncertainty: FDA and Media Messages About Tysabri
Challenges and Opportunities to Improve Communication About Uncertainty
6 FINAL REFLECTIONS ON WAYS TO CHARACTERIZE AND COMMUNICATE UNCERTAINTY
Identifying and Mitigating Uncertainty Through Maximizing the Value of Evidence
Characterizing and Understanding Uncertainties
Eliciting Values from Stakeholders, Particularly Patients
Communicating Uncertainty About Benefit and Risk Assessments of Pharmaceutical Products
Boxes, Figures, and Table
BOXES
1-1 FDA PDUFA V Plan and the Characterization of Uncertainties in Benefits and Risks
1-2 Statement of Task for the Workshops
1-3 The Range of Sources of Uncertainty
1-4 Themes Observed by Workshop Co-Chairs
2-1 Considerations for Designing an Approach to Addressing Uncertainty
3-1 FDA Approaches to Inform Risk Communication
3-2 Exploring Patient Perspectives on Uncertainty Through PFDD
3-3 Decisions Made Under Uncertainty: Tysabri and Anoro Ellipta Case Studies
AC | advisory committee |
BHU | Benefit–Harm–Uncertainty Initiative (Health Canada) |
CDER | Center for Drug Evaluation and Research, FDA |
COPD | chronic obstructive pulmonary disease |
CV | cardiovascular |
DRDC | Defence Research and Development Canada |
EMA | European Medicines Agency |
FDA | U.S. Food and Drug Administration |
IMI | Innovative Medicines Initiative |
IOM | Institute of Medicine |
JCV | John Cunningham virus |
LABA | long-acting beta2-adrenergic agonist |
LAMA | long-acting antimuscarinic agent |
MACE | major adverse cardiac event |
MI | myocardial infarction |
MS | multiple sclerosis |
NEJM | New England Journal of Medicine |
NIH | National Institutes of Health |
PDUFA | Prescription Drug User Fee Act |
PFDD | Patient-Focused Drug Development |
PML | progressive multifocal leukoencephalopathy |
PRO | patient-reported outcome |
PROMPT | Prospective Routine Observational Monitoring Program Tools |
PROTECT | Pharmacoepidemiological Research on Outcomes of Therapeutics by a European Consortium |
RCT | randomized controlled trial |
REMS | Risk Evaluation and Mitigation Strategy |
SMU | Special Medical Use |
SPRC | Stanford Prevention Research Center |