National Academies Press: OpenBook
« Previous: Appendix C: Workshop Two Agenda
Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×

Appendix D

Workshop Three Agenda

Workshop Three: Application

July 17–18, 2018

National Academy of Sciences Building, Lecture Room
2101 Constitution Avenue, NW, Washington, DC 20418

The National Academies of Sciences, Engineering, and Medicine is convening a three-part workshop series, sponsored by the U.S. Food and Drug Administration, examining how real-world evidence development and uptake can enhance medical product development and evaluation. The workshops will advance discussions and common knowledge about complex issues relating to the generation and usage of real-world evidence, including fostering development and implementation of the science and technology of real-world evidence generation and usage.

  • Workshop One (September 1920, 2017) focused on how to align incentives to support collection and use of real-world evidence in health product review, payment, and delivery. Incentives need to address barriers impeding the uptake of real-world evidence, including barriers to transparency.
  • Workshop Two (March 67, 2018) illuminated what types of data are appropriate for what specific purposes and suggested practical approaches for data collection and evidence use by developing and working through example use cases.
Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
  • Workshop Three (July 1718, 2018) examined and suggested approaches for operationalizing the collection and use of real-world evidence through discussing “decision aids” about specific topics in study design. The decision aids are question lists developed to inform discussion around specific topics addressed at the third workshop. These discussions may help inform workshop attendees and other stakeholders about study design choices, including potential risks, costs, and reporting/transparency expectations.

DAY 1: JULY 17, 2018

8:00 a.m. Breakfast Available Outside the Lecture Room
8:15 a.m. Welcome and Opening Remarks
MARK MCCLELLAN, Workshop Series Co-Chair

Director

Duke-Margolis Center for Health Policy

GREGORY SIMON, Workshop Series Co-Chair

Investigator

Kaiser Permanente Washington Health Research Institute

Session I: Key Considerations for Real-World Evidence Application

Session Objectives:

  • Examine how some organizations are currently considering traditional and real-world evidence.
  • Discuss factors that may be influencing overall cost and time investment required by traditional evidence generation.
  • Consider when non-traditional data sources may be beneficial to assess outcomes.
8:45 a.m. Update on the Innovative Medicines Initiative’s GetReal and View from the National Institute for Health and Care Excellence, United Kingdom
PALL JONSSON

Associate Director, Research and Development

National Institute for Health and Care Excellence

Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
9:05 a.m. Drivers of Expense and Delay
ELLIOTT LEVY

Senior Vice President, Global Development

Amgen Inc.

9:25 a.m. Patient-Collected and -Owned Data
KOMATHI STEM

Chief Executive Officer and Founder

monARC Bionetworks

9:45 a.m. BREAK

Session II: When Is a Real-World Data Element Fit for Assessment of Eligibility, Treatment Exposure, or Outcomes?

Session Objectives:

  • Discuss potential bias-introducing steps in evidence generation from real-world data.
  • Suggest key considerations in the data collection and evidence-generation processes that influence reliability of real-world data.
  • Discuss how a decision aid laying out key questions and considerations might help inform current and future studies.
10:05 a.m. Introduction: A Proposed Framework for a Decision Aid
PALL JONSSON, Session Moderator

Associate Director, Research and Development

National Institute for Health and Care Excellence

10:15 a.m. Looking Back: How Might a Decision Aid Inform a Real-World Example?
JEFF ALLEN

President and Chief Executive Officer

Friends of Cancer Research

Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
10:35 a.m. Looking Forward: How Decision Aid Might Apply to Future Studies

Panel Discussion and Audience Q&A

AYLIN ALTAN

Senior Vice President of Research

OptumLabs

ROBERT BALL

Deputy Director, Office of Surveillance and Epidemiology

Center for Drug Evaluation and Research

U.S. Food and Drug Administration

LUCA FOSCHINI

Co-Founder and Chief Data Scientist

Evidation Health

BRANDE YAIST

Senior Director, Global Patient Outcomes and Real-World Evidence

Eli Lilly and Company

12:00 p.m. BREAK (Lunch Available Outside the Lecture Room)

Session III: Obscuring Intervention Allocation in Trials to Generate Real-World Evidence: Why, Who, and When?

Session Objectives:

  • Discuss how variability in knowledge of treatment assignment group affects:
    • Provider and patient adherence and outcomes.
    • Study cost and reliability.
  • Suggest key factors that could affect decisions to obscure intervention allocation.
  • Discuss how a decision aid laying out key questions and considerations might help inform current and future studies.
Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
1:00 p.m. Introduction: A Proposed Framework for a Decision Aid
JONATHAN WATANABE, Session Moderator

Associate Professor of Clinical Pharmacy

National Academy of Medicine Anniversary Fellow in Pharmacy

University of California, San Diego

1:10 p.m. Looking Back: How Might a Decision Aid Inform a Real-World Example?
JOHN GRAHAM

Head, Value Evidence and Outcomes

GlaxoSmithKline

ORLY VARDENY

Minneapolis Department of Veterans Affairs Center for Chronic Disease Outcomes Research

Associate Professor of Medicine

University of Minnesota

1:30 p.m. Looking Forward: How a Decision Aid Might Apply to Future Studies

Panel Discussion and Audience Q&A

CATHY CRITCHLOW

Vice President, Center for Observational Research

Amgen Inc.

NANCY DREYER

Chief Scientific Officer

IQVIA

ALEX JOHN LONDON

Clara L. West Professor of Ethics and Philosophy

Carnegie Mellon University

JAMES P. SMITH

Deputy Director, Division of Metabolism and Endocrinology Products

Center for Drug Evaluation and Research

U.S. Food and Drug Administration

Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
2:50 p.m. BREAK

Session IV: How Tightly Should Investigators Attempt to Control or Restrict Treatment Quality in a Pragmatic or Real-World Trial?

Session Objectives:

  • Discuss how variability in treatment delivery and adherence can affect results, including
    • Potential influence of variation in standard treatment practice, and
    • Considerations for balancing participant autonomy and safety.
  • Suggest key factors that could help determine the base comparison and level of control suited to a particular trial.
  • Discuss how a decision aid laying out key questions and considerations might help inform current and future studies.
3:10 p.m. Introduction: A Proposed Framework for a Decision Aid
JENNIFER GRAFF, Session Moderator

Vice President of Comparative Effectiveness Research

National Pharmaceutical Council

3:20 p.m. Looking Back: How Might a Decision Aid Inform a Real-World Example?
LARRY ALPHS

Deputy Chief Medical Officer

Newron Pharmaceuticals

3:40 p.m. Looking Forward: How a Decision Aid Might Apply to Future Studies

Panel Discussion and Audience Q&A

JUDITH CARRITHERS

Director of Regulatory Services

Advarra

W. BENJAMIN NOWELL

Director, Patient-Centered Research

Global Healthy Living Foundation

Co–Principal Investigator, ArthritisPower Patient Powered Research Network

Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
PETER STEIN

Deputy Director, Office of New Drugs

Center for Drug Evaluation and Research

U.S. Food and Drug Administration

4:50 p.m. Day 1 Wrap-Up and Concluding Thoughts/Discussion with Audience
5:00 p.m. ADJOURN WORKSHOP DAY 1

DAY 2: JULY 18, 2018

8:00 a.m. Welcome
MARK MCCLELLAN, Workshop Series Co-Chair

Director

Duke-Margolis Center for Health Policy

GREGORY SIMON, Workshop Series Co-Chair

Investigator

Kaiser Permanente Washington Health Research Institute

Session V: How Can Bias in Observational Comparisons Be Assessed and Minimized?

Session Objectives:

  • Discuss methods to assess presence of and optimally reduce bias from unmeasured confounding.
  • Suggest key considerations for assessing—and communicating—uncertainty in observational studies.
  • Discuss how a decision aid laying out key questions and considerations might help inform current and future studies.
8:10 a.m. Introduction: A Proposed Framework for a Decision Aid
DAVID MARTIN

Associate Director for Real-World Evidence Analytics U.S. Food and Drug Administration

Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
8:20 a.m. Looking Back: How Might a Decision Aid Inform a Real-World Example?
HECTOR IZURIETA

Epidemiologist, Office of Biostatistics and Epidemiology

Center for Biologics Evaluation and Research

U.S. Food and Drug Administration

8:40 a.m. Looking Forward: How a Decision Aid Might Apply to Future Studies

Panel Discussion and Audience Q&A

GREGORY DANIEL, Session Moderator

Deputy Director

Duke-Margolis Center for Health Policy

JESSICA FRANKLIN

Assistant Professor of Medicine

Harvard Medical School

NICOLE GORMLEY

Team Lead, Division of Hematologic Products

Office of Hematology and Oncology Products

Center for Drug Evaluation and Research

U.S. Food and Drug Administration

JAVIER JIMENEZ

Vice President and Global Head for Real-World Evidence and Clinical Outcomes

Sanofi

HENG LI

Mathematical Statistician

Center for Devices and Radiological Health

U.S. Food and Drug Administration

MARK VAN DER LAAN

Professor, Biostatistics and Statistics

University of California, Berkeley

10:00 a.m. BREAK
Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×

Session VI: FDA Panel

Session Objectives:

  • Hear updates and perspective of current thinking about real-world evidence in Europe.
  • Discuss challenges, opportunities, and remaining gaps for moving forward with real-world evidence application.
10:15 a.m. A European Perspective
ALASDAIR BRECKENRIDGE, Session Moderator

Emeritus Professor of Clinical Pharmacology

University of Liverpool

10:30 a.m. Reflections from FDA
STEVEN ANDERSON

Director, Office of Biostatistics and Epidemiology, Center for Biologics Evaluation and Research

U.S. Food and Drug Administration

JACQUELINE CORRIGAN-CURAY

Director, Office of Medical Policy, Center for Drug Evaluation and Research

U.S. Food and Drug Administration

JEFF SHUREN

Director, Center for Devices and Radiological Health

U.S. Food and Drug Administration

11:15 a.m. Panel Discussion with Audience
11:50 a.m. Synthesis of Workshop Discussions
MARK MCCLELLAN, Workshop Series Co-Chair

Director

Duke-Margolis Center for Health Policy

GREGORY SIMON, Workshop Series Co-Chair

Investigator

Kaiser Permanente Washington Health Research Institute

12:00 p.m. ADJOURN WORKSHOP DAY 2
Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×

This page intentionally left blank.

Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
Page 199
Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
Page 200
Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
Page 201
Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
Page 202
Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
Page 203
Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
Page 204
Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
Page 205
Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
Page 206
Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
Page 207
Suggested Citation:"Appendix D: Workshop Three Agenda." National Academies of Sciences, Engineering, and Medicine. 2019. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. Washington, DC: The National Academies Press. doi: 10.17226/25352.
×
Page 208
Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series Get This Book
×
Buy Paperback | $65.00 Buy Ebook | $54.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Randomized controlled trials (RCTs) have traditionally served as the gold standard for generating evidence about medical interventions. However, RCTs have inherent limitations and may not reflect the use of medical products in the real world. Additionally, RCTs are expensive, time consuming, and cannot answer all questions about a product or intervention. Evidence generated from real-world use, such as real-world evidence (RWE) may provide valuable information, alongside RCTs, to inform medical product decision making.

To explore the potential for using RWE in medical product decision making, the National Academies of Sciences, Engineering, and Medicine planned a three-part workshop series. The series was designed to examine the current system of evidence generation and its limitations, to identify when and why RWE may be an appropriate type of evidence on which to base decisions, to learn from successful initiatives that have incorporated RWE, and to describe barriers that prevent RWE from being used to its full potential. This publication summarizes the discussions from the entire workshop series.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!