THE LEARNING HEALTH SYSTEM SERIES
Artificial Intelligence
in Health Care
The Hope, the Hype, the Promise, the Peril
Michael Matheny,
Sonoo Thadaney Israni, Mahnoor Ahmed,
and Danielle Whicher, Editors
WASHINGTON, DC
NAM.EDU
NATIONAL ACADEMY OF MEDICINE 500 Fifth Street, NW Washington, DC 20001
This publication has undergone peer review according to procedures established by the National Academy of Medicine (NAM). Publication by the NAM signifies that it is the product of a carefully considered process and is a contribution worthy of public attention, but does not constitute endorsement of conclusions and recommendations by the NAM. The views presented in this publication are those of individual contributors and do not represent formal consensus positions of the authors’ organizations; the NAM; or the National Academies of Sciences, Engineering, and Medicine.
International Standard Book Number-13: 978-1-947103-17-7
Library of Congress Control Number: 2020938860
Copyright 2022 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
Suggested citation: Matheny, M., S. Thadaney Israni, M. Ahmed, and D. Whicher, Editors. 2022. Artificial Intelligence in Health Care: The Hope, the Hype, the Promise, the Peril. Washington, DC: National Academy of Medicine.
“Knowing is not enough; we must apply.
Willing is not enough; we must do.”
—GOETHE
ABOUT THE NATIONAL ACADEMY OF MEDICINE
The National Academy of Medicine is one of three Academies constituting the National Academies of Sciences, Engineering, and Medicine (the National Academies). The National Academies provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine.
The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president.
The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. John L. Anderson is president.
The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on issues of health, health care, and biomedical science and technology. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president.
Learn more about the National Academy of Medicine at NAM.edu.
AUTHORS
MICHAEL MATHENY (Co-Chair), Vanderbilt University Medical Center and the Department of Veterans Affairs
SONOO THADANEY ISRANI (Co-Chair), Stanford University
ANDREW AUERBACH, University of California, San Francisco
ANDREW BEAM, Harvard University
PAUL BLEICHER, OptumLabs
WENDY CHAPMAN, University of Melbourne
JONATHAN CHEN, Stanford University
GUILHERME DEL FIOL, University of Utah
HOSSEIN ESTIRI, Harvard Medical School
JAMES FACKLER, Johns Hopkins School of Medicine
STEPHAN FIHN, University of Washington
ANNA GOLDENBERG, University of Toronto
SETH HAIN, Epic
JAIMEE HEFFNER, Fred Hutchinson Cancer Research Center
EDMUND JACKSON, Hospital Corporation of America
JEFFREY KLANN, Harvard Medical School and Massachusetts General Hospital
RITA KUKAFKA, Columbia University
HONGFANG LIU, Mayo Clinic
DOUGLAS MCNAIR, Bill & Melinda Gates Foundation
ENEIDA MENDONÇA, Regenistrief Institute
JONI PIERCE, University of Utah
W. NICHOLSON PRICE II, University of Michigan
JOACHIM ROSKI, Booz Allen Hamilton
SUCHI SARIA, Johns Hopkins University
NIGAM SHAH, Stanford University
RANAK TRIVEDI, Stanford University
JENNA WIENS, University of Michigan
NAM Staff
Development of this publication was facilitated by contributions of the following NAM staff, under the guidance of J. Michael McGinnis, Leonard D. Schaeffer Executive Officer and Executive Director of the Leadership Consortium for a Value & Science-Driven Health System:
DANIELLE WHICHER, Senior Program Officer (until September 2019)
MAHNOOR AHMED, Associate Program Officer
JESSICA BROWN, Executive Assistant to the Executive Officer (until September 2019)
FASIKA GEBRU, Senior Program Assistant
JENNA OGILVIE, Deputy Director of Communications
REVIEWERS
This Special Publication was reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with review procedures established by the National Academy of Medicine (NAM). We wish to thank the following individuals for their contributions:
The reviewers listed above provided many constructive comments and suggestions, but they were not asked to endorse the content of the publication, and did not see the final draft before it was published. Review of this publication was overseen by DANIELLE WHICHER, Senior Program Officer, NAM; MAHNOOR AHMED, Associate Program Officer, NAM; and J. MICHAEL MCGINNIS, Leonard D. Schaeffer Executive Officer, NAM. Responsibility for the final content of this publication rests with the editors and the NAM.
This page intentionally left blank.
FOREWORD
In 2006, the National Academy of Medicine (NAM) established the Roundtable on Evidence-Based Medicine for the purpose of providing a trusted venue for national leaders in health and health care to work cooperatively toward their common commitment to effective, innovative care that consistently generates value for patients and society. The goal of advancing a “Learning Health System” quickly emerged and was defined as “a system in which science, informatics, incentives, and culture are aligned for continuous improvement and innovation, with best practices seamlessly embedded in the delivery process and new knowledge captured as an integral by-product of the delivery experience.”1
To advance this goal, and in recognition of the increasingly essential role that digital health innovations in data and analytics contribute to achieving this goal, the Digital Health Learning Collaborative was established. Over the life of the collaborative, the extraordinary preventive and clinical medical care implications of rapid innovations in artificial intelligence (AI) and machine learning emerged as essential considerations for the consortium. The publication you are now reading responds to the need for physicians, nurses and other clinicians, data scientists, health care administrators, public health officials, policy makers, regulators, purchasers of health care services, and patients to understand the basic concepts, current state of the art, and future implications of the revolution in AI and machine learning. We believe that this publication will be relevant to those seeking practical, relevant, understandable, and useful information about key definitions, concepts, applicability, pitfalls, rate-limiting steps, and future trends in this increasingly important area.
Michael Matheny, M.D., M.S., M.P.H., and Sonoo Thadaney Israni, M.B.A., have assembled a stellar team of contributors, all of whom enjoy wide respect in their fields. Together, in this well-edited volume that has benefitted from the thorough review process ingrained in the NAM’s culture, they present expert,
___________________
1 See https://nam.edu/wp-content/uploads/2015/07/LearningHealthSystem_28jul15.pdf.
understandable, comprehensive, and practical insights on topic areas that include the historical development of the field; lessons learned from other industries; how massive amounts of data from a variety of sources can be appropriately analyzed and integrated into clinical care; how innovations can be used to facilitate population health models and social determinants of health interventions; the opportunities to equitably and inclusively advance precision medicine; the applicability for health care organizations and businesses to reduce the cost of care delivery; opportunities to enhance interactions between health care professionals and patients, families, and caregivers; and the role of legal statutes that inform the uptake of AI in health care.
As the co-chairs of the Digital Health Learning Collaborative, we are excited by the progress being demonstrated in realizing a virtuous cycle in which the data inevitably produced by every patient encounter might be captured into a “collective memory” of health services to be used to inform and improve the subsequent care of the individual patient and the health system more generally. Enormous datasets are increasingly generated not only in the formal health care setting, but also from medical and consumer devices, wearables, and patient-reported outcomes, as well as environmental, community, and public health sources. They include structured (or mathematically operable) data as well as text, images, and sounds. The landscape also includes data “mash-ups” from commercial, legal, and online social records.
AI has been the tool envisioned to offer the most promise in harvesting knowledge from that collective memory, and as this volume demonstrates, some of that promise is being realized. Among the most important of these promises in the near term is the opportunity to assuage the frustration of health care providers who have been clicking away on electronic health records with modest benefit beyond increased data transportability and legibility. Our hope is that AI will be the “payback” for the investment in both the implementation of electronic health records and the cumbersomeness of their use by facilitating tasks that every clinician, patient, and family would want, but are impossible to do without electronic assistance—such as monitoring a patient for emergent sepsis 24 × 7 × 365 and providing timelier therapy for a condition in which diagnostic delay correlates with increased risk of death.
However, we also appreciate that AI alone cannot cure health care’s ills and that new technologies bring novel and potentially under-appreciated challenges. For example, if a machine learning algorithm is trained with data containing a systematic bias, then that bias may be interpreted as normative, exacerbating rather than resolving disparities and inequities in care. Similarly, association of data does not prove causality, and it may not even be explanatory, suggesting that a simultaneous revolution in research methods is also necessary. Finally, the mere
existence of substantial and sensitive data assets raises concerns about privacy and security. Aspiring to the promise of AI requires both continuing innovation and attention to the potential perils.
In our opinion, this publication presents a sober and balanced celebration of accomplishments, possibilities, and pitfalls. We commend Drs. Michael McGinnis and Danielle Whicher for their thoughtful sponsorship of the NAM Consortium and Digital Health Learning Collaborative, Dr. Matheny and Mrs. Thadaney Israni for their leadership in producing this volume, and to all the contributors who have produced an exceptional resource with practical relevance to a wide array of key stakeholders.
Jonathan B. Perlin, M.D., Ph.D., MACP
Reed V. Tuckson, M.D., FACP
Co-Chairs, Digital Learning Collaborative, Consortium on Value and Science-Driven Health Care, National Academy of Medicine
This page intentionally left blank.
CONTENTS
1 Artificial Intelligence in Health Care: The Hope, the Hype, the Promise, the Peril
NAM Leadership Consortium: Collaboration for a Value & Science-Driven Learning Health System
Digital Health Learning Collaborative
Promoting Trust, Equity, and Inclusion in Health Care AI
2 Overview of Current Artificial Intelligence
3 How Artificial Intelligence Is Changing Health and Health Care
AI Solutions for Patients and Families
AI Solutions for the Clinician Care Team
AI Solutions for Population/Public Health Program Management
AI Solutions for Health Care Business Administrators
4 Potential Trade-Offs and Unintended Consequences of Artificial Intelligence
How Could Improper AI Hurt Patients and the Health System?
How Could AI Reshape Medicine and Health in Unintended Ways?
How Will AI Transform Patient, Provider, and Computer Interactions?
What Will Happen to Acceptance, Trust, and Liability in a Human and Machine AI Future?
How Will Health Care Provider Roles Be Conceptualized?
Why Should This Time Be Any Different?
5 Artificial Intelligence Model Development and Validation
6 Deploying Artificial Intelligence in Clinical Settings
Settings for Application of AI in Health Care
Applications of AI in Clinical Care Delivery
Framework and Criteria for AI Selection and Implementation in Clinical Care
7 Health Care Artificial Intelligence: Law, Regulation, and Policy
Overview of Health Care AI Laws and Regulations in the United States
Safety and Efficacy of Clinical Systems
Privacy, Information, and Data, 220
8 Artificial Intelligence in Health Care: Hope Not Hype, Promise Not Peril
Summary of Challenges and Key Priorities
A Additional Key Reference Materials
This page intentionally left blank.
BOXES, FIGURES, AND TABLES
BOXES
5-1 Key Considerations in Model Development
5-2 Key Definitions in Model Development
7-1 Federal Food, Drug, and Cosmetic Act (21 U.S.C. § 360j) Medical Device Definition
FIGURES
S-1 Advancing to the Quintuple Aim
1-1 Life expectancy gains and increased health spending, selected high-income countries, 1995–2015
1-2 A summary of the domains of artificial intelligence
1-3 A summary of the most common methods and applications for training machine learning algorithms
1-4 Growth in facts affecting provider decisions versus human cognitive capacity
1-5 Framework for implementing artificial intelligence through the lens of human rights values
5-2 Patient timeline and associated data-gathering opportunities
6-1 Determinants of population health
6-2 Relationship of population health to public health and standard clinical care
6-3 Developmental life cycle of artificial intelligence applications
7-1 Different forms of transparency
8-4 Relationship between regulation and risk
TABLES
1-2 Relevant Ethical Codes, Frameworks, and Guidelines
3-1 Examples of Artificial Intelligence Applications for Stakeholder Groups
5-1 Example of Artificial Intelligence Applications by the Primary Task and Main Stakeholder
6-1 Leveraging Artificial Intelligence Tools into a Learning Health System
6-2 Key Considerations for Institutional Infrastructure and Governance
ACRONYMS AND ABBREVIATIONS
| ACM | Association of Computing Machinery |
| AI | artificial intelligence |
| AMA | American Medical Association |
| API | application programming interface |
| ATM | automated teller machine |
| AUROC | area under the ROC curve |
| BBC | British Broadcasting Corporation |
| CDC | Centers for Disease Control and Prevention |
| CDM | common data model |
| CDS | clinical decision support |
| CGMP | Current Good Manufacturing Process |
| CLIA | Clinical Laboratory Improvement Amendments |
| CMS | Centers for Medicare & Medicaid Services |
| CONSORT | Consolidated Standards of Reporting Trials |
| CPIC | Clinical Pharmacogenetics Implementation Consortium |
| CPU | central processing unit |
| DARPA | Defense Advanced Research Projects Agency |
| DHLC | Digital Health Learning Collaborative |
| DOJ | U.S. Department of Justice |
| ECA | embodied conversational agent |
| ECG | electrocardiogram |
| EHR | electronic health record |
| EU | European Union |
| FAIR | findability, accessibility, interoperability, and reusability |
| FDA | U.S. Food and Drug Administration |
| FDCA | Federal Food, Drug, and Cosmetic Act |
| FHIR | Fast Healthcare Interoperability Resource |
| fRamily | friends and family unpaid caregivers |
| FTC | Federal Trade Commission |
| FTCA | Federal Trade Commission Act |
| GDPR | General Data Protection Regulation |
| GPS | global positioning system |
| GPU | graphics processing unit |
| HAZOP | hazard and operability study |
| HHS | U.S. Department of Health and Human Services |
| HIE | health information exchange |
| HIPAA | Health Insurance Portability and Accountability Act |
| HITECH Act | Health Information Technology for Economic and Clinical Health Act |
| HIV | human immunodeficiency virus |
| i2b2 | Informatics for Integrating Biology & the Bedside |
| ICD-10 | International Classification of Diseases, 10th Revision |
| IEEE | Institute of Electrical and Electronics Engineers |
| IOM | Institute of Medicine |
| IoT | Internet of Things |
| IMDRF | International Medical Device Regulators Forum |
| IP | intellectual property |
| IT | information technology |
| IVD | in vitro diagnostic device |
| IVDMIA | in vitro diagnostic multivariate index assay |
| JITAI | just-in-time adaptive intervention |
| LDT | laboratory-developed test |
| Leadership Consortium | National Academy of Medicine Leadership Consortium: Collaboration for a Value & Science-Driven Learning Health System |
| LHS | learning health system |
| LOINC | Logical Observational Identifiers Names and Codes |
| MIT | Massachusetts Institute of Technology |
| NAM | National Academy of Medicine |
| NAS | National Academy of Sciences |
| NeurIPS | Conference on Neural Information Processing Systems |
| NHTSA | National Highway Traffic Safety Administration |
| NIH | National Institutes of Health |
| NITRC | Neuroimaging Informatics Tools and Resources Clearinghouse |
| NLP | natural language processing |
| NNH | number needed to harm |
| NNT | number needed to treat |
| NPV | negative predictive value |
| NRC | National Research Council |
| NSTC | National Science and Technology Council |
| OHDSI | Observational Health Data Sciences and Informatics |
| OHRP | Office for Human Research Protections |
| OMOP | Observational Medical Outcomes Partnership |
| ONC | The Office of the National Coordinator for Health Information Technology |
| PARiHS | Promoting Action on Research Implementation in Health Services |
| PCORnet | Patient-Centered Clinical Research Network |
| PDSA | plan-do-study-act |
| PFS | physician fee schedule |
| PHI | protected health information |
| PPV | positive predictive value |
| PR | precision-recall |
| Pre-Cert | Digital Health Software Precertification Program |
| QI | quality improvement |
| QMS | quality management system |
| R&D | research and development |
| ROC | receiver operating characteristic |
| RWD | real-world data |
| RWE | real-world evidence |
| SaMD | software as a medical device |
| SDLC | software development life cycle |
| SDoH | social determinants of health |
| SMART | Substitutable Medical Apps, Reusable Technology |
| STARD | Standards for Reporting of Diagnostic Accuracy Studies |
| TPR | true positive rate |
| TPU | tensor processing unit |
| UDN | Undiagnosed Diseases Network |
| WEIRD | Western, educated, industrialized, rich, and democratic |
