ASSESSING AND IMPROVING THE
INTERPRETATION OF
BREAST IMAGES
WORKSHOP SUMMARY
Sharyl J. Nass and Margie Patlak, Rapporteurs
National Cancer Policy Forum
Board on Health Care Services
Institute of Medicine
THE NATIONAL ACADEMIES PRESS
Washington, DC
www.nap.edu
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
This project was supported by the American Cancer Society through a generous donation from the Longaberger Company’s Horizon of Hope Campaign®. The views presented in this publication do not necessarily reflect the views of the organizations or agencies that provided support for the activity.
International Standard Book Number-13: 978-0-309-37835-2
International Standard Book Number-10: 0-309-37835-4
DOI: 10.17226/21805
Additional copies of this workshop summary are available for sale from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
Copyright 2015 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2015. Assessing and improving the interpretation of breast images: Workshop Summary. Washington, DC: The National Academies Press.
The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Ralph J. Cicerone is president.
The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. C. D. Mote, Jr., is president.
The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president.
The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine.
Learn more about the National Academies of Sciences, Engineering, and Medicine at www.national-academies.org.
This page intentionally left blank.
WORKSHOP PLANNING COMMITTEE1
DIANA BUIST (Chair), Principal Investigator, Group Health Breast Cancer Surveillance
LORA BARKE, Radiologist, Radiology Imaging Associates, Invision Sally Jobe Breast Centers
PATRICIA A. CARNEY, Associate Director for Population Studies, Oregon Health & Sciences University Cancer Institute
PATRICIA GANZ, Distinguished Professor, Health Policy & Management and Medicine, University of California, Los Angeles (UCLA) Fielding School of Public Health, David Geffen School of Medicine at UCLA; Director, Cancer Prevention and Control Research, UCLA Jonsson Comprehensive Cancer Center
ABBE HERZIG, Patient Advocate
GRETA MASSETTI, Associate Director for Science, Division of Cancer Prevention and Control, Centers for Disease Control and Prevention
DIANA MIGLIORETTI, Dean’s Professor in Biostatistics, Department of Public Health Sciences, University of California, Davis
BARBARA MONSEES, Evans Professor of Women’s Health, Emeritus Chief, Breast Imaging Section, Washington University School of Medicine
TRACY ONEGA, Professor, Section of Biostatistics & Epidemiology, Dartmouth Medical School
ETTA D. PISANO, Vice President for Medical Affairs, Dean, Medical University of South Carolina
Project Staff
SHARYL J. NASS, Director, National Cancer Policy Forum
PATRICK ROSS, Research Assistant
HANNAH DURING, Senior Program Assistant
_____________
1 Institute of Medicine planning committees are solely responsible for organizing the workshop, identifying topics, and choosing speakers. The responsibility for the published workshop summary rests with the workshop rapporteurs and the institution.
This page intentionally left blank.
NATIONAL CANCER POLICY FORUM1
MICHAEL CALIGIURI (Chair), Chief Executive Officer, James Cancer Hospital and Solove Research Institute; Director, Ohio State University Comprehensive Cancer Center
PATRICIA A. GANZ (Vice Chair), Distinguished Professor, Health Policy & Management and Medicine, University of California, Los Angeles (UCLA) Fielding School of Public Health, David Geffen School of Medicine at UCLA; Director, Cancer Prevention and Control Research, UCLA Jonsson Comprehensive Cancer Center
AMY P. ABERNETHY, Chief Medical Officer and Senior Vice President for Oncology, Flatiron Health; Professor, Division of Medical Oncology, Duke University School of Medicine; Director, Center for Learning Health Care, Duke Clinical Research Institute
LUCILE ADAMS-CAMPBELL, Professor of Oncology, Associate Director for Minority Health and Health Disparities Research, Georgetown University Lombardi Cancer Center
KENNETH ANDERSON, Kraft Family Professor of Medicine, American Cancer Society Clinical Research Director, Jerome Lipper Multiple Myeloma Center, Harvard Medical School, Dana-Farber Cancer Institute
LOYCE PACE BASS, Health Policy Director, LIVESTRONG Foundation
MONICA BERTAGNOLLI, Professor of Surgery, Harvard University Medical School
OTIS BRAWLEY, Chief Medical Officer and Executive Vice President, American Cancer Society
CARLTON BROWN, Director of Professional Services, Oregon Nurses Association, and past president, Oncology Nursing Society
SERGIO CANTOREGGI, Chief Scientific Officer and Global Head of Research and Development, Helsinn Group
ROBERT W. CARLSON, Chief Executive Officer, National Comprehensive Cancer Network
_____________
1 Institute of Medicine forums and roundtables do not issue, review, or approve individual documents. The responsibility for the published workshop summary rests with the workshop rapporteurs and the institution.
GREGORY CURT, Executive Director for External Relations in US Medical Affairs, AstraZeneca; Co-Chair, Life Sciences Consortium Task Force
WILLIAM S. DALTON, Chief Executive Officer, M2Gen Personalized Medicine Institute, Moffitt Cancer Center; Chair, American Association for Cancer Research Science Policy & Legislative Affairs Committee
GWEN DARIEN, Executive Vice President, Programs and Services, Cancer Policy Institute, Cancer Support Community
WENDY DEMARK-WAHNEFRIED, Associate Director for Cancer Prevention and Control, University of Alabama at Birmingham Comprehensive Cancer Center
JAMES DOROSHOW, Director, Division of Cancer Treatment and Diagnosis, Deputy Director, Clinical and Translational Research, National Cancer Institute
CAROL A. HAHN, Associate Professor of Radiation Oncology, Duke University Medical Center; Medical Director of Radiation Oncology, Duke Raleigh Hospital; Clinical Affairs and Quality Council Chair, American Society for Radiation Oncology
LORI HOFFMAN HŌGG, Veterans Health Administration National Oncology Clinical Advisor, Office of Nursing Services; Cancer Program Director, Albany Stratton Veterans Affairs Medical Center
SAMIR N. KHLEIF, Director, Georgia Health Sciences University Cancer Center, Georgia Regents University Cancer Center
LEE KRUG, Disease Area Head, Immuno-Oncology, Bristol-Myers Squibb
RICHARD A. LARSON, Hematology/Oncology Director, University of Chicago
MICHELLE M. LE BEAU, Arthur and Marian Edelstein Professor of Medicine, Director, Comprehensive Cancer Center, University of Chicago
SHARI LING, Deputy Chief Medical Officer, Center for Clinical Standards and Quality, Centers for Medicare & Medicaid Services
GRETA MASSETTI, Associate Director for Science, Division of Cancer Prevention and Control, Centers for Disease Control and Prevention
DANIEL R. MASYS, Affiliate Professor, Biomedical Informatics, University of Washington
MARTIN J. MURPHY, Chief Executive Officer, CEO Roundtable on Cancer
RICHARD PAZDUR, Director, Office of Oncology and Hematology Products, Food and Drug Administration
STEVEN PIANTADOSI, Phase One Foundation Endowed Chair and Director, Samuel Oschin Comprehensive Cancer Institute, Cedars-Sinai Medical Center
JENNIFER A. PIETENPOL, Director, Vanderbilt-Ingraham Cancer Center, Benjamin F. Byrd, Jr., Professor of Oncology, Professor of Biochemistry, Vanderbilt University
MACE L. ROTHENBERG, Chief Medical Officer & Senior Vice President, Clinical Development & Medical Affairs, Pfizer Oncology
AUGUST SALVADO, Senior Vice President, US Oncology Clinical Development & Medical Affairs, Novartis Oncology
ANDREW SCHIERMEIER, Senior Vice President, Head of Global Oncology, Merck Serono
RICHARD SCHILSKY, Chief Medical Officer, American Society of Clinical Oncology
DEBORAH SCHRAG, Chief, Division of Population Sciences, Professor of Medicine, Department of Medical Oncology, Dana-Farber Cancer Institute, Harvard Medical School
YA-CHEN TINA SHIH, Professor of Health Economics, Chief, Section of Cancer Economics and Policy, Department of Health Services Research, University of Texas MD Anderson Cancer Center
ELLEN V. SIGAL, Chair and Founder, Friends of Cancer Research
RALPH WEICHSELBAUM, Daniel K. Ludwig Professor and Chair, Department of Radiation Oncology; Director, Ludwig Center for Metastasis Research, The University of Chicago Medical Center
GEORGE J. WEINER, Director, Holden Comprehensive Cancer Center, University of Iowa, President, Association of American Cancer Institutes
National Cancer Policy Forum Staff
SHARYL J. NASS, Forum Director and Director, Board on Health Care Services
PATRICK ROSS, Research Assistant
HANNAH DURING, Senior Program Assistant
PATRICK BURKE, Financial Associate
This page intentionally left blank.
Reviewers
This workshop summary has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published workshop summary as sound as possible and to ensure that the workshop summary meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the process. We wish to thank the following individuals for their review of this workshop summary:
Elena Elkin, Memorial Sloan Kettering Cancer Center
Louis Henderson, University of North Carolina
Ed Sickles, University of California, San Francisco
Dana Smetherman, Oschner Health Service
Although the reviewers listed above have provided many constructive comments and suggestions, they did not see the final draft of the workshop summary before its release. The review of this report was overseen by Sue Curry, University of Iowa. She was responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the rapporteurs and the institution.
This page intentionally left blank.
Acknowledgments
We thank the American Cancer Society for sponsoring for this project through a generous donation from the Longaberger Company’s Horizon of Hope Campaign®.
Support from the many annual sponsors of the Institute of Medicine’s National Cancer Policy Forum is crucial to the work of the Forum. Federal sponsors include the Centers for Disease Control and Prevention and the National Cancer Institute/National Institutes of Health. Nonfederal sponsors include American Association for Cancer Research, American Cancer Society, American Society for Radiation Oncology, American Society of Clinical Oncology, American Society of Hematology, Association of American Cancer Institutes, AstraZeneca, Bristol-Myers Squibb, Cancer Support Community, CEO Roundtable on Cancer, EMD Serono, Helsinn Healthcare SA, LIVESTRONG Foundation, National Comprehensive Cancer Network, Novartis Oncology, and the Oncology Nursing Society.
The Forum wishes to express its gratitude to the expert speakers whose presentations helped define the current challenges and opportunities for assessing and improving the quality of imaging interpretation in breast cancer screening and diagnosis. The Forum also wishes to thank the members of the planning committee for their work in developing an excellent workshop agenda.
This page intentionally left blank.
Contents
HISTORY OF MAMMOGRAPHY OVERSIGHT AND EFFORTS TO IMPROVE QUALITY
Breast Imaging Centers of Excellence
Digital Mammography, CAD, and 3D Mammograms
Ongoing Need for Quality Improvement
CHALLENGES OF QUALITY INTERPRETATION
Radiologist Training and Certification
Radiologic Technologist Training, Certification, and Licensing
ASSESSING INTERPRETIVE PERFORMANCE
Audits of Interpretive Performance
Accounting for Variability in Audit Metrics
Other Factors That Can Affect Reader Metrics
Criteria for Adequate Interpretive Performance
Implementing Performance Criteria
INTERVENTIONS FOR IMPROVING INTERPRETIVE PERFORMANCE
Test Sets for Quality Assurance
Lessons Learned from Test Set Use
Correlation with Clinical Performance
Assessing and Improving Mammography (AIM) Intervention
MAMMOGRAPHY REGULATION AND QUALITY ASSURANCE IN OTHER COUNTRIES
POTENTIAL OPTIONS FOR ADDRESSING MAMMOGRAPHY QUALITY CHALLENGES
Seeding Positive Mammograms in Clinical Practice
Prior Exams and Supplemental Images
Boxes, Figures, and Tables
BOXES
1 Suggestions Made by Individual Participants
4 Explanations of the Themes That Categorize MISA Examination Questions
FIGURES
1 National Mammography Database participating facilities, March 2015
2 ACR Breast Imaging Centers of Excellence
3 Variability in mammography interpretive performance in the United States
4 Average performance by years of experience for screening mammography
5 Analysis of first reader performance in the U.K. National Breast Screening Program
TABLES
2 Disparities in Breast Cancer Screening Access, Use, and Outcomes
3 Final Cut-Points for Screening Mammography Using the Angoff Method
4b Final Cut-Points for Diagnostic Mammography to Work Up a Breast Lump Using the Angoff Method
5 Combined Minimally Acceptable Performance Criteria for Radiologists with Complete Cancer Capture
6 Analysis of First Reader Performance in the U.K. National Breast Screening Program
ACR |
American College of Radiology |
ACS |
American Cancer Society |
AHRQ |
Agency for Healthcare Research and Quality |
ARRT |
American Registry of Radiologic Technologists |
BCSC |
Breast Cancer Surveillance Consortium |
BI-RADS |
Breast Imaging Reporting and Data System |
CAD |
computer-aided diagnosis |
CME |
Continuing Medical Education |
CMS |
Centers for Medicare & Medicaid Services |
DCIS |
ductal carcinoma in situ |
FDA |
Food and Drug Administration |
FN |
false negative |
FP |
false positive |
HIPAA |
Health Insurance Portability and Accountability Act of 1996 |
IOM |
Institute of Medicine |
MISA |
Mammography Interpretive Skills Assessment |
MOC |
maintenance of certification |
MQSA |
Mammography Quality Standards Act |
MRI |
magnetic resonance imaging |
NCI |
National Cancer Institute |
NMD |
National Mammography Database |
NMQAAC |
National Mammography Quality Assurance Advisory Committee |
PERFORMS |
Personal Performance in Mammography Screening |
PPV |
positive predictive value |
RT |
Registered Technologist |
RVU |
relative value unit |
TN |
true negative |
TP |
true positive |