Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Consensus Development at the NIH Improving the Program Report of a Study by a Committee of the Institute of Medicine Council on Health Care Technology NATIONAL ACADEMY PRESS · WASHINGTON, D.C. · 1990
NOTICE: The project that is Me subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of me committee responsible for the report were chosen for their special competences and with regard for ap- propriate balance. This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The Institute of Medicine was chartered in 1970 by the National Academy of Sciences to enlist distinguished members of the appro- priate professions in the examination of policy matters pertaining to the health of the public. In this, the Institute acts under both the Academy's 1863 congressional charter responsibility to be an ad- viser to me federal government and its own initiative in identifying issues of medical care, research, and education. Support for this project was provided by the National Institutes of Health, Office of Medical Applications of Research, pursuant to Contract No. NO 1-OD-8-21 10. Library of Congress Catalog Card Number 90-60935 International Standard Book Number 0-309-04242-9 Publication IOM-90-04 Additional copies of this report are available from: National Academy Press 2101 Constitution Avenue, N.W. Washington, D.C. 20418 S131 Printed in the United States of America
Acknowledgments The committee thanks the individuals who provided insight into the National Institutes of Health Consensus Development Program. In particular, the committee acknowledges Donald S. Fredrickson, former director of the National Institutes of Health; John Ferguson, director of the Office of Medical Applications of Research; Jerry Elliott, program analyst, Office of Medical Applications of Research; and Jacqueline Kosecoff, executive vice president of Value Heath Sciences, Inc., and professor of medicine and public health, Univer- sity of California at Los Angeles. The committee would like to express its appreciation of the Insti- tute of Medicine staff who facilitated the work of this committee, especially Elaine Steen, Clifford Goodman, Sharon Baratz, and Ethan Halm. We are grateful to Holly Dawkins and Evanson Joseph for logistical support and help in preparing this manuscript. The committee also thanks the members of He Council on Health Care Technology and the Institute of Medicine Council, who re- viewed and provided useful comments on the report. · · ~
COMMITTEE TO IMPROVE THE NATIONAL INSTITUTES OF HEALTH CONSENSUS DEVELOPMENT PROGRAM COUNCIL ON HEALTH CARE TECHNOLOGY Chair J. SANFORD SCHWARTZ Executive Director, Leonard Davis Institute of Heady Eco- nomics, Associate Professor of Medicine and Health Care Systems, and Robert D. Eilers Associate Professor of Health Care Management and Econom- ics, University of Pennsylvania MARCIA ANGELL Executive Editor, The New England Journal of Medicine ROBERT H. BROOK* Deputy Director, Health Sci- ences Program, The RAND Cor- poration, and Chief, Division of Geriatrics and Professor of Medicine and Public Health, University of California at Los Angeles FLOYD W. DENNY* Professor of Pediatrics, School of Medicine, University of North Carolina at Chapel Hill ROBERT M. DONALDSON, JR. Deputy Dean, Yale University School of Medicine ~Member, l~stitute of Medicine V DENNIS PAYBACK Professor of Industrial Engineer- ing and of Preventive Medicine, University of Wisconsin-Madi- son ARNOLD D. KALUZNY Professor of Health Policy and Administration, School of Public Health, University of North Carolina at Chapel Hill R. BRIAN HAYNES Professor of Clinical Epidemiol- ogy and Medicine, McMaster University JONATHAN LEFF Editonal Consultant, Former Director, Consumer Reports Books, South Hero, Vermont STEPHEN G. PAUKER Professor and Chief, Division of Clinical Decision Making, De- pa~tment of Medicine, New England Medical Center, Tufts University School of Medicine CAROL K. REDMOND Professor and Chair, Depart- ment of Biostatistics, Graduate School of Public Health, Uni- versity of Pittsburgh
INSTITUTE OF MEDICINE SAMUEL O. THIER, President ENRIQUETA C. BOND, Executive Officer Project Staff CLIFFORD GOODMAN, Director, Council on Health Care Technology ELAINE B. STEEN, Project Officer SHARON R. BARATZ, Research Associate HOLLY DAWKINS, Project Assistant EVANSON JOSEPH, Project Assistant V1
Preface One of the goals of the Institute of Medicine's Council on Health Care Technology has been to promote the development and applica- tion of technology assessment in health care. Assessment of medi- cal technologies currently relies extensively on group judgment or "consensus development" processes. These methods are used by medical professional societies, private and public third-party payers, biomedical research agencies, and others to assess state-of-the-art medical and surgical procedures and other technologies, to define standard and accepted medical practices, to bridge gaps and resolve disparities among research findings, and to establish coverage and reimbursement policies. Despite the often weighty implications for health policy of decisions based on group judgment, these processes often are applied with little systematic or critical examination of their conduct, utility, or appropriateness for technology assessment. The panels convened for group judgment must weigh, integrate, and interpret the available evidence, experience, beliefs, and values in order to formulate guidelines, recommendations, or other find- ings. The technologies assessed may be evolving rapidly; the evi- dence available to the panels may consist of a sparse patchwork of research results of varying quality. Panelists may be subject to biases and errors of reasoning; experts and nonexperts alike may be vii
V111 PREFACE subject to oversimplification, empiricism, case-selection biases, in- centives, and advocacy. Group judgment methods have been developed and used in many fields, including the social, political, physical, and biomedical sci- ences; engineering; defense; environmental studies; and other do- mains. To improve group judgment for assessing medical technolo- gies, the council urges programs to learn from alternative techniques and models used in other fields for engaging groups of experts in rendering well-founded and informed findings. The council is conducting a three-part examination of group judg- ment methods for assessing medical technologies: an international workshop on consensus development for medical technology, a study to improve the National Institutes of Health (NIH) Consensus De- velopment Program, and a workshop to improve group judgment for medical practice and technology assessment. First, in conjunction with the June 1989 annual meeting of the International Society for Technology Assessment in Health Care, the council coordinated a workshop that compared and contrasted national consensus develop- ment programs. The proceedings of the workshop (Improving Con- sensus Development for Health Technology Assessment: An Inter- national Perspective, C. Goodman and S. Baratz, eds., 1990), in which representatives of ~ ~ countries participated, include a consoli- dated set of participants' recommendations for improving these ap- proaches. Second, as requested by the NTH Office of Medical Ap- plications of Research, a committee operating under the aegis of the council conducted a November 1989 meeting and follow-up work to review and provide recommendations for improving the NTH Con- sensus Development Program. The committee's recommendations are presented in this report. Third, a May 1990 workshop sponsored by the council will convene those who conduct, participate in, and use the results of group judgment efforts of medical professional societies, third-party payers, regulatory agencies, and others. This workshop will focus on improving key methodologic aspects of these programs and sharing newer approaches to improve their application for rendering practice guidelines, coverage decisions, and other as- sessment-related findings and policies. The councilts goal in undertaking this three-part effort is to im- prove group judgment for assessing medical technologies. This re- port makes a significant contribution toward that goal. Although written specifically for the NIH Consensus Development Program,
PREFACE 1X the recommendations presented in this report address many of the challenges of any group process intended to produce a consolidated, well-substantiated expert judgment for direct application to policy- making. Furthermore, by requesting this evaluation, NIH may en- courage others in the field to undertake efforts to examine and im- prove their group judgment efforts.
Contents Summary ....... 1. Introduction......................... 2. Recommendations for Program Structure 3. Recommendations for Program Function. References ............ · 3 9 .18 34 Appendixes A. Background Paper for the Committee to Improve the National Institutes of Health Consensus Development Program .... 41 Sharon R. Baratz B. NTH Consensus Development Conferences.............. C. Questions from NIH Consensus Development Conferences (October 1987-September 1989~. D. General Bibliography on Consensus Development X1 65 71 75 · · ~
