NATIONAL ACADEMY PRESS
2101 Constitution Avenue, N.W. Washington, D.C. 20418
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
This study was supported by Contract No. 282-95-0034 between the National Academy of Sciences and the U.S. Department of Health and Human Services. Support of the work of the Committee on National Statistics is provided by a consortium of federal agencies through a grant from the National Science Foundation (No. SBR-9709489). Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that provided support for this project.
Library of Congress Cataloging-in-Publication Data
Health performance measurement in the public sector : principles and policies for implementing an information network / Edward B. Perrin, Jane S. Durch, and Susan M. Skillman, editors ; Panel on Performance Measures and Data for Public Health Performance Partnership Grants, Committee on National Statistics, Commission on Behavioral and Social Sciences and Education, National Research Council.
p. cm.
Includes bibliographical references (p. ).
ISBN 0-309-06436-8 (pbk.)
1. Public health—United States—Information services. 2. Public health—United States—Evaluation—Data processing. I. Perrin, Edward. II. Durch, Jane. III. Skillman, Susan M. IV. National Research Council (U.S.). Panel on Performance Measures and Data for Public Health Performance Partnership Grants.
RA423.2 .H43 1999
362.1'0973—dc21 99-6159
Additional copies of this report are available from
National Academy Press,
2101 Constitution Avenue, N.W., Box 285, Washington, D.C. 20055. Call (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area). This report is also available on line at http://www.nap.edu
Printed in the United States of America
Copyright 1999 by the National Academy of Sciences. All rights reserved.
PANEL ON PERFORMANCE MEASURES AND DATA FOR PUBLIC HEALTH PERFORMANCE PARTNERSHIP GRANTS
EDWARD B. PERRIN (Chair),
Department of Health Services, School of Public Health and Community Medicine, University of Washington
JOHN C. BAILAR III,
Department of Health Studies, University of Chicago
THOMAS J. BARRETT,
Mental Health Services, Colorado Department of Human Services
DAVID W. FLEMING,
Center for Disease Prevention and Epidemiology, Oregon Health Division
V. JOSEPH HOTZ,
Departments of Economics and Policy Studies, University of California, Los Angeles
IRA R. KAUFMAN,
University of Medicine and Dentistry of New Jersey, Department of Environmental and Community Medicine, Robert Wood Johnson Medical School
JOHN R. LUMPKIN,
Illinois Department of Public Health
WILLIAM A. MORRILL,
Mathtech, Inc., Princeton, New Jersey
R. HEATHER PALMER,
Center for Quality of Care Research and Education, Harvard School of Public Health
J. SANFORD SCHWARTZ,
School of Medicine and the Wharton School, University of Pennsylvania
MARY E. STUART,
Maryland Collaboratory for Business and Health, Department of Sociology and Anthropology, University of Maryland, Baltimore County
CYNTHIA P. TURNURE,
Minnesota Department of Health
PAUL J. WIESNER,
DeKalb County Board of Health, Decatur, Georgia
GAIL R. WILENSKY,
Project Hope, Bethesda, Maryland
JANE S. DURCH, Study Director
JEFFREY J. KOSHEL, Study Director (through December 1997)
SUSAN M. SKILLMAN, Senior Research Associate
TELISSIA M. THOMPSON, Senior Project Assistant
COMMITTEE ON NATIONAL STATISTICS 1998–1999
JOHN E. ROLPH (Chair),
Department of Information and Operations Management, University of Southern California
JOSEPH G. ALTONJI,
Institute for Research on Poverty and Department of Economics, Northwestern University
JULIE DaVANZO, RAND,
Santa Monica, California
WILLIAM F. EDDY,
Department of Statistics, Carnegie Mellon University
WILLIAM KALSBEEK,
Department of Biostatistics, University of North Carolina
RODERICK J.A. LITTLE,
School of Public Health, University of Michigan
THOMAS A. LOUIS,
School of Public Health, University of Minnesota
CHARLES F. MANSKI,
Department of Economics, Northwestern University
WILLIAM NORDHAUS,
Department of Economics, Yale University
JANET L. NORWOOD,
Urban Institute, Washington, D.C.
EDWARD B. PERRIN,
Department of Health Services, School of Public Health and Community Medicine, University of Washington
PAUL R. ROSENBAUM,
Department of Statistics, Wharton School, University of Pennsylvania
FRANCISCO J. SAMANIEGO,
Division of Statistics, University of California, Davis
RICHARD L. SCHMALENSEE,
Sloan School of Management, Massachusetts Institute of Technology
MIRON L. STRAF, Director
ANDREW A. WHITE, Deputy Director
Contents
Preface
This is the second of two reports from the Panel on Performance Measures and Data for Public Health Performance Partnership Grants. In summer 1995, as part of its response to the need to ensure that public funding of health programs is related to documented program performance, the Office of the Assistant Secretary for Health of the U.S. Department of Health and Human Services (DHHS) requested that the National Research Council (NRC) convene an expert panel to examine and report on the technical issues involved in establishing performance measures in ten substantive program areas. Such measures were to be required as part of the proposed Performance Partnership Grants (PPG) Program, under which each state was to negotiate with DHHS an action plan with performance objectives that would be specific in terms of outcomes, processes, and capacity to be achieved within a 3- to 5-year period.
The panel divided its work into two phases. The objective of the first phase was to develop a report to the Secretary of DHHS on performance measures in specified areas that would be useful to the PPG Program in its first 3–5 years. The report Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health (National Academy Press, Washington, D.C., 1997) presents the findings and recommendations resulting from the first phase of the panel's work. The focus of the second phase was on recommended improvements to existing local, state, and federal data systems that would facilitate the collection of data for use with existing and proposed performance measures. This two-phase approach was adopted by the panel because of our conclusion that developmental work is needed in both the public and private sectors to adapt, refine, or add to existing data systems to make them more useful in performance measure-
ment, and to delineate more clearly the complex causal links between program processes and outcomes.
Although formal legislation mandating PPGs has not been adopted, various DHHS programs and a growing number of states and communities are moving to monitor and analyze health outcomes on their own. Moreover, there appears to be a growing consensus within the public health, substance abuse, and mental health communities about the value of performance measurement. Indeed, many people believe that the case for increasing, or even maintaining, public funding will depend on documented program performance. The development of performance measures is a continuation of earlier efforts to assess progress toward important public health goals. The broad acceptance of immunization rates and other such measures developed for Healthy People 2000, for example, has been instrumental in the creation of databases and the mobilization of resources in many jurisdictions to assess progress toward those objectives.
The work of this panel was furthered significantly by four regional meetings of state officials and consumers convened by DHHS early in the process—in Portland, Oregon; San Francisco, California; Chicago, Illinois; and Philadelphia, Pennsylvania—and by input from several national associations of state agency administrators (the Association of State and Territorial Health Officials, the National Association of County and City Health Officials, the National Association of State Alcohol and Drug Abuse Directors, and the National Association of State Mental Health Program Directors). Our work during the second phase of the project was aided by the contributions of participants in a workshop held by the panel in July 1997. The workshop agenda and participants are listed in Appendix B. We also wish to thank Winnie Mitchell of the Substance Abuse and Mental Health Services Administration and David Moriarty of the Centers for Disease Control and Prevention, who provided special assistance by arranging several meetings with agency staff members.
The panel was fortunate to have for its deliberations two specially commissioned working papers. ''Improving Federal-State Data Collection to Monitor Program Performance" was prepared by the Science and Epidemiology Committee of the Association of State and Territorial Chronic Disease Program Directors and the Council of State and Territorial Epidemiologists. Christopher Maylahn, New York State Department of Health, chaired the group that prepared this paper. "Quality Education Data: Unprecedented Opportunity for a Decade to Build" was prepared by Emerson Elliott, National Council for Accreditation of Teacher Education, and John Ralph, National Center for Education Statistics. Copies of these papers are available from the Committee on National Statistics (2101 Constitution Avenue, N.W., Washington, DC 20418).
This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the NRC's Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institu-
tion in making the published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We thank the following individuals for their participation in the review of this report: Thomas F. Babor, School of Medicine, University of Connecticut Health Center; Maureen M. Henderson, School of Medicine, University of Washington (emeritus); Ed Lichtenstein, Oregon Research Institute, Eugene; Thomas Louis, School of Public Health, University of Minnesota; J. Michael McGinnis, scholar in residence, National Research Council; Barbara J. McNeil, Department of Health Care Policy, Harvard Medical School; Donald M. Steinwachs, School of Hygiene and Public Health, Johns Hopkins University; Bernard Turnock, Community Health Sciences, University of Illinois, Chicago; and Albert W. Wu, Health Services Research Center, Johns Hopkins University.
While the individuals listed above have provided constructive comments and suggestions, it must be emphasized that responsibility for the final content of this report rests entirely with the authoring committee and the institution.
The panel appreciates the assistance of the staff of the Committee on National Statistics (CNSTAT) in preparing this report. We wish to thank in particular Miron Straf, staff director of CNSTAT, for developing the original project design; Telissia Thompson for organizing panel meetings and carrying out innumerable administrative tasks; and Kathleen Saslaw for providing financial management for the study. Several former CNSTAT staff members provided valuable assistance at various stages in the production of the panel's two reports: Anu Das, Theresa Raphael, Michelle Ruddick, and Ashley Bowers. We appreciate the editorial work of Rona Briere and the guidance of Eugenia Grohman, associate director for reports of the Commission on Behavioral and Social Sciences and Education. Thanks also go to Sue Skillman, University of Washington, for her valued assistance to the committee chair. Finally, the panel extends its special thanks to Jeff Koshel, the panel's study director for the first part of the study, and to Jane Durch, for her very capable management of and contribution to the process leading to the production of this second report.
EDWARD B. PERRIN, CHAIR
PANEL ON PERFORMANCE MEASURES AND DATA FOR PUBLIC HEALTH PERFORMANCE PARTNERSHIP GRANTS
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M. Alberts is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. William A. Wulf is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce M. Alberts and Dr. William A. Wulf are chairman and vice chairman, respectively, of the National Research Council.
Health Performance Measurement in the Public Sector
This page in the original is blank. |