Commentary13
The workshop discussions served as the basis for a commentary on community-based performance monitoring and issues to which the committee should give further attention. It was noted that the day's discussions focused broadly on community health improvement and community activation, rather than focusing more narrowly on performance indicators. This perspective is consistent with many community-based efforts to reduce health risks and prevent disease, such as the National Heart Lung and Blood Institute's cardiovascular risk reduction programs or the Kaiser Family Foundation's community health promotion grants program. Coalition building was central to these programs. They emphasized ensuring community involvement and participation of key stakeholders; needs assessment; project implementation based on the needs assessment; and program monitoring and evaluation.
It was suggested that although this approach, which is based on collaboration and community empowerment, is consistent with public health values, the evidence to date suggests that the model, as implemented in the past, may not work. Coalitions include varied interest groups and may be swayed by political concerns. The process may not select the most effective interventions at a population level. Efforts are being made, however, to bridge the gap that seems to exist between the community activation approach and the science of health improvement (Wandersman et al., 1995).
In contrast, the HEDIS approach relies on central planning and oversight. Although its top-down approach may conflict with the values and instincts of public health practitioners, it appears to be effective in promoting change. Its effectiveness was attributed to its visibility, its evidence-based approach, and its use of measures that lend themselves to managerial action.
The speaker proposed a new paradigm for community health improvement based on a synthesis of community partnerships with an evidence-based approach. First, cooperation with the private sector, particularly medical care, would be a key element. Second, the private sector requires a business reason such as competition to participate. Third, performance indicators should be used to focus attention on those health issues and interventions supported by scientific evidence, as well as to generate and sustain accountability. Finally, the partnership should generate specific implementation strategies. In sum, performance indicators should support a community participation model by helping community partnerships set priorities and design interventions based on evidence.
Also critical to consider is the issue of accountability. In the speaker's view, accountability should be clearly assigned within the community. It must also
rest on all who have responsibility to act. For there to be true accountability, performance must be monitored.
The workshop discussions suggested that performance indicators are used for multiple purposes: to identify problems and generate hypotheses, as political tools for mobilization in the community, to suggest ideas for improvement, and in fact, to monitor the performance of specific sectors of the community. Among the characteristics of useful performance indicators is a focus on populations and rates, rather than on absolute numbers of contacts involved in the interventions. In other words, the denominator is as important as the numerator.
Indicators were described as most useful when they focus on areas where improvement is possible. Global health status indicators often have little practical use for guiding health improvement strategies. More useful are indicators that incorporate a "theory of improvement"—that is, they suggest a clear means of moving from measurement to action. Indicators that have been shown to change in intervention studies should be preferred over those that may be more conceptually elegant but may not be able to capture the impact of an intervention. "Responsive" indicators of this sort allow real change to be distinguished from random variation.
The value of standard epidemiologic health needs assessments was questioned. Often, needs assessments merely document problems that are already well known. On the other hand, needs assessments focused on factors in the community that influence program implementation—politics, resources, barriers, key players—may be very useful. The speaker also emphasized that although coalitions are an essential component of community-based health improvement projects, they can consume substantial resources. Participants may, for example, spend an average of 3–4 hours a month conducting coalition-related work. It has also proved difficult to document a relationship between the characteristics of coalition operations and health outcomes achieved. The contributions that coalitions make to health improvement activities need to be better understood.
The committee was urged to articulate a model (or models) of health improvement that specifies use of performance indicators and holds social and nonclinical improvement strategies to the same evidence base as clinical strategies. Such a model should
- help communities clarify accountability and consider ways in which to include the private and public sectors as accountable entities;
- identify performance indicators in the model of health improvement; indicators should not be expected to generate models of community improvement;
- illustrate its concepts with the selection of a limited number of "performance areas" that are characterized by (a) evidence that services affect
