Using Performance Monitoring to Improve Community Health: Conceptual Framework and Community Experience (1996)

The Calhoun County Health Improvement Program (CCHIP) is a community-based program that began in 1993. Its mission is to improve community health in Calhoun County, Michigan. The county has a population of 136,000, with a minority population of 17,000. The county's health status indicators fall below statewide averages. CCHIP was developed with funding from the W.K. Kellogg Foundation. A participatory approach that includes providers, consumers, and payers is taken. It views ''health care'' more broadly than merely the provision of "medical care." Personal responsibility and primary prevention are central to its vision. Its organizational structure is conceived as a circle with improved health at the intersection of four quadrants: neighborhood groups, membership organizations, a governing board, and implementation teams.

Health assessment is conducted through collaboration between the Community Assessment Implementation Team (a CCHIP-based team) and the Calhoun County Health Department. Together, they have developed a health outcomes report and have shared responsibility for community response to the report.

The program has developed long-range goals based on a five-year plan with six focus areas: (1) community decision making, (2) community-wide health care coverage, (3) a comprehensive delivery system, (4) an integrated administrative structure, (5) a community-based health care information system, and (6) community assessment. The goals include decreasing the risks to health, increasing access to cost-effective health care through the establishment of a purchasing cooperative, improving decision making through a community health information network, and changing local and state policy to reflect community values and community decision-making processes.

In response to questions from the committee, workshop panelists discussed their experiences with performance monitoring. They specifically addressed identifying stakeholders, selecting health priorities and indicators, using indicators for accountability, gaining community support for performance monitoring, and implementing a performance monitoring system.

As described by the panelists, identification of stakeholders proceeds in one of two ways, depending on whether the stakeholders are involved in defining or responding to the problem. When stakeholders are involved in problem identification, it is best to cast a wide net, leading particular groups to self-identify as stakeholders and become active collaborators. The stakeholder group may evolve as the process moves from problem identification to intervention to evaluation. It was noted that in the public sector different agencies or different personnel within agencies may become involved depending on the stage of the program.

When a problem is already defined and an intervention is suggested by existing data (as in TAPII), the participation of key stakeholders able to produce the desired results can be actively sought. In Arizona, the process was facilitated by participation of the governor, who convened a meeting of identified stakeholders. Groups with divergent interests may be able to cooperate in implementing solutions to defined problems when data and interventions are available to focus their joint efforts.

Participation by "consumers," that is, members of the general public, was mentioned as an important stakeholder issue. Ideally, participants should reflect the various population groups in the community, based on factors such as age, race or ethnicity, and neighborhood. All members of coalitions are consumers in a sense, but most participants are invested in particular interests. Engaging those participants who are not affiliated with particular stakeholder groups can be difficult. Barriers include the difficulty in identifying interested individuals and the commitment in time and energy that is required. Other barriers to participation can include meetings scheduled for normal working hours or added costs such as transportation and child care. TAPII found that because consumers participated for only a short period of time focus groups and community surveys were helpful for bringing their perspectives into the process.

Epidemiologic data are often used to guide the selection of performance indicators, but community interest may argue for focusing attention on specific issues or indicators, even in the absence of supporting epidemiologic data. It was noted that in some cases, a "triggering event" may focus attention on a particular health issue. The trigger might be a severe adverse event such as a measles epidemic or a more positive stimulus such as the availability of funds and other resources earmarked for specific topics.

Social determinants of health (e.g., income, family structure) and epidemiology are sometimes viewed as separate issues because epidemiology traditionally is associated with the biomedical model. However, the scope of

epidemiology has expanded to include measurement of social factors, and epidemiologic data can drive the development of interventions in the social realm. Some workshop participants believe that the distinction between the two is artificial.

A discussion of the utility of epidemiologic health status data based on small sample sizes generated dissenting views. Some participants commented that sampling error is too large to make detailed follow-up measurements a worthwhile use of resources. Changes on the order of a few percent per year are extremely difficult to measure at the community level. Even if changes are measurable, communities may lack the resources to collect such data accurately. Other participants suggested that although measurement error can be a problem, it is essential to quantify problems and the effects of interventions. Otherwise, efforts to solve problems could be completely misguided. They suggested that combining quantitative and qualitative information can provide a more meaningful picture of a community's health.

Workshop participants noted that there is tension between the need for standardized performance indicators and the need for community flexibility in defining indicators. Standardized indicators are advantageous for making comparisons within the between communities, for simplifying the synthesis of data from different sources, and for developing data systems. However, in designing and monitoring interventions in individual communities, the development of more specific indices may be helpful and standardization may be less important.

It was also suggested that the dichotomy between standardization and individualization is artificial. Most programs would benefit from a combination of the two. A basic set of indicators could be developed, with modifications based on specific community needs. Alternatively, a broad spectrum of questions could be developed from which communities could choose appropriate subsets. The selection of indicators may be especially difficult in a diverse community. Participants pointed out that performance indicators could be coordinated with currently existing health indicators in the private and public sectors such as HEDIS 2.0 (NCQA, 1993), APEXPH (NACHO, 1991), Model Standards (APHA et al., 1991), and Healthy People 2000 (USDHHS, 1991).

Development of performance indicators requires stakeholder involvement. Some participants emphasized the importance of including private-sector stakeholders early in the planning stages in order to frame questions in a manner compatible with existing data bases. Distinguishing performance indicators from overall program evaluation was raised as an important distinction. The speaker suggested that performance monitoring should focus on the component parts of

an intervention whereas program evaluation should examine the overall outcome.

Workshop participants agreed that defining accountability—which "actors" are responsible for what functions—is extremely important. Accountability for both process and outcome goals needs to be determined. The programs described at the workshop vary in how they hold groups accountable. For example, TAPII has led to ties between capitation and immunization rates, which fosters competition among health plans. Competition as a mechanism to gain accountability may be especially useful in the private sector. The Escondido program addresses accountability through formal written agreements detailing participation in the project, contract arrangements with providers, target rates for screening and performance, and management information systems for the integrated services. In other programs represented at the workshop, accountability is less explicitly defined. According to one panelist, it is hoped that "providers (will) come forward and increase their provision of services or education as it relates to the objectives."

The workshop participants reported that positive outcomes engender positive community response and that achievement of short-term objectives serves to cement community cooperation. They also commented on the importance of communicating realistic expectations to prevent discouragement with slower progress toward long-term objectives. Balancing short-term and long-term objectives helps maintain motivation.

Strong leadership is necessary to prevent coalitions from splintering into groups with self-serving agendas. On the other hand, outside leadership cannot substitute for communities' developing their own momentum to maintain programs. It was suggested that community groups need to be involved from the start in order for a community to be empowered and to continue projects regardless of changes in political personnel.

Tension between health problems articulated by the community and health problems identified by data analysis can potentially undermine community support for performance monitoring and health improvement activities. Participants suggested that the specific approach to selecting health issues should involve a larger community collaborative. However, the conceptual bases for selecting health issues should be founded on research that suggests that effective interventions will be possible.