Clinical Practice Guidelines: Directions for a New Program (1990)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

ATTRIBUTES OF GOOD PRACTICE GUIDELINES 60 relationship between the evidence and recommendations.3 In the context of the Forum's practical needs, the committee recommends that an assessment of validity look for 11 elements in a set of guidelines. These elements are listed below: • Projected health outcomes • Projected costs • Relationship between the evidence and the guidelines • Preference for empirical evidence over expert judgment • Thorough literature review • Methods used to evaluate the scientific literature • Strength of the evidence • Use of expert judgment • Strength of expert consensus • Independent review • Pretesting. Projected Health Outcomes A key reason for developing and using practice guidelines is the expectation that they will improve health outcomes. Ideally, a set of guidelines should give practitioners, patients, and policymakers an explicit description of the projected health benefits (for example, a reduction in postoperative infection rates from 4 to 2 percent) and the projected harms or risks (for example, an increase in the risk of incontinence from 10 to 20 percent). If reasonable and technically feasible, the net effects of a course of action—the balance of benefits against risks or harms—also need to be estimated. In addition, projected outcomes should be compared with those for alternative courses of care for the clinical condition in question. The ideal set of projections just described will often be technically or practically beyond the reach of guidelines developers. In most situations, the assistance of outside consultants or specialized technical advisory panels will be at least helpful or at most essential; yet even with such help, projecting health outcomes is intrinsically a complex and subjective process. The nature of the process makes it particularly important that the methods 3 The committee discussed four types of validity: face validity, content validity, criterion validity, and construct validity. These concepts may have the following connotations when applied to practice guidelines. First, the content of guidelines and their development processes need to be plausible, at first pass, to practitioners—to have face validity. Second, content validity has to be assessed by reviewing the scientific evidence on which a set of guidelines are based—how much evidence there is, how clear it is, how directly it relates to the guidelines, how sound its methodology is. Third, for a prospective assessment of criterion validity, one judges whether the guidelines would be likely to produce predicted results when applied in the real world of health care delivery. Construct validity involves the fit of the guideline to broader scientific theories. 

ATTRIBUTES OF GOOD PRACTICE GUIDELINES 61 for projecting outcomes, the limitations in these methods, and the evidence for such projections be described. INFORMATION ABOUT CLINICAL BENEFITS AND ABOUT RISKS OR HARMS 1. What are the potential benefits and risks (type and importance) for individual patients associated with an intervention or procedure? 2. What is the probability that a benefit or harm will occur? 3. Do benefits exceed harms? By how much? 4. What characteristics of delivery settings or patients affect the probability of a benefit or harm? 5. How are benefits and harms distributed across time and populations? 6. How do these benefits and harms compare for alternative practices? INFORMATION ABOUT COSTS AND SAVINGS 1. What is the production cost/price of a particular test, intervention, etc.? What are other important costs, such as for program administration? 2. What is the total cost, given the projected number of services provided? 3. What is the cost per unit of some benefit, including not only the immediate cost of providing service but the cost of follow-up services (for example, the cost of screening for cancer and the cost of treating cancers that are detected)? 4. What costs may be averted or saved (individual, total)? 5. How do costs and savings compare for alternative courses of action? FIGURE 3-1 A possible checklist for describing benefits, risks, and costs. SOURCE: This figure is adapted in part from the National Research Council report, Improving Risk Communication (1989). When empirical evidence is limited, potential effects may only be listed, not quantitatively compared or weighed. In addition, in cases in which patient preferences about different risks and benefits may differ, practice guidelines will need to be sensitive to such variation, and a comprehensive statement of net effects may have to be omitted (Mulley, 1990). In any event, a systematic effort should be made to provide practitioners, patients, and others with information that will help them make their own judgments of the balance of benefits and risks. Figure 3-1 provides a simple checklist of outcomes that might be estimated. The particular outcomes to be considered will vary with the clinical conditions and practices under consideration. To support the eventual evaluation of the actual impact of guidelines, guidelines developers should indicate what information related to outcomes will be needed, where it can be obtained, and whether better means for collecting and analyzing data need to be established to permit evaluation.