National Academies Press: OpenBook

Adverse Effects of Pertussis and Rubella Vaccines (1991)

Chapter:3 Methodologic Considerations in Evaluating the Evidence

« Previous: 2 Histories of Pertussis and Rubella Vaccines
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 32

3
Methodologic Considerations in Evaluating the Evidence

THE NATURE OF THE EVIDENCE

The committee has undertaken the task of judging whether each of a set of adverse events may occur as a result of exposure to pertussis or rubella vaccines. These judgments have both quantitative and qualitative aspects, and they reflect the evidence examined and the approach taken to evaluate it. In this chapter, the committee describes more fully how it has approached its task, in the hope that readers may then be in a better position to assess and interpret the committee's findings. By offering this information, the committee wishes to make the report useful to those who may seek to update its conclusions as new information is obtained. This chapter outlines the specific questions the committee posed, the types of evidence it identified, its approaches to evaluating reports both singly and collectively, and the nature of the conclusions it felt that logic and evidence permitted. Against this background, details of the analysis and specific conclusions concerning each type of adverse event appear in subsequent chapters.

Attributes both of the adverse events being considered and of the population exposed to vaccine influenced the committee's analysis. The events can be characterized, for example, by their frequency, discreteness, and specificity and by prior knowledge of their etiology and pathogenesis. Events that occur only rarely in exposed persons are more difficult to study than those that occur more frequently. Conditions that are ill-defined, that are

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 33

known to occur in the absence of vaccine exposure, or that generally have unknown causes or mechanisms of development are also inherently difficult to investigate. Under these circumstances, epidemiologic studies offer important advantages over clinical experience and intuition, although these limiting characteristics affect epidemiologic studies also.

When the great majority of the population is exposed, as is generally true for pertussis and rubella vaccines, comparisons between exposed and nonexposed persons become clouded. This is due to the potential for selective factors against vaccination to confound the relation between immunization status and the occurrence of adverse events. For example, a decreased relative risk of SIDS has been observed in several studies in the time period immediately following DPT immunization. Although a protective effect of vaccine cannot be ruled out, it is more plausible that children who are not immunized by the recommended age are at increased risk for SIDS because of other factors, such as socioeconomic status, that are associated with both delaying immunization and SIDS (see Chapter 5). Other aspects of vaccine exposure, such as changes in vaccine formulation, single versus multiple occasions of administration, and the age pattern of administration also bear on interpretation of the evidence.

QUESTIONS TO BE ADDRESSED

What would it mean to say that a vaccine causes one or another type of adverse event? It would not mean that exposure invariably produces the adverse event, nor that all cases of the event were due to the vaccine. Such complete correspondence between exposures and events is by far the exception in public health and does not occur in the present context, or the present review would not be required.

In general matters of health and disease, different causal explanations may apply even to a single disease. For example, the question of what causes typical cases of a particular disease is quite distinct from the question of what causes epidemic outbreaks of that disease. The answers are also distinct, in that the first might be a specific microorganism and exposure conditions of the individual case, whereas the second could entail complex ecologic and social factors suddenly favoring the widespread transmission of the microorganism. Clearly, different senses of ''cause" are implied in these two questions.  Although each of these questions concerns the causation of disease, the answers require different types of evidence. This example suggests the importance of stating clearly the questions about causation to be answered.

In the present review, the committee has been concerned with causal questions of three kinds. The first of these questions about exposure to pertussis or rubella vaccine is, in general, can it cause the specified adverse

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 34

condition? For example, can rubella vaccine cause chronic arthritis? If the conclusion is affirmative, a second question becomes pertinent: How frequently does it cause that condition? Or, how frequently is arthritis a result of rubella vaccination? The third question, which applies to a particular instance or case of an adverse event, is did it cause that specific event? Or, did rubella vaccine cause this particular individual to develop arthritis? Discussion of each of these three types of questions will help to indicate the committee's view of its task.

(1) Can vaccine cause the adverse event?

While the nature of causation has a deep philosophical underpinning, the work of the committee necessarily focused on a pragmatic question: What is the nature of the evidence relevant to drawing its conclusion about causation? In pursuing this question, the committee recognized that an absolute conclusion about the absence of causation may never be attained. As in science generally, studies of adverse events following vaccination are not capable of demonstrating a zero effect, that is, that the purported effect is impossible or could not ever occur. Any instrument of observation has a limit to its resolving power, and this is true as well of randomized clinical trials and epidemiologic studies. Hence, the committee could not prove the absence of any possibility of an adverse event caused by vaccine. Rather, in the absence of evidence suggesting an effect, and especially in the presence of evidence not consistent with causation, the committee could only conclude that the evidence fails to demonstrate an effect.

The evidentiary base that the committee found to be most helpful derived from epidemiologic studies of populations. Here, the primary question is whether an association exists between the exposure (immunization) and the event. To determine whether an association exists, epidemiologists estimate the magnitude of an appropriate quantitative measure (such as the relative risk or the odds ratio1) that describes the joint occurrence of exposures and events in defined populations or groups. Values of relative risk greater than 1 may indicate a positive, or direct (harmful), association and are emphasized in this discussion; values between 1 and 0 may indicate a negative, or inverse (protective), association. The observed relative risk in the study population must be sufficiently distant from unity to meet a stated criterion of significance before an association is said to be apparent.

1 Usage of "relative risk," "odds ratio," or "estimate of relative risk" is not consistent in the literature reviewed and cited in this report. In its own usage, the committee intends that ''relative risk" be used to refer to the results of cohort studies and "estimates of relative risk" or "odds ratio" be used to refer to the results of case-comparison studies (see Glossary of Terms for definitions).

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 35

Formally, in planning an investigation, an epidemiologist poses a hypothesis to the effect that the exposures and events under study are independent, or not associated. Under this hypothesis, the value of the measure of association used is theoretically expected to be approximately 1. This is termed the null hypothesis or the hypothesis of no association. The measure of association derived from the investigation is then tested statistically. To "reject the null hypothesis," or to conclude that exposures and events are not independent, is to conclude that there is evidence of an association.

When more than one epidemiologic study has been conducted, it may be instructive to combine their results so as to reach a stronger conclusion than a single study can provide. This process is described more fully later in this chapter.

Determining whether an observed association is causal requires additional scrutiny. This is because there may be alternative explanations for an observed association. These include errors in the design, conduct, or analysis of the investigation; bias, or a systematic tendency to distort the measure of association from representing the true relation between exposures and events; confounding, or distortion of the measure of association because another factor, related to both exposures and events, has not been recognized or taken into account in the analysis; and chance, the effect of random variation in producing observations that can, in reality, only be approximations to the truth and can, with some probability, sometimes depart widely from the truth.

In deciding whether vaccine can cause any particular type of event, then, it has been the committee's task to judge in each instance whether there is evidence of association from the available studies, and if so, whether it is direct or inverse, and whether it is due to error, bias, confounding, or chance or, instead, due to a causal relation between vaccine and event.

(2) How frequently does vaccine cause the event?

The second type of causal question, which becomes pertinent only if the answer to the first question is affirmative, concerns the proportion of individuals in a specified population who experience the adverse event because of the exposure. The most desirable evidence as a basis for answering this type of question involves knowledge of the rate of occurrence of the event in those who are exposed, the rate in those who are not exposed (the "background" rate of the event in the population), and the degree to which any other differences between exposed and unexposed persons influence the difference in rates. The term attributable risk is generally used to denote the difference in rates between exposed and unexposed groups. This is a simple measure of the frequency with which the occurrence of the event in exposed persons may be due to the vaccine exposure and not to the other

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 36

causes that account for the event in the absence of vaccine exposure (see Glossary of Terms). By also taking into account the frequency of exposure in the population, a further measure can be calculated which indicates the number of cases in a total population that may be due to the exposure.

When information was available, in the presence of an association judged by the committee to be causal, the committee has attempted to indicate how much of the occurrence of the event in question might be caused by vaccine exposure.

(3) Did vaccination cause a particular case of the adverse event?

A third type of causal question is whether, in a specific instance of exposure and an adverse event, it can be concluded that the event was caused by the vaccine exposure. This question is especially pertinent to those types of adverse events for which case reports constitute a substantial part of the evidence available for review.  Three different sets of circumstances bear on the nature of this question. First, it may have been judged in general that the exposure can cause this type of event. In this instance the question concerning any particular case needs to consider the similarity between the circumstances of that case and the circumstances in which the general conclusion was reached that such causation can occur.  If other causes of the same type of event are known, their possible role in this individual case must be considered also.

Second, most evidence for a general proposition of causation may be negative, yet circumstances surrounding a single case may raise the question of whether causation may be attributed to the exposure of this individual. To judge that a particular case was caused by the exposure under these circumstances would entail the reversal of the more general conclusion on the basis of a single case. It seems extremely unlikely that such a reversal could be justified on the basis of the evidence concerning an individual case.

Third, there may be no evidence of the sort required to judge the presence or absence of association. Here the judgment in a specific case would depend on the circumstances concerning that case alone, in isolation from any conclusion about a causal relation in general. On consideration of several aspects of the evaluation of case reports, discussed in the section below, even the strongest affirmative answer based on the individual case would be inherently weaker, or less securely supported, than it would be if a general conclusion supporting vaccination as a cause of the event had been reached. This is not to say that any such conclusion reached on the basis of the case report alone is necessarily incorrect. However, it would be highly subject to error in light of the properties of most of the events considered here and the nature of the evidence available about them in the case report.

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 37

THE BURDEN OF PROOF

In approaching its task, the committee considered the concept of "burden of proof' and its place in such an evaluation. This concept implies that one position or another concerning causation is presumed to be true unless it is offset by evidence to the contrary.  The prior position might be either affirmative or negative. That is, it may be assumed that an exposure is harmful unless sufficient evidence of safety is present; alternatively, it may be assumed that an exposure is safe unless convincing evidence of harmful effects is present. In either case, it is sometimes argued that a burden of proof must be fulfilled before the presumed position is rejected.

In general, it is desirable to avoid making an error in either direction, concluding either that there is or that there is not a causal relation when the opposite is true. Reducing the chance of such mistaken conclusions depends on careful assessment of the evidence, including consideration of possible errors, bias, and confounding.

The role of chance in leading to erroneous conclusions as a result of random variation in sampling or in other respects is customarily handled through formal statistical analyses, which are based on assumptions from probability theory. Statistical measures can suggest the likelihood that conclusions of the presence or absence of association will each be in error. In general, a result is said to have greater statistical significance as the probability of error in accepting an association becomes smaller.2The likelihood that a true association will be correctly detected in an investigation is a statistical property of the investigation, termed its power. Both statistical significance and power reflect the role of chance in scientific observations and the concomitant uncertainty in all scientific conclusions. One obvious implication of this understanding is that the concept of "proof" in its common-sense meaning is not strictly applicable to scientific observations. Even when scientists conclude that an experiment demonstrates ("proves") causation, they know there is a small, statistically definable probability that the conclusion is incorrect.

The committee began its evaluation presuming neither the existence nor the absence of association. It has sought to characterize and weigh the strengths and limitations of the available evidence. Subsequent chapters of the report summarize the evidence concerning each vaccine-event relation under review and present the committee's conclusions. If the first question (can the vaccine cause the adverse event?) was answered affirmatively, and

2 More technically, the level of statistical significance is the probability of observing by chance at least as great a difference as that observed between an "experimental" (exposed) and a "control" (unexposed) group, if the risk of the adverse event were, in truth, identical in the two groups.

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 38

if available information permitted, the second question (how frequently does the vaccine cause the adverse event?) was answered. In a few cases, conclusions based on the answer to the third question (did the vaccine cause the adverse event in this individual case?) are proposed (see, for example, the section on hemolytic anemia in Chapter 6). Of necessity, such a conclusion is especially tentative if the more general question of whether the vaccine can cause the adverse event remains unanswered. Furthermore, the committee's task was not to judge individual cases, except when this was the only way to shed light on the general question of causality.

It should be noted that the committee's charge was to focus on questions of causation and not broader topics, such as cost-benefit or risk-benefit analyses of vaccination, which are not considered in this report. With this orientation to the committee's task and approach in mind, the following sections discuss the characteristics of the types of evidence that bear on the causal questions at hand.

CATEGORIES OF EVIDENCE

Experiments in Humans
Randomized Controlled Trials

Theoretically, the ideal method for assessment of causal relations between treatments and adverse events is the randomized controlled trial because, when appropriate and feasible, it is the most scientifically rigorous method for testing such hypotheses. Randomized controlled trials are experiments in which subjects are randomly allocated, often in a masked fashion, into "treatment" and "control" groups, to receive or not to receive an intervention such as, in the present context, a monovalent vaccine; the control group receives an injection of an inert substance (placebo) or an established alternative vaccine; and both groups are followed up in a strictly comparable manner to determine the relative frequencies of outcomes and events of interest.

Although they are theoretically ideal, such trials have ethical and practical limitations for investigating the causal connections of concern here. Widespread acceptance of routine immunization against pertussis and rubella creates ethical barriers to withholding vaccination from some participants to permit a placebo-controlled trial (Cherry et al., 1988). In addition, because these adverse events are generally rare, the sample sizes required to detect them reliably would be much larger than those required to evaluate vaccine efficacy. In fact, since the first reports of serious adverse events following administration of pertussis and rubella vaccines (Madsen, 1933; Modlin et al., 1975), virtually no placebo-controlled or other experimental

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 39

studies in humans of the adverse events covered in this report have been published (Cherry et al., 1988; Plotkin, 1988; Preblud, 1985).

A number of early studies of pertussis vaccine in the United States and the United Kingdom did include unexposed controls, but these studies were primarily concerned with efficacy and not with adverse events (e.g., Kendrick and Eldering, 1939; Lapin, 1943; Medical Research Council, 1951, 1956, 1959). More recently, a small number of studies have compared adverse reaction rates in children following DPT versus DT vaccine administration (Cody et al., 1981; Pollock et al., 1984) or in adults following rubella vaccine versus natural rubella infection (Tingle et al., 1986). Predominantly, however, pertinent randomized trials have focused on comparisons of pertussis vaccine reaction rates between different injection sites, vaccines of different manufacturers, prior reaction histories, or different immunization schedules (Baraff et al., 1984, 1985; Barkin and Pichichero, 1979). Other studies have compared reaction rates by vaccine type, for example, whole-cell versus acellular pertussis vaccines (e.g., Edwards et al., 1986; Lewis et al., 1986; Pichichero et al., 1987) or between rubella vaccine strains (e.g., Barnes et al., 1972; Isacson et al., 1971; Polk et al., 1982). In general, these studies have been primarily concerned with evaluation of transient reactions, whether local or systemic, and not chronic adverse events of the types included in this review. One exception is a randomized, double-masked, placebo-controlled trial of rubella vaccine and chronic arthritis that is currently in progress in Vancouver, British Columbia, Canada. Results are expected in 1992 (A. Tingle, British Columbia Children's Hospital, personal communication, 1991).

Other Experimental Studies

Thus, few randomized controlled trials contribute to the assessment of the causal questions considered in this report. And although it should be noted for completeness that other experimental approaches, such as formal communitywide comparisons of the impact of vaccination programs, including both beneficial outcomes and adverse events, are applicable in principle, evidence of this type is also generally unavailable.

Experiments in Animals: Animal Models

In principle, experimental studies in animals allow for both rigid control over vaccine exposure and intensive observation for any adverse events that may follow. If an animal model is to be considered valid for the study of a human disease, however, the manifestations of the disease should be similar in the two species. The starting point is generally what is currently known about the human disease.

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 40

With respect to evaluation of pertussis vaccine, the committee found that the information gained from animal models of whooping cough is difficult to apply to humans for two critical reasons: Knowledge of virulence factors for the organism and of the pathogenesis of whooping cough is incomplete and largely superficial, and Bordetella pertussis is not a natural pathogen for animals. Nonetheless, several studies have been conducted and are reviewed below. With respect to rubella, the committee could find no studies of animal models of either the disease or rubella vaccine-related illness that were specific to the adverse events under consideration. The discussion below therefore refers specifically to pertussis.

Several additional factors make it difficult to apply findings in animal models of infection to their human counterparts, and these difficulties hold for the study of pertussis. These factors are either specific characteristics of infection and response in a particular species or more general considerations in judging the relevance to humans of studies in animals.

First, the capacity of any particular organism to infect (i.e., to colonize and replicate in) a host varies a great deal across host species. This variability depends in turn on several factors, including how closely the organism's surface antigens resemble those of the host. If the microbe's prominent antigens are like those of the host, rapid replication might occur before the host would recognize the invader as foreign. Some organisms initiate their relationship with the host by binding to specific host receptors or antigens; these binding sites vary among species. An animal's immunologic repertoire, consisting of antibodies and programmed lymphocytes, is generated by natural exposure to antigens. Across species this experience with antigens varies with diet and habitat, particularly proximity to humans or other animals.

Second, disease is the product of a stimulus (such as a bacterium and its toxins) and the host response. The inflammatory response is similar but far from identical across mammalian species, and the disease can be expressed differently in different species. For example, different animals vary in their capacity to inactivate bacterial endotoxin and in their susceptibility to allergic anaphylaxis.

Third, results in animal models of infectious diseases can vary greatly with the conditions of the initial exposure of the host to the organism. If animal models of infection involve the same point of entry, the same vehicle (e.g., dust or aerosolized droplets), and similar numbers of organisms in order to produce natural infection, they are more likely than less analogous models to predict the results that will be obtained in humans.

Fourth, the material used for studies in animals (in this case, the organism and its products) should, ideally, be the same as that to which the human is exposed. Attention must be paid to the bacterial strain and to possible differences in metabolism and virulence within the strain.

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 41

Fifth, host defense mechanisms mature at markedly different rates in different species. The stage of immunologic development in the test animal needs to be understood in relation to development of the immune system in the human.

Results with animal models can only suggest possible relations or outcomes in humans. Observations made in animal models represent only an initial step in the process of applying animal experience to human disease and its prevention. Findings from the animal model must be confirmed by the study of humans, and that principle is clearly relevant to the study of either whooping cough or the adverse events that follow pertussis vaccination. With respect to the study of adverse events following exposure to rubella or rubella vaccine, such questions of relevance are moot, since it has not been possible to develop an animal model for rubella infection.

These general requirements for the applicability of data from animal models to human conditions limit the usefulness of the information currently available regarding pertussis vaccine. No pertinent information is available from animal models regarding rubella vaccine. Observational studies in humans have been a more useful basis for making judgments about the possibility of causation of adverse events by pertussis and rubella vaccines. (See Appendix C for further discussion of the animal models used to study pertussis and pertussis vaccine.)

Controlled Epidemiologic Studies (Observational)

In contrast to randomized controlled trials and other experimental studies in humans, many epidemiologic investigations are observational. This means simply that the occurrences of exposures or events of concern, such as pertussis vaccination or a particular adverse event, are studied as they arise in the usual course of life and not under the conditions of a planned experiment.

Observational studies in populations are often controlled, however, through various strategies of formal comparative investigation. For example, the experience of adverse events in a group after receiving pertussis vaccine can be compared with that in an unvaccinated group (unexposed control group). Alternatively, the prior vaccination history of a group that has developed irreversible encephalopathy can be compared with that of a group free of this condition (unaffected control group). In these two strategies, the experience of the control or comparison group provides an estimate of the frequency either of events in the absence of exposure or of exposure in the absence of the event, as experienced in the general population. Thus, the contribution of the control group in such studies is analogous to that of the placebo group in a controlled trial.

The most relevant types of such controlled, observational studies for the

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 42

present review and their main characteristics are described in this section. Examples of studies related to pertussis or rubella vaccine and adverse events serve for illustration.

Cohort Studies

Cohort studies track groups that are defined by common characteristics, including their exposure status, for example, vaccinated and unvaccinated, at the starting point of observation. The rates of occurrence of adverse events are compared between these groups over time. All study participants are known or presumed to be free of the disease or events under investigation at the start of the study. In the well-designed cohort study, reliable estimates of event rates in each group can be obtained.  Especially for uncommon events, large populations, prolonged periods of observation, or both are required (Last, 1988).

Such studies can provide evidence that bears on both the first and second types of causal questions discussed earlier in this chapter. By direct comparison of rates in exposed versus nonexposed groups, a measure of association, termed the relative risk, is derived. From the same results, the attributable risk can be determined, and with knowledge of the frequency of exposure in the general population, the population attributable risk can also be calculated. Thus, the questions of whether exposure is associated with the event and, if so, to what degree can both be answered with the results of the cohort study.

The starting point of the investigation can be either contemporaneous or in the past. In the first case, termed concurrent cohort studies, all observations, including both exposures and events, may be subject to direct observation by the investigator. In the second case, which typically depends on the availability of records of past exposures and events, the entire study may relate to experience prior to the start of the investigation. Such studies are termed historical cohort studies. Some features are common to both types of cohort studies, and others are distinct in accordance with their different temporal strategies.

For example, in a historical cohort study, Griffin and colleagues (1990) evaluated the risks of seizures and encephalopathy in a cohort of 38,171 children in Tennessee on Medicaid who had collectively received a standard schedule of 107,154 DPT immunizations in the first 3 years of life (see Chapter 4 for details). The use of historical records permitted a more rapid and less expensive investigation than would have been afforded by a concurrent cohort design.

One potential weakness of the historical cohort study design, the dependence on possibly incomplete and unreliable historical records for information, was reduced in the study of Griffin and colleagues because of the

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 43

availability of multiple sources of high-quality information. Still, dependence in that study on hospital-based medical encounters to identify cases of seizures may have meant that the rates in both groups were underestimated if, in some cases, no medical encounter or only an outpatient contact occurred. The effect of missed cases would be to reduce the sensitivity (power) of the study (see more detailed discussion below). If, in addition, the likelihood of missed events had, for any reason, been disproportionate between the vaccinated and unvaccinated groups, there would be a corresponding bias in the comparison of rates between groups. A further concern in such studies of pertussis or rubella vaccination is the very high proportion of the population that is exposed. This circumstance requires consideration of factors that might affect selection for nonvaccination status and their possible role as confounders between vaccination status and the occurrence of health outcomes, including the adverse events under study.

The attempt to investigate encephalopathy, as well as seizures, in the study of Griffin and colleagues illustrates another feature of cohort studies generally: the fact that especially rare conditions may not be detectable within the limits of sample size and duration which characterize many such studies. No cases of encephalopathy were identified within 2 weeks of immunization, the period judged to be relevant to the question of a causal association. Because the outcomes in question are generally rare, the case-comparison design has more often been used in the investigation of the rare adverse events considered in this report.

Case-Comparison Studies

Controlled epidemiologic studies in which the subjects are selected in accordance with their disease status, for example, with or without encephalopathy, and investigated to determine their prior histories of exposure, for example, whether or not they had received pertussis vaccine, are termed case-comparison or case-control studies. Unlike cohort studies, case-comparison studies do not provide direct estimates of adverse event rates in the study groups. This is because the groups are defined by the presence or absence of events, and the cases are not ordinarily drawn from a defined population in a way that permits the calculation of event rates.

Instead, the result of the case-comparison study is expressed as the ratio of the odds of having been exposed as a member of the case group versus the odds of having been exposed as a member of the comparison group. As such, the odds ratio is a measure of association that can contribute to answering the first type of causal question, whether exposure can cause the adverse event under consideration. If there is no association between exposure status and the adverse event, the expected odds ratio is 1. If, on the other hand, the risk of the adverse event is higher in the exposed group,

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 44

even though the risk cannot be directly observed, the expected odds ratio is greater than 1. Because this strategy of investigation begins with the identification of cases, such as those identified from existing hospital or other medical records, it is not dependent, as is the cohort study, on the gradual accumulation of sufficient numbers of events for analysis. Therefore, results can often be obtained in much less time and at a lower cost than they can with the cohort approach.

The National Childhood Encephalopathy Study (NCES) of Great Britain is an example of the case-comparison design applied to the study of rare neurologic events following pertussis vaccination (see Chapter 4 for further details). In that investigation, physicians were asked to notify study personnel of all cases of serious neurologic illnesses in infants and children ages 2 to 36 months. Over a 3-year period, 1,182 cases were identified. Two control children were selected for each case from immunization or birth registers and were matched to the case by sex, age (within 1 month), and residential area. Information on immunization history for both cases and controls was obtained from the children's medical records, which were kept by the local health authority or family doctor.

Immunization histories of cases and controls were then compared at a series of intervals prior to various defined reference dates. For cases, two reference dates were used: the date of hospital admission and the date of onset of the relevant neurologic illness. The reference date for controls was the date on which the control child was exactly the same age as the corresponding index child on either the day of admission or the estimated day of disease onset.

A key concern in interpreting the results of case-comparison studies such as the NCES is the potential for bias in the selection of cases and controls. Evaluation of an association depends on having valid estimates of the exposure frequencies in both the case and the comparison groups. An atypical case or comparison group may seriously bias the odds ratio. In the NCES, for example, investigators debated whether to choose controls from patients admitted to the same hospital as the cases (hospital controls) or from registers of children in the same neighborhood as the cases (neighborhood controls). The likelihood of being admitted to a hospital when ill can vary not only with the nature and severity of the illness but also with many social and cultural factors. With respect to the NCES, it was thought that control children admitted to hospitals in the study area would be more likely both to come from homes of lower socioeconomic status and to have had a recent illness than would controls chosen as a random sample of the population. Since both of these factors would tend to reduce the likelihood of recent immunization, selection of hospital controls would tend to lead to an artificially low frequency of exposure in the controls and a correspondingly high odds ratio. The NCES investigators decided on neighborhood controls in-

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 45

stead, recognizing that such controls, chosen as they were from birth registers or health visitors' lists, would favor the inclusion of less mobile families, which might carry an opposite bias (Alderslade et al., 1981). Some case-comparison studies include both neighborhood and hospital controls to permit estimation of such potential selection biases, but this was not done in the case of NCES for reasons of cost. Such choices and their possible effects make it difficult to ensure the comparability of cases and controls; the desired comparability is that which is achieved in principle by randomization at the outset of a randomized controlled trial. Merely increasing the size of a study will not avoid the bias which is implied by insufficient comparability between groups.

Another potential source of bias in many case-comparison studies relates to the information about exposure collected from cases and controls. Under the circumstances of many case-comparison studies, there is some prior knowledge about the problem under study on the part of study subjects as well as data collectors, such as interviewers or record abstractors. There may also be suspicion on the part of respondents about factors thought to explain the occurrence of the event. These circumstances lead to potential for bias, especially if information about exposure is not pursued in comparable ways between the case and comparison groups. Thus, ''information bias" is a concern in case-comparison studies that does not arise in the same way in cohort studies, in which exposure is documented in advance of the adverse event.

A number of other sources of bias potentially enter into case-comparison studies, cohort studies, or both. Some apply to experimental studies also. Variously defined and described in standard texts (e.g., Kleinbaum et al., 1982; Mausner and Kramer, 1985; Rothman, 1986), they all require consideration before the results of any study are accepted as providing valid evidence. Similarly, potential confounding is considered in the process of evaluating each study.

Other Controlled Studies

The design of both cohort and case-comparison studies can vary in many respects, and still other types of controlled or comparative studies may be carried out, such as comparative cross-sectional surveys or studies of long-term trends in the frequencies of adverse events of interest, perhaps linked with trends in the frequency of vaccination. In some instances, groups of cases are compared in a manner distinct from the case-comparison approach described above. For example, Melchior (1977) compared the distributions of ages of onset of infantile spasms for two calendar periods during which different immunization schedules were used (see Chapter 4 for additional details). In that study, both the pertussis component of the vaccine and its

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 46

adjuvant varied between the two time periods examined. Thus, in such further variations of controlled observational approaches, there are also issues of interpretation to be addressed, here concerning the comparability of the exposures and methods of case ascertainment among persons immunized in the two time periods being studied. In addition, methods of case identification were noted to have varied. The potential effects of these changes therefore had to be considered. Another example is the use of data concerning time intervals between pertussis immunization and the occurrence of sudden infant death syndrome (SIDS) (see Chapter 5 and Appendix D). The analysis included estimation of the expected frequencies of SIDS in specific intervals determined from background rates in the general population. 

Case Reports and Case Series

Observations of single or multiple human cases of a disease, without formal comparison with a reference group, are termed case reports and case series. Such reports appear in several types of sources, including peer-reviewed literature; passive surveillance reports of organizations such as the Food and Drug Administration, the CDC, or vaccine manufacturers; and informal reports from physicians, parents, or others. In the present context, such reports generally describe clinical situations in which a child has received a vaccine and subsequently manifested one or more clinical events.

Case reports and case series constitute a substantial proportion of the total number of reports on possible adverse consequences of pertussis immunization. Following Madsen's first report of serious adverse events following immunization (Madsen, 1933), numerous cases have been reported in the peer-reviewed literature (e.g., Berg, 1958; Byers and Moll, 1948; Corsellis et al., 1983; Martin and Weintraub, 1973; Tingle et al., 1979, 1985) and elsewhere (e.g., Centers for Disease Control, 1984, 1986, 1989; Coulter and Fisher, 1985; M. Grant, Determined Parents to Stop Hurting Our Tots, case reports, personal communication to the committee, 1989). To date, such noncontrolled clinical studies provide the only source of information for 7 of the 20 adverse events under review by the committee: in relation to pertussis vaccine, erythema multiforme and other rash, GuillainBarrè syndrome, peripheral mononeuropathy, hemolytic anemia, and thrombocytopenia; and in relation to rubella vaccines, radiculoneuritis and other neuropathies, and thrombocytopenic purpura. 

Searching for Patterns

One application of the case reports and case series available concerning a particular adverse event is to seek clinical or epidemiologic patterns within the

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 47

set of collected cases. Such an analysis would proceed by coding case information from each of these reports and searching systematically for distinct clusters that might turn out to be causally related. The committee attempted to carry out such a strategy but was unsuccessful for several reasons.

First, cases reported in the medical literature and to surveillance systems are not a random sample of all adverse events that are potentially vaccine-related. In particular, cases that match prevailing clinical and popular conceptions about what causes the adverse event and the pattern of associated signs and symptoms are more likely to be reported than others. As a result, the data available for analysis will be dominated by a pattern that matches the prevailing conceptions, and any statistical search for patterns will tend to detect this same pattern.

Second, for most of the adverse events under consideration, the cases potentially caused by a vaccine constitute a small proportion of all cases of the adverse event. Thus, even if there is a distinct cluster of vaccine-caused cases, they will represent a small part of the statistical sample available for analysis. This makes their detection difficult.

More immediate limitations lie in the nature of the information often available concerning the reported cases. Most cases presented in the literature are severely underdocumented. The less the information reported about each case, the less the chance of finding constellations of features with possible causal significance. Many reports in the literature provide only the barest summary information about each case, with little indication of how the data were obtained that would permit an independent judgment of their reliability. In addition, frequently the data reported are simply not comparable from one report to another. Diagnostic criteria or ascertainment techniques for such data as time of vaccination and onset of the adverse event can differ so much that no meaningful comparison can be made between reports.

In view of such difficulties, the committee used information abstracted from case reports for the limited purpose of testing propositions found in the literature that particular constellations of features exist for events that are caused by vaccine. The information obtained is shown in the Case Report Review Form (Appendix A). Data included the source of the report, characteristics of the exposure and the subject, clinical features of the subject, clinical characteristics of the adverse event, and permanent sequelae of the event, if any. Despite the more than 100 cases abstracted, the committee was not able to identify a specific and consistent syndrome of vaccine-induced encephalopathy, the condition for which this question was considered most worth investigating in this manner.

In special circumstances, for example, in the absence of a controlled study or other epidemiologic data and in the presence of supportive clinical evidence, the committee considered that specific reports of an adverse event following vaccination might suggest a biologically plausible relation. This

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 48

was true, as noted above, for the adverse events anaphylaxis and erythema multiforme or other rash. In the case of hemolytic anemia, a single striking case report was sufficient to suggest biologic relevance (see Chapter 6).

Historical Comparisons

Another aspect of case reports and case series that bears on the use of such evidence is their lack of external comparative observations. Accordingly, the measures of association described earlier in this chapter that provide evidence on the question of causation in general and, if affirmative, the extent of such causation are not attainable from these reports. In unusual circumstances, a case series may reflect the ascertainment of all cases from a defined population over a given period of time. It may therefore be possible to estimate rates of events for that population, and if such information is available for two distinct periods or over a very extended interval, historical comparisons or trends might be constructed. Such opportunities were not found, however, in the material available concerning pertussis or rubella vaccine and adverse events. Were it available, evidence of this kind would be applicable to the question of how much vaccine exposure might contribute to the occurrence of the event in the population. 

Causation of the Individual Case

Case reports and case series data also bear on the question of causation of an individual case, for example, whether a given adverse event is an allergic response to pertussis vaccine. Individual cases of the adverse event could be examined to determine whether the event occurred in a clear sequence following each pertussis immunization in the series. An increasing severity of the event with increasing dose number would tend to support a causal interpretation. If the event tended to diminish in severity or was absent for later doses in the series, this evidence would tend to detract from a causal interpretation.

Although the temporal relation between the exposure and the event can at times seem to provide compelling evidence toward a causal interpretation, the possibility of simple coincidence, in light of the known background occurrence of most of the events under consideration, adds to the importance of careful evaluation of alternative explanations. Rarely, on the basis of case information alone, can alternative explanations be excluded.

Monitoring System for Adverse Events Following Immunization

Because of its prominence among sources of case reports, special note should be made of the CDC's Monitoring System for Adverse Events Fol-

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 49

lowing Immunization (MSAEFI). In this system (now replaced by the Vaccine Adverse Event Reporting System or VAERS; see below), data were collected on illnesses temporally associated with the administration of publicly purchased vaccines. Unlike "active" disease surveillance systems that use aggressive methods to identify all new or existing cases of disease (e.g., local cancer registries that actively review, on an ongoing basis, all discharge summaries at all hospitals within their catchment areas), MSAEFI calculated its incidence rates of adverse events following vaccination on the basis of voluntary reports from physicians, allied health professionals, and others. Being dependent on voluntary reports, MSAEFI is a "passive" surveillance system. Cases reported to MSAEFI between 1978 and 1990 were enumerated for each of the adverse events considered in this report (J. Mullen, Centers for Disease Control, personal communication, 1991).

The inherent limitations of MSAEFI must be kept in mind when interpreting these data. First, despite efforts to encourage physicians and allied health professionals to submit reports, MSAEFI is likely to have represented only partial ascertainment of reportable cases. This is because reporting depended on retention and reading of reporting guidelines, the presence of a suspicion of a possible connection between an event and immunization, and a decision by the parents or guardians to make a report. It is reasonable to expect that reporting of a given adverse event was therefore influenced by a number of factors, including the certainty of its clinical presentation or its proximity in time to vaccination. Second, clinical information was often obtained from family members and collected by nonmedical personnel. The information was sometimes clinically nonspecific, and although the MSAEFI reporting form was standardized, it may have been completed incorrectly. Third, all reports submitted to the CDC for illnesses with onset within 28 days of receipt of a vaccine and requiring a visit to a medical care provider were included in the MSAEFI data base, regardless of whether these were confirmed or whether alternative explanations of the illness were known. Neither of these factors was routinely recorded.

Changes effected in January 1985 alleviated some of these limitations, although MSAEFI was still not able to provide information about all illnesses following immunization. The system did generate useful data on secular trends of selected illnesses and indices of rates of illnesses and assisted in identifying risk factors that may predispose an individual to adverse events following immunization (Centers for Disease Control, 1984, 1986, 1989).

In November 1990, a new federal system replacing MSAEFI, VAERS, went into effect for the monitoring of adverse events after vaccination (Centers for Disease Control, 1990). Reports of adverse events from a variety of sources (e.g., health care providers, parents, and manufacturers) are now received by a single office. VAERS is operated by a private contractor with

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 50

the support of the Public Health Service. VAERS data relevant to the committee's task were not available by the time of completion of this report.

ASSESSMENT OF THE EVIDENCE

Evaluation of Single Studies

The foregoing discussion of categories of evidence has indicated many features of experiments in humans and animals, controlled and uncontrolled observational studies, and comparative case series which are considered in the evaluation of single reports of such studies. It is important to emphasize the evaluation of the single study, because synthesis of an identified body of evidence begins with this step.

A systematic evaluation of a single study will take account of the hypothesis, the design, the methods of conduct and analysis,3and both the author's and the reviewer's conclusions and interpretations. This information was part of that abstracted from single studies as the first step in the committee's evaluation. (For details concerning the process of abstraction, the information obtained, and the Controlled Study Review Form, see Appendix A.)

In its review of single studies, the committee recognized that the completeness and reliability of the data, the appropriateness of analysis, and the exclusion of alternative interpretations or explanations of the results all require evaluation. The potential roles of bias and confounding need to be taken into account in studies in which measures of association are produced. Assessing the role of chance requires detailed information about the design and conduct of the study, but this can sometimes be accomplished even if the necessary calculations were not presented in the original report. (It is beyond the scope of this assessment to discuss the details of these aspects of assessment; standard textbooks of epidemiology [e.g., Kleinbaum et al., 1982; Mausner and Kramer, 1985; Rothman, 1986] may be consulted for further information.)

The validity of a study, which is the focus of this initial assessment, is rarely an absolute judgment but differs by degree for different aspects of the study. Strengths and limitations of individual studies, in various re

3 A special note concerning the reporting of incidence rates of adverse events following pertussis immunization is merited. Incidence rates are usually cited either as rates per individual dose or rates per child (generally three doses). When available in a given study, the committee reported both rates, although the latter was thought to be preferable, since the committee was concerned with risk to the individual and since risk often varies across doses (see the example of shock and "unusual shock-like state" in Chapter 6).

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 51

spects, are to be anticipated. Weighing of these in a quantitative sense is, to varying degrees, possible, especially in the context of certain (but not all) methods of meta-analysis (see section on meta-analysis below). Usually, a subjective sense of the merit of the results of a given study carries over from the initial evaluation to the stage of integrating results from the whole body of available evidence.

Integration of the Collective Results

Inferences concerning causality are commonly made on the basis of epidemiologic and related biomedical evidence. Many policy decisions and practical actions are based on such inferences made by persons with widely varied backgrounds—professional and nonprofessional, technical and nontechnical. The process of reaching such conclusions is ordinarily personal and often is private. By contrast, when this process is conducted formally in the manner of the present review, it is collective and interactive. As indicated earlier in this chapter, it is also desirable that reasoning be made explicit, in order that others may be enabled to evaluate the committee's conclusions independently.

Two aspects of the integration of evidence used by the committee are explained here. First is the quantitative approach of meta-analysis whose principles and methods are discussed briefly below. In the four instances in which it was deemed appropriate, this approach was applied and the results are presented in the corresponding section below. For the remainder of the types of adverse events for which meta-analyses were not undertaken, the committee's rationale for not doing so is provided.

The second aspect is the partially quantitative, but largely qualitative, process sometimes termed causal inference, for which a general approach has long been recognized in the epidemiologic literature. This will also be discussed below, to indicate the committee's view of that approach and its application to the present evaluation.

Meta-Analysis

A review of the major studies on which this report is based suggests that the sample sizes of many studies are insufficient to detect clinically important excess risks. As will be seen in Chapter 5, for instance, a number of the studies on the timing of SIDS deaths relative to DPT immunization (studies that offer the least biased information on the potential impact of the vaccine on SIDS) describe so few cases that large relative risks cannot be ruled out.

When a number of sufficiently similar studies of the same adverse event are available, it is sometimes possible to pool statistical information from

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 52

the studies to develop an estimate of the relative risk, or the odds ratio, of the event in question that is more precise than the estimates from the individual studies. After considering whether meta-analysis could be used to estimate the risk of a number of adverse events following pertussis or rubella vaccination, the committee decided that data adequate to justify a meta-analysis were available for four adverse events following pertussis immunization: febrile seizures, afebrile seizures, SIDS, and hypotonic, hyporesponsive episodes. For instance, seven studies with information on the timing of SIDS had sufficiently large sample sizes to be included in the calculation. Four of these studies had more satisfactory study designs because they included control groups. Because it was not clear that the studies were sufficiently homogeneous to be compared, the committee analyzed all seven studies together and then studied separately the four studies with better control groups. The details of these analyses are given in Chapters 4 and 5 and in Appendix D.

The committee was not able to use meta-analysis data for other adverse events, primarily because a sufficient number of comparable studies were not available. For most of the adverse events under consideration in this report, there exists, at most, one controlled epidemiologic study. There were four controlled epidemiologic studies of encephalopathy reviewed (see Chapter 4, Table 4-4), but these are dominated in size by a single study, the NCES. As described in Chapter 4, the NCES includes 389 children with encephalopathy, whereas the other available studies include a total of 29 cases of encephalopathy. Any pooled estimate based on these studies would clearly be dominated by the NCES results, so the committee judged that meta-analysis would not yield useful information.

Considerations in Inferring Causality

For each adverse event for which the evidence indicated the presence of an association with vaccine exposure, the committee assessed the applicability of each of six general considerations, patterned after those attributed to Hill (1971). Reflecting thought about the problem of causal inferences in chronic disease epidemiology that had evolved over several years, Hill proposed the following guidelines for judgment: strength of association, dose-response relations, temporally correct association, consistency of association, specificity of association, and biologic plausibility. It should be noted that Hill's formulation has often been applied to the question of what considerations are supportive of a causal interpretation of an association. The committee was charged with judging whether a causal interpretation concerning a particular adverse event was either supported or not supported by the evidence. Accordingly, these considerations were applied, where possible, to aid interpretation in both directions.

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 53

Three of these considerations (strength of association, dose-response relation, and temporally correct association) can be applied to the findings of single studies and can therefore be regarded, in part, as measures of internal validity. Any of these considerations can be satisfied in some, but not necessarily in all, studies testing a particular causal hypothesis. The other three considerations (consistency of association, specificity of association, and biologic plausibility) are not necessarily study specific and depend to varying degrees on prior knowledge.

Strength of Association Strength of association is usually expressed in epidemiologic studies as the magnitude of the measure of effect, for example, relative risk or odds ratio. Generally, the larger the relative risk, the greater the likelihood that the vaccine-event association is causal or, in other words, the less likely it is due to undetected error, bias, or confounding. Measures of statistical significance such as p values are not indicators of the strength of association.

Dose-Response Relation The existence of a dose-response relation—that is, an increased strength of association with increased exposures or other appropriate relation—strengthens an inference that an association is causal.

Temporally Correct Association If an observed association is causal, exposure must precede the event by at least the duration of disease induction. The committee, in addition, considered whether the adverse event occurred within a time interval following vaccination that was consistent with current understanding of its natural history. The committee interpreted the lack of an appropriate time sequence as strong evidence against causation, but recognized that insufficient knowledge about the natural history and pathogenesis of many of the adverse events under review limited the utility of this consideration.

Consistency of Association Consistency of association requires that an association be found regularly in a variety of studies, for example, in more than one study population and with different study methods. The committee considered findings consistent across different categories of studies as being supportive of a causal interpretation of the evidence.

Specificity of Association Specificity of association is the degree to which a given exposure predicts the frequency or magnitude of a particular outcome; if the association of the exposure and the event is unique to both, a causal interpretation seems more strongly justified than when the association is nonspecific to both the exposure and the event. The committee

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 54

recognized, however, that perfect specificity could not be expected given the multifactorial etiology of many of the adverse events under examination.

Biologic Plausibility  Biologic plausibility is based on whether a possible causal association fits existing biologic or medical knowledge. The existence of a possible mechanism, such as an established association of the adverse event with natural disease (e.g., thrombocytopenic purpura following natural rubella infection), was thought to increase the likelihood that the vaccine-event association could be causal.

Other Considerations  As noted above, it is important also to consider whether alternative explanations—error, bias, confounding, or chance—might account for the finding of an association. If an association could be sufficiently explained by one or more of these considerations, there would be no need to invoke the several considerations listed above. From this viewpoint, an inference of causation could be based solely on the exclusion of these alternatives.  Because these alternative explanations can rarely be excluded sufficiently, however, assessment of the applicable considerations listed above almost invariably remains. The final judgment is then a balance between the strength of support for the causal interpretation and the degree of exclusion of alternatives.

Other considerations were also entertained in the evaluation of the evidence of association. One special consideration in evaluating summary evidence on the relation of adverse events to pertussis or rubella immunization was that of the variation in vaccine composition observed across manufacturers.

With respect to the whole-cell pertussis vaccine, for example, the committee recognized that methods of production, seed bacteria, preservatives, and adjuvants used in manufacturing the vaccine have varied substantially over the years (Cox et al., 1987; Ross, 1988) and vary even now by manufacturer and country. In some countries, such as the United States, adsorbed vaccines are used exclusively. In others, both plain (fluid) and adsorbed vaccines are available, and in some, only plain vaccines are available. In most countries, pertussis immunization is given in conjunction with diphtheria and tetanus toxoids in a combined DPT product. In countries where inactivated polio vaccine is used, a quadruple antigen (diphtheria, tetanus, pertussis, and polio) vaccine is used. In yet other countries, pertussis vaccine is given primarily as a monovalent vaccine.

The committee also recognized that pertussis immunization schedules have varied markedly by country and time period. In the United States, for example, DPT vaccine is currently administered in five doses at ages 2, 4, 6, and 18 months and 4 to 6 years (American Academy of Pediatrics, 1988).

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 55

In The Netherlands, DPT-polio vaccine is given in four doses at ages 3, 4, 5, and 11 to 14 months (Health Council of The Netherlands, 1987). Rubella-containing vaccines, like pertussis vaccine, have also varied considerably across place and time. For example, three vaccine strains were used initially in the United States following licensure in 1969-1970: HPV-77 (duck embryo), HPV-77 (dog kidney), and Cendehill (rabbit kidney) (Hilleman et al., 1969; Meyer et al., 1969; Prinzie et al., 1969). Soon after, the RA 27/3 human diploid fibroblast vaccine was licensed in Europe (Plotkin et al., 1969), and both the HPV-77 (dog kidney) and the Cendehill vaccines were subsequently withdrawn from U.S. licensure. In 1979, RA 27/3 was licensed in the United States, and the HPV-77 (duck embryo) vaccine was withdrawn, leaving RA 27/3 as the only U.S.-licensed rubella vaccine (Perkins, 1985).

In addition to strain variations, rubella-containing vaccines also vary in composition. In the United States, and increasingly elsewhere, rubella vaccines are combined in a triple vaccine also containing measles and mumps vaccine viruses (MMR). However, bivalent vaccines containing rubella and measles or rubella and mumps vaccines are also used.

What, then, are the implications of these variations in vaccine composition and schedules for the evaluation of vaccine-adverse event associations? With respect to rubella vaccines, rates of arthritis and arthralgia following immunization were found to differ by strain (see Chapter 7). Integration of evidence across studies was therefore problematic. This issue was considered moot with respect to radiculoneuritis and other neuropathies and thrombocytopenic purpura, since the evidence for these adverse events was limited to isolated case reports.

Unlike rubella vaccines, however, the committee considered the potential problem of variability in whole-cell pertussis vaccine composition to be much greater. For example, in the last 10 years of testing of pertussis vaccines at the National Institute for Biological Standards and Control (NIBSC) in the United Kingdom, whole-cell vaccines have shown wide variation in biologic activity from batch to batch, although with no significant time-related trends (K. Redhead, National Institute for Biological Standards and Control, personal communication, 1990). Testing did reveal that plain pertussis vaccines are usually more active than adsorbed vaccines, a finding consistent with studies cited in the report (e.g., Pollock et al., 1984) which indicate that the frequency of moderate systemic reactions following primary immunization with adsorbed DPT vaccine is similar to that with adsorbed DT vaccine, but considerably less than that with plain DPT vaccine.

In summary, the general approach to evaluation outlined in this chapter was applied to each type of adverse event as dictated by the nature of the available evidence.

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 56

THE NATURE OF THE CONCLUSIONS

This chapter has demonstrated that judgments about the possible causation of adverse events by pertussis or rubella vaccine and similar causal questions reflect both quantitative and qualitative reasoning. Some final observations will help to clarify the nature of the committee's conclusions.

Quantitation
Resolution

Resolution refers to the fineness or sharpness of detail that can be discriminated by a particular mode of observation. In light microscopy, for example, observations are described by reference to the optical properties of the lens, such as 10x, 100x, or higher magnification. Electron microscopy, with very much higher resolution, distinguishes structural features not detectable with light microscopy.

Resolution in epidemiologic studies concerns the capacity of a study to discriminate between the frequencies of events or of exposures between groups in order to determine the presence or absence of associations. By analogy, resolution in epidemiology also depends in a sense on magnification, that is, on the order of magnitude of the numbers of participants—for example, from tens to hundreds of cases and controls in case-comparison studies and from hundreds to thousands of exposed and unexposed subjects in cohort studies. With equally valid observations, results based on the experience of increasing numbers of persons, from single individuals to tens, hundreds, or thousands of individuals, provide successively greater resolution.

The resolution or discriminating capacity of epidemiologic studies could theoretically be increased indefinitely through ever larger study populations. However, there are many constraints on the feasibility of large studies. Rarity of exposures or events or other circumstances may limit the resolution even of large studies. Meta-analysis can, under the appropriate circumstances discussed above, be used to offset the limited size of individual studies, but the collective magnitude of the contributing studies may still be less than desired. It should be emphasized that in all such studies the potential for bias is a key problem and that enlarging the study only reduces random error, not systematic error. Therefore, if bias is present, a firmer but still erroneous conclusion will result from a larger study than from a smaller one.

Power calculations indicate the probability of achieving discrimination of a predetermined degree under the design of a given study. Power is thus a quantitative measure of the capacity of a study to achieve a given degree

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 57

of resolution. In particular, it provides guidance against overconfidence in the absence of an association when a study with relatively low power has failed to demonstrate one. As such, it is an aid in appreciating the nature of the evidence that underlies conclusions about causation and is described further here to indicate its role in the present review.

As discussed earlier in this chapter, two types of error must be taken into account in designing and interpreting statistical tests. Epidemiologic studies are often designed to provide statistical tests that minimize type I error, the probability that the null hypothesis is falsely rejected. Commonly, such tests are designed so that there is less than a 5 percent chance that the test will incorrectly indicate an association between a vaccine and an adverse event if no association truly exists. For any given test and sample size, on the other hand, there is some chance that the test will err in failing to find an association when one truly exists. This is called a type II error. The chance of making such an error increases when both the true excess risk and the sample size are small. From another perspective, given a particular sample size and a specified probability of a type I error, one can calculate the power of a test to detect an assumed association of a given magnitude. Because the power of a test is the opposite (technically, the complement) of the probability of making a type II error, the power of a test increases when both the true excess risk and the sample size are large.

For example, Shields and colleagues (Melchior, 1977; Shields et al., 1988) compared the ages of onset of various disorders under different vaccination schedules in two time periods in Denmark and found no statistically significant differences between the two periods in the onset of infantile spasms. However, there were only 80 cases of infantile spasms in the two study periods combined, and the lack of a significant statistical association may therefore reflect the small sample size rather than a true absence of association. In other words, if this investigation were replicated in a country with more cases of infantile spasms than occurred in Denmark, a statistically significant difference might be detected, if the relation was, in truth, causal.

Furthermore, according to the power calculations described in Appendix D, if 50 percent of cases of infantile spasms were caused by pertussis vaccine, there would be nearly a 90 percent chance that a sample of the size used by Shields and colleagues would detect this relationship. If, on the other hand, only 25 percent of the cases of infantile spasms were caused by pertussis vaccine, Shields and colleagues' test has about a 45 percent chance of detecting the relationship.

Power calculations are also valuable in interpreting apparently conflicting results of multiple studies of the same vaccine-adverse event combination. If findings of no association were concentrated in the low-power studies, for instance, the suggestion that no association exists would be weakened.

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 58

Because power calculations help to illuminate an important aspect of the uncertainty in the evidence it evaluated, the committee decided to calculate, whenever possible, the power of the statistical tests on which its conclusions were based. These calculations are described in Appendix D.

Uncertainty and Confidence

All science, including the spectrum from particle physics to astrophysics, is characterized by uncertainty. Scientific conclusions concerning the result of a particular analysis or set of analyses can range from highly uncertain to highly confident. As discussed earlier in this chapter, the theoretical concept of proof does not apply in evaluating actual observations. In its review, the committee attempted to assess the degree of uncertainty associated with the results on which it had to base its conclusions.

For individual studies, confidence intervals around estimated results such as relative risks represent a quantitative measure of uncertainty. Confidence intervals present a range of results that, with a predetermined level of probability, include the true relative risk being estimated. When it is possible to use meta-analysis to combine the results of different studies, a combined estimate of the relative risk and confidence interval may be obtained. Appendix D describes the methods used for meta-analysis in the report.

For an overall judgment about causation based on a whole body of evidence, beyond the results of single studies or of meta-analyses, no quantitative method exists for characterizing the uncertainty of the conclusions. Thus, to assess the appropriate level of confidence to be placed on the ultimate causal conclusions, it may be useful to consider qualitative as well as quantitative aspects.

Quality
Comprehensiveness

An important aspect of the quality of a review such as the present one is comprehensiveness. This is to ensure against the possibility of any serious omission or inappropriate exclusion of evidence from consideration. If any such omission should be identified, a determination would be needed of whether its inclusion would likely affect the overall results and, if so, in what way.

In this report the committee has documented in detail its approach to seeking and identifying the evidence to be reviewed (see Appendix A, Strategies for Gathering Information). Numerous parties were invited to supplement the materials already under review and to notify the committee of any recognized omissions of importance.

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 59

Neutrality

Neutrality is another important consideration in the quality of such conclusions as those presented by the committee. This is to ensure a fair weighing of all of the evidence. In this connection, the committee avoided the posture of the burden of proof approach, as discussed earlier in this chapter. The essential evidence, its main strengths and limitations, and the conclusions that follow are stated for each adverse event considered.

Judgment

The evaluation of evidence to reach conclusions about causation goes beyond quantitative procedures, at several stages: assessing the relevance and validity of individual reports; deciding on the possible influence of error, bias, or confounding on the reported results; integrating the overall evidence, within and across diverse areas of research; and formulating the conclusions themselves. These aspects of the review required thoughtful consideration of alternative approaches at several points. They could not be accomplished by adherence to a prescribed formula.

Rather, the approach described here evolved throughout the process of the review and was determined in important respects by the nature of the evidence, exposures, and events at issue.  Both the quantitative and the qualitative aspects of the process that could be made explicit were important to the overall review. Ultimately, the conclusions expressed in this report about causation are based on the committee's collective judgment. The committee endeavored to express its judgments as clearly and precisely as the available data allowed.

SUMMARY OF THE EVIDENCE

Table 3-1 summarizes the types of evidence reviewed for each adverse event and the respective contribution of each to the committee's judgments about causation. The evidence is organized under five headings: (1) human experiments; (2) animal experiments; (3) case-comparison, cohort, and other controlled studies, (4) case reports and case series; and (5) biologic plausibility. The first four categories were discussed earlier in this chapter. The fifth category, biologic plausibility, includes background knowledge concerning the pathophysiology of an adverse event, attributes of a particular vaccine, or other biologic information derived from research in such areas as immunology and physiology.

Where evidence was available in a particular category, the committee judged whether that evidence was generally supportive or not supportive of causation or whether it was insufficient for a determination. For example,

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 60

TABLE 3-1 Categories of Evidence Reviewed for Each Adverse Event: Is the Evidence Supportive of Causation?a 

Vaccine and Adverse Event

Human Experiments

Animal Experiments

Case-Comparison,
Cohort, and Other Controlled Studies

Case Reports
and Case Series

Biologic Plausibility

(Chapter of Report)

Yesb

?c

Nod

Yes

?

No

Yes

?

No

Yes

?

No

Yes

?

No

                               

DPT

                             

Infantile spasms (4)

               

X

 

X

       

Hypsarrhythmia (4)

   

X

             

X

       

Aseptic meningitis (4)

             

X

   

X

       

Acute encephalopathye (4)

       

X

 

X

   

X

     

X

 

Chronic neurologic damage (4)

       

X

   

X

   

X

   

X

 

Sudden infant death syndrome (5)

               

X

 

X

       

Anaphylaxis (6)

       

X

   

X

 

X

   

X

   

Autism (6)

                             

Erythema multiforme or other rash (6)

                   

X

 

X

   

Guillain-Barrè syndrome (polyneuropathy) (6)

                   

X

       

Peripheral mononeuropathy (6)

                   

X

       

Hemolytic anemia (6)

                   

X

 

X

   
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 61

Vaccine and Adverse Event

Human Experiments

Animal Experiments

Case-Comparison,
Cohort, and Other Controlled Studies

Case Reports
and Case Series

Biologic Plausibility

(Chapter of Report)

Yesb

?c

Nod

Yes

?

No

Yes

?

No

Yes

?

No

Yes

?

No

Juvenile diabetes (6)

       

X

   

X

   

X

       

Learning disabilities and hyperactivity (6)

               

X

 

X

       

Protracted inconsolable crying and screaming (6)

           

X

   

X

   

X

   

Reye syndrome (6)

             

X

     

X

     

Shock and ''unusual shocklike state" (6)

             

X

         

X

 
                               

Thrombocytopenia (6

                   

X

       
                               

RA 27/3 Rubella

                             

Arthritis (7)

                             

Acute

X

         

X

   

X

   

X

   

Chronic

 

X

             

X

   

X

   

Radiculoneuritis and other neuropathies (7)

                   

X

 

X

   

Thrombocytopenic purpura (7)

                   

X

 

X

   

a Blanks for any given category of evidence indicate that evidence of this kind is lacking.

b Yes, Evidence of this kind is supportive of causation.

c ?, Evidence of this kind cannot be classified either as supportive or as not supportive of causation.

d No, Evidence of this kind is not supportive of causation.

e Defined in controlled studies reviewed as encephalopathy, encephalitis, or encephalomyelitis.

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 62

where there were relevant controlled studies which, overall, had relative risks of greater than 1, the evidence was classified as "supportive of causation." Blanks for any given category of evidence indicate that evidence of that type was lacking.  It is important to note that any one category of evidence generally was not sufficient in itself to support a conclusion of causality, since other aspects of the evidence, including the number and quality of contributing studies, the details of results obtained, and other considerations outlined earlier in this chapter all weighed into the committee's evaluation.

The committee found it convenient to classify its conclusions about each adverse event under one of five categories, reflecting the strength and direction of its conclusions. These categories are:

1. No evidence bearing on a causal relation

2. Evidence insufficient to indicate a causal relation

3. Evidence does not indicate a causal relation

4. Evidence is consistent with a causal relation

5. Evidence indicates a causal relation.

The remaining chapters elaborate on the evidence assembled as the basis of the committee's findings and conclusions.

REFERENCES

Alderslade R, Bellman MH, Rawson NSB, Ross EM, Miller DL. 1981. The National Childhood Encephalopathy Study: a report on 1000 cases of serious neurological disorders in infants and young children from the NCES research team. In: Whooping Cough: Reports from the Committee on the Safety of Medicines and the Joint Committee on Vaccination and Immunisation. Department of Health and Social Security. London: Her Majesty's Stationery Office.

American Academy of Pediatrics. 1988. The Red Book. Report of the Committee on Infectious Diseases, 21st edition. Peter G, ed. Elk Grove, IL: American Academy of Pediatrics.

Baraff LJ, Cody CL, Cherry JD. 1984. DTP-associated reactions: an analysis by injection site, prior reactions, and dose. Pediatrics 73:31-36.

Baraff LJ, Cherry JD, Cody CL, Marcy SM, Manclark CR. 1985. DTP vaccine reactions: effect of prior reactions on rate of subsequent reactions. Developments in Biological Standardization 61:423-428.

Barkin RM, Pichichero ME. 1979. Diphtheria-pertussis-tetanus vaccine: reactogenicity of commercial products. Pediatrics 63:256-260.

Barnes EK, Altman R. Austin SM, Dougherty WJ. 1972. Joint reactions in children vaccinated against rubella. Study II: comparison of three vaccines. American Journal of Epidemiology 95:59-66.

Berg JM. 1958. Neurological complications of pertussis immunization. British Medical Journal 2:24-27.

Byers RK, Moll FC. 1948. Encephalopathies following prophylactic pertussis vaccination. Pediatrics 1:437-457.

Centers for Disease Control. 1984. Adverse Events Following Immunization: Surveillance

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 63

Report No.1, 1979-1982. Atlanta: Public Health Service, U.S. Department of Health and Human Services.

Centers for Disease Control. 1986. Adverse Events Following Immunization: Surveillance Report No.2, 1982-1984. Atlanta: Public Health Service, U.S. Department of Health and Human Services.

Centers for Disease Control. 1989. Adverse Events Following Immunization: Surveillance Report No.3, 1985-1986. Atlanta: Public Health Service, U.S. Department of Health and Human Services.

Centers for Disease Control. 1990. Vaccine adverse event reporting system—United States. Morbidity and Mortality Weekly Report 39:730-733.

Cherry JD, Brunell PA, Golden GS, Karzon DT. 1988. Report of the task force on pertussis and pertussis immunization—1988. Pediatrics 81(6, part 21):939-984.

Cody CL, Baraff LJ, Cherry JD, Marcy SM, Manclark CR. 1981. Nature and rates of adverse reactions associated with DTP and DT immunizations in infants and children. Pediatrics 68:650-660.

Corsellis JAN, Janota I, Marshall AK. 1983. Immunization against whooping cough: a neuropathological review. Neuropathology and Applied Neurobiology 9:261-270.

Coulter HL, Fisher BL. 1985. DPT: A Shot in the Dark. New York: Harcourt Brace Jovanovich.

Cox NH, Morley WN, Forsythe A. 1987. Vaccine reactions and thiomersal. British Medical Journal 294:250.

Edwards KM, Lawrence E, Wright PF. 1986. Diphtheria, tetanus, and pertussis vaccine: a comparison of the immune response and adverse reactions to conventional and acellular pertussis components. American Journal of Diseases of Children 140:867-871.

Griffin MR, Ray WA, Mortimer EA, Fenichel GM, Schaffner W. 1990. Risk of seizures and encephalopathy after immunization with the diphtheria-tetanus-pertussis vaccine. Journal of the American Medical Association 263:1641-1645.

Health Council of The Netherlands. 1987. Adverse Reactions to Vaccines in the National Vaccination Programme in 1986. The Hague, The Netherlands: Gezondheidsraad.

Hill AB. 1971. Principles of Medical Statistics, 9th edition. New York: Oxford University Press.

Hilleman MR, Buynak EV, Whitman JE, Weibel RD, Stokes J. 1969. Live attenuated rubella virus vaccines: experiences with duck embryo cell preparations. American Journal of Diseases of Children 118:166-171.

Isacson P, Kehrer AF, Wilson H, Williams S. 1971. Comparative study of live, attenuated rubella virus vaccines during the immediate puerperium. Obstetrics and Gynecology 37:332-337.

Kendrick PL, Eldering G. 1939. A study in active immunization against pertussis. American Journal of Hygiene 29:133-153.

Kleinbaum DG, Kupper LL, Morgenstern H. 1982. Epidemiologic Research: Principles and Quantitative Methods. Belmont, CA: Lifetime Learning Publications.

Lapin JH. 1943. Whooping Cough. Springfield, IL: Charles C Thomas.

Last JM, ed. 1988. A Dictionary of Epidemiology, 2nd edition. New York: Oxford University Press.

Lewis K, Cherry JD, Holroyd HJ, Baker LR, Dudenhoeffer FE, Robinson RG. 1986. A doubleblind study comparing an acellular pertussis-component DTP vaccine with a whole-cell pertussis-component DTP vaccine in 18-month old children. American Journal of Diseases of Children 140:872-876.

Madsen T. 1933. Vaccination against whooping cough. Journal of the American Medical Association 101:187-188.

Martin GI, Weintraub MI. 1973. Brachial neuritis and seventh nerve palsy: a rare hazard of DPT vaccination. Clinical Pediatrics 12:506-507.

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×

Page 64

Mausner JS, Kramer S. 1985. Epidemiology: An Introductory Text, 2nd edition. Philadelphia: W.B. Saunders Co.

Medical Research Council. 1951. The prevention of whooping cough by vaccination. British Medical Journal 1:1463-1471.

Medical Research Council. 1956. Vaccination against whooping cough: relation between protection in children and results of laboratory tests. British Medical Journal 2:454-462.

Medical Research Council. 1959. Vaccination against whooping cough: the final report. British Medical Journal 1:994-1000.

Melchior JC. 1977. Infantile spasms and early immunization against whooping cough. Danish survey from 1970 to 1975. Archives of Disease in Childhood 52:134-137.

Meyer HM, Parkman PD, Hobbins TE, Larson HE, Davis WJ, Simsarian JP, Hopps HE. 1969. Attenuated rubella viruses: laboratory and clinical characteristics. American Journal of Diseases of Children 118:155-165.

Modlin JF, Brandling-Bennett AD, Witte JJ, Campbell CC, Meyers JD. 1975. A review of five years' experience with rubella vaccine in the United States. Pediatrics 55:20-29.

Perkins FT. 1985. Licensed vaccines. Reviews of Infectious Diseases 7:S73-S76.

Pichichero ME, Badgett JT, Rodgers GC, McLinn S, Trevino-Scatterday B, Nelson JD. 1987. Acellular pertussis vaccine: immunogenicity and safety of an acellular pertussis vs. a whole cell pertussis vaccine combined with diphtheria and tetanus toxoids as a booster in 18- to 24-month old children. Pediatric Infectious Disease Journal 6:352-363.

Plotkin SA. 1988. Rubella vaccine. In: Plotkin SA, Mortimer EA, eds. Vaccines. Philadelphia: W.B. Saunders Co.

Plotkin SA, Farquhar JD, Katz M, Buser F. 1969. Attenuation of RA27/3 rubella virus in WI38 human diploid cells. American Journal of Diseases of Children 118: 178-185.

Polk BF, Modlin JF, White JA, DeGirolami PC. 1982. A controlled comparison of joint reactions among women receiving one of two rubella vaccines. American Journal of Epidemiology 115:19-25.

Pollock TM, Miller E, Mortimer JY, Smith G. 1984. Symptoms after primary immunisation with DTP and with DT vaccine. Lancet 2:146-149.

Preblud SR. 1985. Some current issues relating to rubella vaccine. Journal of the American Medical Association 254:253-256.

Prinzie A, Huygelen C, Gold J, Farquhar J, McKee J. 1969. Experimental live attenuated rubella virus vaccine: clinical evaluation of Cendehill strain. American Journal of Diseases of Children 118:172-177.

Ross EM. 1988. Reactions to whole-cell pertussis vaccines. In: Wardlaw AC, Parton R, eds. Pathogenesis and Immunity in Pertussis. New York: John Wiley & Sons.

Rothman KJ. 1986. Modern Epidemiology. Boston: Little, Brown & Co.

Shields WD, Nielsen C, Buch D, Jacobsen V, Christenson P, Zachau-Christiansen B, Cherry JD. 1988. Relationship of pertussis immunization to the onset of neurologic disorders: a retrospective epidemiologic study. Journal of Pediatrics 113:801-805.

Tingle AJ, Ford DK, Price GE, Kettyls DWG. 1979. Prolonged arthritis in identical twins after rubella immunization (brief report). Annals of Internal Medicine 90:203-204.

Tingle AJ, Chantler JK, Pot KH, Paty DW, Ford DK. 1985. Postpartum rubella immunization: association with development of prolonged arthritis, neurological sequelae, and chronic rubella viremia. Journal of Infectious Diseases 152:606-612.

Tingle AJ, Allen M, Petty RE, Kettyls GD, Chantler JK. 1986. Rubella-associated arthritis. I. Comparative study of joint manifestations associated with natural rubella infection and RA 27/3 rubella immunisation. Annals of the Rheumatic Diseases 45:110-114.

Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page32
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page33
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page34
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page35
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page36
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page37
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page38
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page39
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page40
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page41
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page42
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page43
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page44
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page45
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page46
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page47
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page48
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page49
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page50
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page51
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page52
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page53
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page54
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page55
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page56
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page57
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page58
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page59
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page60
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page61
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page62
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page63
Suggested Citation:"3 Methodologic Considerations in Evaluating the Evidence." Institute of Medicine. 1991. Adverse Effects of Pertussis and Rubella Vaccines. Washington, DC: The National Academies Press. doi: 10.17226/1815.
×
Page64
Next: 4 Evidence Concerning Pertussis Vaccines and Central Nervous System Disorders, Including Infantile Spasms, Hypsarrhythmia, Aseptic Meningitis, and Encephalopathy »
Adverse Effects of Pertussis and Rubella Vaccines Get This Book
×
Buy Paperback | $100.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Parents have come to depend on vaccines to protect their children from a variety of diseases. Some evidence suggests, however, that vaccination against pertussis (whooping cough) and rubella (German measles) is, in a small number of cases, associated with increased risk of serious illness.

This book examines the controversy over the evidence and offers a comprehensively documented assessment of the risk of illness following immunization with vaccines against pertussis and rubella. Based on extensive review of the evidence from epidemiologic studies, case histories, studies in animals, and other sources of information, the book examines:

  • The relation of pertussis vaccines to a number of serious adverse events, including encephalopathy and other central nervous system disorders, sudden infant death syndrome, autism, Guillain-Barre syndrome, learning disabilities, and Reye syndrome.
  • The relation of rubella vaccines to arthritis, various neuropathies, and thrombocytopenic purpura.

The volume, which includes a description of the committee's methods for evaluating evidence and directions for future research, will be important reading for public health officials, pediatricians, researchers, and concerned parents.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!