Ranking Vaccines A Prioritization Framework: Phase I: Demonstration of Concept and a Software Blueprint (2012) / Chapter Skim
Currently Skimming:
3 Data Evaluation and Software Development
3 Data Evaluation and Software Development
Pages 67-108
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 67... ...
A second reason for selecting South Africa was the availability of input data for disease burden and vaccine estimates, which were necessary to populate and test the model. The five hypothetical candidate vaccines chosen were a universal influenza vaccine plus vaccines against tuberculosis, group B streptococcus, malaria, and rotavirus.
|
From page 68... ...
The committee chose to develop reasonable estimates for data based on literature reviews and expert opinion, and it sometimes also relied upon committee-generated assumptions because much of the information required for the model, especially information concerning South Africa, was not available. It is thus reasonable to view the data inputs as characterizing hypothetical vaccines against influenza-like, tuberculosis-like, and group B streptococcus–like syndromes.
|
From page 69... ...
:1162–1170. • Hourly wage rate is The BLS Current Population Wage Rate for South Africa Wage Rate gathered from the Bureau Survey data were used was crudely estimated of Labor Statistics.
|
From page 70... ...
• CDC, WHO See disease and vaccine Annual tables in Appendix Incidence Rate • Published literature C for details. • Published literature Case Fatality Rate • Expert opinion Vaccine Coverage Vaccine Effectiveness Assumed to be 100 Herd Immunity percent due to infectious Threshold nature of each disease.
|
From page 71... ...
DALY weights, proxies were used to estimate values for Disability tolls and disability weights. Weight Duration • Published literature See disease and vaccine Vaccine- Morbidity tables in Appendix Related • Expert opinion Probability C for details.
|
From page 72... ...
Vaccine efficacy, vaccine-associated compli cations, coverage, and the number of doses required for immunity were estimated from the literature, whereas time to adopt a vaccine within an immunization scheme, development risk, and innovation for new delivery methods were guided by expert opinions. Data on health care costs for disease and vaccine candidates were obtained from both a literature review and governmental Web sites such as those for HCUP and CDC for the United States and WHO's Choos ing Interventions that are Cost-Effective (CHOICE)
|
From page 73... ...
As previously shown (Figure 2-1) , the computational submodel calculates multiple health and economic measures associated with new vaccine candidates.
|
From page 74... ...
Development of the computational submodel The computational submodel contains expressions for health and economic values that are based on a population process model. The process model is initialized at year i = 0 for a stationary population with: no vaccine (i.e., the baseline population)
|
From page 75... ...
at model initialization. The age-specific population process model simulates measures of population size for the total population, the target population, the vaccinated immune members of the populations, the vaccinated susceptible members, the not-vaccinated immune members (i.e., those who have indirect protection through herd immunity)
|
From page 76... ...
Within these tables, vaccine population references are assumed to be the populations where the vaccine is first introduced. Evaluation of the computational submodel The computational submodel has been evaluated using four base cases for preventative vaccine candidates.
|
From page 77... ...
Premature 12,095 1,248 671 28,973 Deaths Averted Incident Cases 6,123,612 14,841 7,451 140,239 Prevented QALYs Gained 21,011 3,571 1,373 40,680 DALYs Averted 8,665 1,170 622 21,421 Economic Attributes Vaccine Steady State (per Year) −$95,357,702 Net Direct $1,929,730,356 $274,313,238 $253,174,240 Costs (Delivery -- Health Care)
|
From page 78... ...
Intuitively, the resulting health attribute measures should increase as a result of the indirect protection associated with herd immunity. Finally, the vaccine coverage for tuberculosis in the South Africa base case was increased.
|
From page 79... ...
Health Attributes Vaccine Steady State (per Year) Premature 12,095 1,248 838 43,459 Deaths Averted Incident Cases 6,123,612 14,841 9,314 210,358 Prevented QALYs Gained 21,011 3,571 1,719 61,020 DALYs Averted 8,665 1,170 777 32,131 Economic Attributes Vaccine Steady State (per Year)
|
From page 80... ...
The lower block and the bar chart display outputs of the multi attribute model. The selected attributes from the upper areas have been assigned weights, the categorical achievement levels on each attribute have been scaled with the weights, and the scaled weights have been summed to display a total priority score for each of the five vaccines at the bottom in the orange colored rows.
|
From page 81... ...
This requirement was added after some concept evaluators of SMART Vaccines Beta strongly endorsed
|
From page 82... ...
In this example, the user (say, a vaccine manufacturer) has entered numbers from 1 to 11 in the second column to indicate the rank order of importance of the selected attributes, with 1 being most important and 11 being the least important.
|
From page 83... ...
In the figure, the influenza vaccine scored a total of 0.635, which multiplied by 100 is 63.5; the tuberculosis vaccine scored 35.3, the group B streptococcus vaccine scored 61.9, and vaccines D and E scored 22.9 and 61.1, respectively. The weights in the table are assigned using the rank order centroid method described in Chapter 2.
|
From page 84... ...
This experiment was conducted while the prototyping of the model was ongoing, so the vaccines evaluated and the assumptions were slightly different from the completed SMART Vaccines Beta. The committee members and staff ranked the following six vaccines: 1.
|
From page 85... ...
The group included individuals with diverse perspectives, from infectious disease epidemiologists and authorities on health care in low-income countries to experts in health economics, systems engineering, and decision sciences. The weights were then applied against the vaccines under consideration through SMART Vaccines Beta to calculate each person's priority score for the vaccine and disease combinations.
|
From page 86... ...
86 TABLE 3-4 Scores from the Value Experiment 72 42 36 68 61 43 56 61 52 54 36 63 36 49 64 Influenza, 1-year immunity 76 52 36 70 69 47 57 77 54 66 44 69 47 67 67 Influenza, 5-year immunity 34 35 24 46 14 13 21 10 18 28 27 11 20 18 22 Tuberculosis, 3-year immunity Tuberculosis, lifetime immunity 43 44 35 29 47 20 13 58 10 28 29 42 26 42 24 Influenza, 1-year immunity, 50% increased coverage 86 42 36 89 69 55 77 69 73 70 44 76 41 58 84 Tuberculosis, 3-year immunity, 100 times increased incidence 49 87 92 66 80 89 70 65 58 53 57 58 71 59 59 A B C D E F G H I J K L M N Aggregate Table 3-3 EDITABLE.eps Broadside, now
|
From page 87... ...
What do the scores mean? The most important result they offer is to provide rank orderings.
|
From page 88... ...
In its current version SMART Vaccines Beta has a six-step process for producing a value score for vaccine candidates. All the data and the results shown in the screenshots are hypothetical and should not be interpreted as any form of endorsement by the committee or the Institute of Medicine.
|
From page 89... ...
in mind on which to base the ranking of his or her set of vaccine candidates. Step 1 in the process has the user specify the attributes of importance toward the ultimate ranking of candidate vaccines.
|
From page 90... ...
SMART Vaccines Beta converts the rank order of attributes selected in the drag-and-drop box into numerical weights to be used in the multi attribute value model. Chapter 2 described this process and provides refer ences to justify this approach.
|
From page 91... ...
Screen Figure 3-6.eps software requires complete templates for males and females, provided in 5-year age intervals for adults and more refined for children. While the data demands in this step seem considerable, the data can be readily imported from available databases at the World Health Organization for most populations around the world.
|
From page 92... ...
. Step 3: Disease Burden Step 3 takes the reader into the specification of disease burden (Figure 3-7)
|
From page 93... ...
Health SMART Vaccines Beta automatically fills in the population size in each relevant population group from data shown at Step 2, so the user must fill in population-specific information about the annual disease incidence per 100,000 persons in each age group, the case-fatality proportion, and the herd immunity threshold. The herd immunity threshold provides a simple way to specify whether there is any meaningful herd immunity effect from the vaccine.
|
From page 94... ...
. With transmissible diseases such as influenza, one could set the herd immunity threshold at, say, 80 percent, indicating that once 80 per cent of the population has achieved immunity, the remaining population gain protection through herd immunity.
|
From page 95... ...
data-entry burdens. Ultimately the age categories in Step 3 will be able to take on the same level of refinement as the population data in Step 2.
|
From page 96... ...
DALY disability weights are normally drawn from expert opinion, and typically users find related DALY weights in publica tions from the developers of the DALYs approach. In this example screen (influenza in the United States)
|
From page 97... ...
Screen Figure 3-12.eps impairment as appropriate for each disease. These categories of morbidity are combined with the cost of each type of treatment (see bottom of Step 3 screen)
|
From page 98... ...
. SMART Vaccines Beta automatically fills in the population numbers for each age group.
|
From page 99... ...
. Of course, these are not known with certainty before actual development, so users must use expert opinion to conjecture about the candidate vaccines.
|
From page 100... ...
how the computed attributes and the priority score have changed. This gives an "on the fly" capability to see how these attributes affect rankings and their computed components, and it allows users to consider trade-offs between attributes as they focus product devel opment efforts -- for example, choosing larger research and development costs but reducing the costs to administer by removing cold-chain require ments or product shelf-space demands.
|
From page 101... ...
in Step 3 (Disease Burden) , but in this case they refer to complications of a candidate vaccine rather than to the consequences of unprevented disease.
|
From page 102... ...
Step 5: Value Assessment Step 5 asks users to enter qualitative information about each vaccine. These come in eight categories, as previously shown in Table 2-1.
|
From page 103... ...
Sources of uncertainty and how they affect SMART Vaccines are briefly discussed, along with some possible methods to address these issues in Phase II. Uncertainty About the Likelihood of Successful Licensure SMART Vaccines Beta includes one uncertainty component but instead of listing it as a probability the committee characterized it as a value attribute: "Likelihood of Successful Licensure in 10 Years" under "Scientific and Business Considerations" (Table 2-1)
|
From page 104... ...
The effect of using such an attribute in the value submodel is functionally equivalent to including a direct estimate in the computational submodel -- vaccine candidates that are expected to be licensed sooner will receive higher scores and those not expected to be licensed soon will receive lower scores when everything else is equal. There are advantages to embedding this uncertainty component in the value submodel.
|
From page 105... ...
Parameter Uncertainty The computational submodel in SMART Vaccines Beta, although simplistic in its current form, is a function of many parameters: population modeling, estimates of health burden and benefits, and estimates of health care costs. Each of these parameters has components of uncertainty surrounding it.
|
From page 106... ...
Other parameters in the model whose uncertainty may be best addressed with sensitivity analysis include vaccine effectiveness and the duration of immunity. Computation of outputs which are functions of uncertain inputs can be accomplished either by Monte Carlo simulation, or using Markov Chain Monte Carlo simulation to build a pseudo-distribution for the outputs if simple independent sampling of parameters is not realistic within the computational submodel.
|
From page 107... ...
also identify model uncertainty as uncertainty about whether the computational model itself is an adequate representation of the process that is being investigated. In regards to SMART Vaccines Beta, this uncertainty concerns whether the structure of the computational submodel is adequate.
|
From page 108... ...
Many of the features of SMART Vaccines Beta have already been updated in response to the comments from concept evaluators. More important, many features have the potential to be upgraded in Phase II of this study.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.