Ranking Vaccines A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System (2013) / Chapter Skim
Currently Skimming:
3 Data Synthesis, Software Redesign, and Evaluation
3 Data Synthesis, Software Redesign, and Evaluation
Pages 33-60
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 33... ...
The early part of this chapter is devoted to describing the committee's data synthesis efforts and the latter part toward describing the software prototyping efforts. Selection of Vaccine Candidates In Phase I the committee selected influenza, tuberculosis, and group B streptococcus as test vaccine candidates for the United States, and tuberculosis as a test vaccine candidate for South Africa.
|
From page 34... ...
In this study, the data for human papillomavirus, pneumococcal infection, and rotavirus were collected for both the United States and South Africa. Thus, a total of six datasets for the United States and four for South Africa are available as downloadable spreadsheets (along with the SMART Vaccines software package)
|
From page 35... ...
They are neither precise projections nor comprehensive analyses. For example, there are data available on the burden of influenza and on the impact of seasonal influenza vaccines in the United States, but because there are no currently licensed vaccines for group B streptococcus, the only data available from the United States for that disease concern the disease burden, with nothing on the impact of a vaccine if it were licensed; thus, the vaccine information for group B streptococcus is largely hypothetical.
|
From page 36... ...
These same colors appear in the bar graph at the lower right corner of the screen that shows the calculated SMART Scores for five hypothetical candidate vaccines: an influenza vaccine with 1-year efficacy, an influenza vaccine with 10-year efficacy, a group B streptococcus vaccine costing $100 per dose, a group B streptococcus vaccine priced at $50 per dose, and a tuberculosis vaccine that does not achieve any herd immunity. Each vaccine bar is divided into colored sections showing how much each of the nine attribute categories adds to the SMART Score for that vaccine.
|
From page 37... ...
This spreadsheet design informed the subsequent redesign of SMART Vaccines in a MATLAB platform.
|
From page 38... ...
Interface Redesign for SMART Vaccines 1.0 In Phase I the blueprint of SMART Vaccines Beta was developed using three software tools: MATLAB for the algorithm, Java servlets for the middleware, and Axure for visual interface design, with Microsoft SQL Server used for preliminary database management. Stakeholder feedback made it clear that SMART Vaccines needed to be developed in a simpler, platformindependent fashion to aid the end users.
|
From page 39... ...
shows a typical data page -- in this instance, demographic data for females in the United States that can be specified using a pull-down menu. As noted earlier, the basic population data can normally be taken directly from institutions that maintain various databases, such as the World Health Organization.
|
From page 40... ...
In the example shown in Figure 3-4, the first block of data describes the disease impact on the relevant population (in this case, females in the United States) , categorized by age group, but in less refined groupings than the actual population data.
|
From page 41... ...
In this example, which involves information concerning an influenza vaccine for the U.S. female population, separated into several age groups, the user specifies (using check marks)
|
From page 42... ...
In the next step, the user selects the vaccine attributes of interest. The attributes selected and the weights attached to them apply to every candidate vaccine (see Figure 3-6)
|
From page 43... ...
The weights calculated by the rank order centroid method appear in the bar chart on the right, with the greatest weight being applied to the attribute with the highest ranking. As with the prototype discussed earlier, the slider bars allow the user to modify these preliminary weights (calculated by the rank order centroid method)
|
From page 44... ...
. The resulting SMART Scores are conditional on the specific numbers assigned for the vaccine attributes and disease burden (additional information can be found in Appendix D)
|
From page 45... ...
The user is provided an option to compare multiple vaccine candidates using the horizontal pull-down menus for the population originally selected. The computed values appear automatically for each of the vaccine candidates that are selected, with scoring indicated in parenthesis.
|
From page 46... ...
The user is allowed to carry out real-time sensitivity analysis by making changes to three key components of the SMART Score that rely on user input -- the utility weights, the vaccine characteristics, and the disease burden data. The user can also make changes to the weights that are pre-applied and see instantaneous shifts in the SMART Scores on different screens (see Figures 3-13, 3-14, and 3-15)
|
From page 47... ...
Rank order centroid weights are used in all the scenarios as an illustration, although, as noted earlier, it is possible to adjust the weights with the slider bars in accordance to the user's preferences.
|
From page 48... ...
Users with Different Attributes and Different Ranking Systems for Two New Vaccines Hypothetical user A is a federal agency director in the United States interested in evaluating two new vaccine candidates: a preventive vaccine for human papillomavirus and an influenza vaccine. He sets his value preference with highest ranks for health burden reduction, through the measures of premature deaths averted per year (weighted at 34 percent)
|
From page 49... ...
Figure 3-19 indicates SMART Scores of 88 for a new human papillomavirus and of 65 for a new influenza vaccine that are based on user B's selected attributes. This scenario demonstrates how user A and user B selected and ranked different attributes in their prioritizations of identical new vaccine candidates and obtained different results.
|
From page 50... ...
If user X and user Y had settled on other sets of attributes and value judgments, then their preferences could have led to quite different results, as often happens in real-world scenarios. Regardless of the outcome, however, the SMART Scores can help start a discussion between the users in which they compare their differing values and results.
|
From page 51... ...
The reactions of the evaluators were overall very positive concerning the design and innovation underlying SMART Vaccines. In addition to this positive overall response, the consultants also provided feedback about possible further improvements and explored potential additional applications of SMART Vaccines, which are discussed in Chapter 4.
|
From page 52... ...
Subsequently, the prototype evaluators were re-engaged to allow hands-on interaction with SMART Vaccines and to provide additional feedback prior to the software and report release. BOX 3-1 Framing Questions for Evaluators of SMART Vaccines 1.0 • Do you foresee using SMART Vaccines in the decision-making process of your organization?
|
From page 53... ...
Data Synthesis, Software Redesign, and Evaluation 53 FIGURE 3-16 Attribute structure and ranks created by a hypothetical federal agency director (user A) for evaluating a new human papillomavirus vaccine and a new influenza vaccine.
|
From page 54... ...
54 RANKING VACCINES: A Prioritization Software Tool FIGURE 3-17 Comparison of SMART Scores for two hypothetical new vaccines resulting from user A's selected attributes and ranking system.
|
From page 55... ...
Data Synthesis, Software Redesign, and Evaluation 55 FIGURE 3-18 Attribute structure and ranks created by a hypothetical senior executive of a major pharmaceutical company (user B) for prioritizing development between a new human papillomavirus vaccine and a new influenza vaccine.
|
From page 56... ...
56 RANKING VACCINES: A Prioritization Software Tool FIGURE 3-19 Comparison of SMART Scores for two hypothetical new vaccines based on user B's selected attribute and ranking structure.
|
From page 57... ...
Data Synthesis, Software Redesign, and Evaluation 57 FIGURE 3-20 Attribute and rank structure selected by a hypothetical health minister (user X) in South Africa.
|
From page 58... ...
58 RANKING VACCINES: A Prioritization Software Tool FIGURE 3-21 Attribute and rank structure selected by a hypothetical finance and trade minister (user Y) in South Africa.
|
From page 59... ...
Data Synthesis, Software Redesign, and Evaluation 59 FIGURE 3-22 Comparison of SMART Scores for a new rotavirus vaccine and a new pneumococcal vaccine with user X's rank and value structures.
|
From page 60... ...
60 RANKING VACCINES: A Prioritization Software Tool FIGURE 3-23 Comparison of SMART Scores for a new rotavirus and a new pneumococcal vaccine with user Y's rank and value structures.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.