National Academies Press: OpenBook
« Previous: 2 Data Synthesis and Framework
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

3

Use Case Scenarios and Design Enhancements

The committee worked with three user groups to obtain evaluations of SMART Vaccines from various perspectives. The groups were the Public Health Agency of Canada, New York State Department of Health, and the Serum Institute of India. Profiles of these three user groups are presented in Box 1-2.

User Group Scenarios

Users were asked to choose a test case that was useful and applicable to their organization and that required a comparison of at least two vaccines. The committee offered the following initial guidance to the user groups concerning how they should apply SMART Vaccines to a real challenge they had faced, were facing, or expected to face:

  1. Identify a policy question or challenging decision for which you require a comparative analysis of two or more vaccines.
  2. Select the population in which you wish to analyze the impact of the vaccines and provide the necessary life-tables information.
  3. Specify the burden of the diseases being targeted by the vaccines of interest.
  4. Choose two or more vaccine candidates to evaluate. These can be single vaccines for each chosen disease or multiple candidate vaccines for a single disease (i.e., determining the ranking for vaccines with different bundles of attributes) or some combination thereof.
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

A template spreadsheet developed by the Phase II committee was provided to the three user groups to guide their data compilation. Additional guidance was provided to the user groups to support their data collection, with the specific guidance varying according to the details of the particular test case and the resources available to the user group. Some user groups needed minimal assistance, while others lacked the necessary expertise for gathering population-specific disease burden and vaccine data.

The Public Health Agency of Canada (PHAC) chose to use SMART Vaccines to prioritize vaccines for research and development. The agency chose chlamydia and tuberculosis within the Canadian population as its test case. Although Canada represents only a small fraction (1 to 2 percent) of the world market for vaccines, the PHAC believes that it can influence vaccine development by working with vaccine developers to use Canada as a test bed for early use of vaccines; in this way the PHAC can play a significant role in prioritization despite Canada’s relatively small portion of the world market.

The New York State Department of Health (NYSDOH) decided to use SMART Vaccines to compare two existing rotavirus vaccines, Rotateq and Rotarix, for use within New York. The department also used SMART Vaccines to help determine which of four existing influenza vaccines might best serve the population of New York, where vaccine delivery takes place through a variety of private providers as well as some public health clinics.

The Serum Institute of India used SMART Vaccines to prioritize between two vaccines, a vaccine for dengue and a vaccine for respiratory syncytial virus, for use in India. Currently no vaccines exist for either disease.

After the user groups collected the data relevant to their scenarios, they provided the data to the committee, which then sent each group an updated version of the SMART Vaccines that had been preloaded with the data that group had provided. Then each user group tested SMART Vaccines for its chosen scenario. The PHAC team consisted of staff experts in disease spread modeling, policy research, and health economics; the combination allowed the team members to efficiently gather and test data for its use cases. The NYSDOH team included a group of health officials, an epidemiologist, a computer scientist, and immunization officers who supported the effort to compile disease burden and vaccine data. The use case scenario of the Serum Institute of India was spearheaded by its corporate medical director, who was supported by a project assistant.

The users provided feedback about their experience to help the committee understand how each group used the software to analyze its

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

scenario, the usefulness of various aspects of SMART Vaccines such as sensitivity analysis, and the groups’ preferences regarding the software interface. This study was led by an independent consultant to the committee from Microsoft Corporation. The groups’ feedback is summarized by the consultant in Appendix A, and the committee’s corresponding responses or actions are provided in Appendix B.

The fourth use case scenario—which will be discussed later in this chapter—focused on using SMART Vaccines as a reverse engineering tool to determine the SMART Scores of potential vaccines for a single disease and thus offer guidance to vaccine developers concerning the most desirable bundles of attributes for potential vaccines. In some sense, this scenario expands upon the typical target product profile discussions already common in the world of vaccine discovery and production.

Data Sourcing Guidance to the User Groups

Over a 5-month period, the committee partnered with the user groups to provide general and specific advice for data collection and to answer queries regarding software requirements, data needs, and other user or interface concerns.

The users were provided with general sources for finding relevant data; however, each user group also required sources of specific information concerning its identified population. To help the user groups find such information, the committee provided specific research help for the different users. For instance, NYSDOH required state-specific data on disease burden. To compare the two rotavirus vaccines, highly granular data were needed for Rotateq and Rotarix vaccines within New York, and the committee offered customized help concerning such data. All of the datasets compiled by the user groups in conjunction with the committee are available upon request through the Public Access Records Office accessible from the Current Projects System page of the National Academies website.

Updated Features in SMART Vaccines 1.1

The committee found the usability studies with the three user groups to be very useful and productive. As a result of those studies, several updates and enhancements to SMART Vaccines were made. These updates and enhancements are illustrated with various screenshots in this section.

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

Terms of Use

The opening page of SMART Vaccines 1.1 presents the terms of use and a disclaimer from the National Academy of Sciences (see Figure 3-1). Once users click the “Accept” button, they enter the program. From this point forward, navigation occurs by using the “Continue” button at the upper right corner of the screen. Subsequent screenshots from the SMART Vaccines 1.1 demonstrate the functions of each page. The functions are grouped into two sections: Specify and Evaluate.

Specifications

The Specify group contains three separate pages for the choice and entry of data that are used in subsequent analyses. The categories of data include Population, Disease, and Vaccine.

SMART Vaccines 1.1 has built-in population data and estimated wage rate data for the 34 countries in the Organisation for Economic Cooperation and Development (OECD) as well as for India, New York State, and South Africa. To navigate to a specific nation’s population page, users need to click and select from the drop-down list that gives access to specific country-level populations (see Figure 3-2).1

Just as in version 1.0, SMART Vaccines 1.1 provides detailed population data, including life-table information and average hourly wage rates, all of which are used in subsequent calculations for determining the effects of various vaccines (see Figure 3-3).2

On the page requesting information on disease burden (see Figure 3-4), users enter population-specific information about the burden of various diseases of interest. Vaccines targeting these diseases will be available for later comparison and evaluation. For each disease of interest, users must enter two types of information: disease burden data and illness descriptors.

The disease burden data require standard epidemiologic estimates of annual incidence and case-fatality rates for diseases in four age groups: infants, children between 1 and 19 years of age, adults between the ages of 20 and 65, and the elderly, that is, those of age 65 and above. Once a dis-

____________

1 SMART Vaccines 1.1 currently does not have the capability for users to define their own subpopulation—for example, a state or a province—or to do a collective analysis of a vaccine’s impact on a group of nations, although future versions could accommodate this feature.

2 SMART Vaccines 1.1 eliminates a column that SMART Vaccines 1.0 contained where information was requested on Health Utilities Index 2 for age-specific determination of quality-adjusted life years (QALYs). That variable, used only in one attribute’s calculations, is not available except for few national populations (e.g., the British Commonwealth nations, Canada, and the United States).

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-1
Opening page containing the software’s terms of use. SMART Vaccines 1.1 is currently functional only on the Windows platform.

ease is specified (e.g., pneumococcal infection) and the relevant data are entered, users can save the data for subsequent use. Users can define as many diseases as they desire, and they can define multiple potential vaccines targeting each specific disease.3

On the disease page, users must also identify for each disease specified the types of illness outcomes that the disease might cause. These might simply be different degrees of severity (e.g., mild or severe), or they might be distinct diseases (such as, in the pneumococcal infection example in Figure 3-4, meningitis, sinusitis, or otitis media). Users specify the mix of these outcomes (percentage of cases, which must add up to 100 percent), and for each disease state users specify the disutility associated with the condition (e.g., 0.02 for meningitis), the disability weight, the duration (in days) of the condition or its treatment, and the annual costs of treating that disease. The duration measure is used in the calculation of

____________

3 SMART Vaccines 1.1 cannot analyze vaccines that affect multiple diseases, for example, combination vaccines that protect against diphtheria, tetanus, and pertussis (DTP).

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-2
SMART Vaccines 1.1 population selection map.

quality-adjusted life years (QALYs, which also use the disutility toll) and of disability-adjusted life years (DALYs, which also use the disability weight).

Users are also asked to specify a single value for the annual costs of treating each disease condition. SMART Vaccines 1.0 sought highly detailed data with which to calculate the annual costs. User feedback indicated to the committee that the format was overly restrictive, and the committee responded by replacing that detailed matrix for data entry with a single value (total cost) in SMART Vaccines 1.1. Users need to estimate that total cost offline by using the best data and the best analytic approach that their local resources permit (which may range from an informed expert’s best estimate to richly supported true cost data). Users are also asked to specify the costs of a death occurring due to the disease, such as the $4,453 shown in Figure 3-3 as the cost of a death from pneumococcal infection.

Once they have finished entering all of these data for each relevant disease, users hit the “Continue” button at the upper right corner of the page, which takes them to the next page, where vaccine characteristics are defined.

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-3
Population specification screen in SMART Vaccines 1.1. For user convenience, demographic characteristics—population data, life-table information, and wage data—have been preloaded for 34 OECD countries as well as for India, New York State, and South Africa.

Vaccines

Having defined their disease or diseases of interest, users next specify the attributes of a single vaccine or multiple vaccines with different design features that would protect against each disease. When using SMART Vaccines to set priorities for new vaccine development, these attributes are necessarily hypothetical. For some other uses (e.g., selecting among existing vaccines, as one of the user groups chose to do), the vaccine attributes are known with much greater certainty.

Using the same four age brackets as used for the disease burden data, the Vaccines page asks users to indicate with a check box whether or not the vaccine targets each age group and to specify the percentage of each age-group expected to receive the vaccination (“coverage”) and the percentage of those vaccinated persons who will gain immunity (“effectiveness”). The number of individuals in the age-specific population groups is brought directly from the previously chosen population profiles (see Figure 3-5).

For each vaccine, users are asked to specify with a check box whether “herd immunity” applies to this vaccine-disease combination. In SMART

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-4
Disease specification screen in SMART Vaccines 1.1. Having been streamlined since SMART Vaccines 1.0, this page requires user single entry for costs associated with the treating the disease burden.

Vaccines 1.1 (as well as in version 1.0), the herd immunity feature specifies that if greater than 80 percent of the total population receives effective immunity—that is, if the product of the coverage and the effectiveness percentages is greater than 80 percent—then it is assumed that the entire population is protected. Later enhancements of SMART Vaccines may wish to provide more disease-specific models of herd immunity, but currently this simple approach is used.

Users can specify more than one vaccine for each disease. This provides a ready mechanism to determine the value (as measured by the SMART Score) of vaccines with different design profiles. This approach can illuminate desirable features in vaccine design in the development stage, or, as one user group did, the approach can be used to assist in choosing among a set of existing vaccines available on the market. Users can also combine the two, determining which combinations of new (improved) attributes for a vaccine would make it worthwhile to encourage the development of a new vaccine in those cases where existing vaccines provide at least partial protection against a disease. The committee’s fourth use

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-5
SMART Vaccines 1.1 screen for defining characteristics of the vaccine candidates considered for prioritization.

case scenario (described later in this chapter) took this approach to reverse engineer the desirable set of attributes of vaccines in pneumococcal vaccines for South Africa.

Upon completing data entry to specify vaccines, users use the “Continue” button to proceed to the Evaluation section of the program.

Evaluation

SMART Vaccines offers a choice among 28 attributes in eight categories as well as allowing for 7 user-defined attributes. Turning on any of the radio buttons (e.g., the economic attributes in Figure 3-6) takes the user to a set of attributes from which the user may choose one or more for a subsequent evaluation of the vaccine candidates. Because of the high similarity between DALYs and QALYs in the “health” group, users may not select both, and choosing one causes the option for the other to be grayed out. Furthermore, if a user selects, say, QALYs as a health outcome, then the user is only allowed to choose the economic variable of $/QALY—and not $/DALY—in the “economic” group.

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-6
Attribute selection page in SMART Vaccines 1.1. Options from nine attribute groups can be selected by the user for comparing vaccine candidates, including up to seven user-defined custom attributes.

Reports from earlier phases of this project emphasized the importance of careful judgment in selecting attributes. There are several reasons for this. First, if many attributes are chosen, then the weights assigned to those at the bottom of the priority list will have little meaningful effect on the rankings of candidate vaccines. Second, even with the elimination of double counting with DALYs and QALYs, users can still select sets of attributes that could create additional double counting. For example, in the “Health” section, selecting “life years saved” and either DALYs or QALYs could lead to double counting. Both because many of the attributes on long lists of attributes will inevitably be essentially irrelevant and because of “real estate” issues in screen display, SMART Vaccines limits users to selecting no more than 10 attributes.

The “user-defined” category allows users to specify their own attributes. Figure 3-7 shows the creation of a user-defined attribute evaluating the impact of a vaccine on public education. When the user completes selection of attributes, hitting the “Continue” button at the upper right corner of the screen takes the user to the next step in the Evaluate section—the determination of weights to be used in the SMART Score calculation.

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-7
Creation and inclusion of a user-defined attribute. These entries are answered with a yes or a no response in SMART Vaccines 1.1.

The first step in the Weights section asks the user to rank the selected attributes by order of importance. Figure 3-8 shows a set of attributes that a hypothetical decision maker has selected. The standard approach in using this sort of ranking has the user specify first the most important attribute—number 1—using the pull-down box associated with each attribute. The user should then select the least important attribute (number 7 among 7 attributes selected previously). Next the user selects the most important of the remaining attributes (number 2), and then the least important of the remaining attributes (number 6), and so on, proceeding in this way until all attributes have been ranked.

SMART Vaccines uses these ranks to provide an initial estimate of the weights the user might wish to assign to each attribute, with the weights summing to 100 percent. The weights are calculated using the rank-order centroid process, which is described in detail in the Phase I report (IOM, 2012). Essentially, the rank-order centroid process calculates the average of all possible combinations of weights that are consistent with the original rank ordering specified by the user, and then that set of weights is assigned

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-8
SMART Vaccines 1.1 screen for ranking and weighting attributes. Initial weights are produced by the software using the rank order centroid method, but they can be modified easily by the users with the provided slider bars.

to the attributes. This set of initial weights appears automatically once the user has completed the ranking process (see Figure 3-8).

Users can then freely adjust the weights attached to each attribute by using the slider bars for each attribute (the Modify option), after which the determination of the weights is complete (see Figure 3-9). For example, the hypothetical user chose to reduce the 37 percent weight applied to “incident cases prevented per year” in Figure 3-8 to 25 percent and increase the weight on “impact on public education” in Figure 3-9 to 15 percent. The other weights are automatically adjusted so that they all add to 100 percent. At this point, using the “Continue” button will take the user to the page where SMART Scores are calculated.

On the Priorities page the user can select up to five vaccine candidates for simultaneous comparison. The limit of five candidates is determined by screen real estate, but users can always calculate SMART Scores for a set of five candidates, save the results using the Print button at the lower right corner of the page, and then proceed to define another set of

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-9
SMART Vaccines 1.1 screen showing the alteration of the weights based on the user’s preference.

five candidates. The calculated SMART Scores will be the same as if all 10 candidates had been analyzed simultaneously.4

For each candidate vaccine selected, users must fill in the appropriate value for all of the selected attributes that are not calculated by SMART Vaccines.5 As Figure 3-10 shows, some of these attributes have values calculated from previously entered data—in particular, the health and economic attributes. Other attributes must be defined by the user.

As the attribute values are completed for a candidate vaccine, a SMART Score appears in the display box on the right side of this screen. In this hypothetical example, the user’s selection of vaccine candidates for pneumococcal infection, human papillomavirus, and rotavirus results in

____________

4 This is possible because multi-attribute utility models are independent of irrelevant alternatives (IIA), meaning that the scores are independent of the actual comparison set. As the Phase 1 report discussed in more detail, other multi-criteria decision analysis models (including the Analytic Hierarchy Process) do not possess this desirable feature.

5 The reader is referred to the discussion in Chapter 2 regarding boundary setting for the software-defined attributes.

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-10
SMART Score output screen showing vaccines being compared, their computed or selected attribute values, and the color-coded final SMART Scores, a composite of quantitative and qualitative values, highlighting the relative differences between the candidates.

the scores of 52, 37, and 33, respectively. The bar graphs not only provide the total score but also show how much each domain of attributes (e.g., health, economic, demographic, and other categories) contributes to the total score by dividing the bar into color-coded sections.

It is important to keep in mind that the SMART Scores do not provide relative values. A score of 90 is not twice as good as a score of 45, for example, although it is 45 points higher. In other words, the differences in scores have meaning, but their relative sizes do not. Both the Phase I and the Phase II reports discuss this feature in detail (IOM, 2012, 2013). For a simple but useful analogy, users should think of these scores as temperatures that can be given using either the Fahrenheit or the Celsius scale. In neither of these scales is 20 degrees twice as warm as 10 degrees, and 20°F is not the same as 20°C, but the concept of “a difference of 20 degrees” (in either Fahrenheit or Celsius) does have a consistent meaning. Similarly, the SMART Scores of one user do not correspond to those of another, but it still makes sense to speak of differences in the SMART Scores in a single user’s analysis.

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

A small box in the lower right corner of the screen allows the user to select different ways of conducting a sensitivity analysis (the Analysis option). If the user selects the Weight button, then the user simultaneously has the ability to adjust the weights on each attribute and see the effect on SMART Scores. Users should set the weights before conducting the analysis and generally should not modify those weights once they are established. One could, however, use this capability legitimately to explore the scores of a person or entity with a differing viewpoint (e.g., a health minister versus a vaccine developer), which is why this capability is made available.

At any point during the process, the user can capture the state of the program along with a time stamp by using the Print button, enter notes on that analysis for future reference, and save it as a portable document file (PDF). In this screen the upper box shows the weights attached to each attribute and the values each vaccine creates along that attribute’s dimension, the lower box describes the vaccine product profile as specified for each vaccine, and the box on the right shows the SMART Score of each candidate (see Figure 3-11).

Two Aspects of SMART Scores

Users should be aware of two features in the display of SMART Scores. First—consistent with the way that multi-attribute utility models generally work—the SMART Scores can go above 100 or below 0. A score above 100 occurs if a candidate vaccine has an attribute outcome (e.g., cases averted) that substantially exceeds the “best-case” outcome boundary established for the population, coupled with a significant weight placed by the user on that attribute. For example, if a candidate vaccine achieves a score of 300 on a single attribute and the user has placed a weight of 40 percent on that attribute, then the multi-attribute utility algorithm adds 120 points to the SMART Score, and the total score will include that 120 value plus contributions from other attributes. The vertical axis on the SMART Score range dynamically adjusts to accommodate scores outside the 0 to 100 range.

Attribute values—and hence also SMART Scores—can also fall below 0 if an attribute value is worse than the “worst-case” outcome established for that attribute. For example, the worst case for a $/QALY cost-effectiveness ratio is set at 15 times the per capita income in the population of interest (e.g., in the United States, at $150,000). If the $/QALY for a candidate vaccine was actually $250,000, then it would have an attribute value of $100,000 more than the worst-case boundary, and hence receive an attribute score of –67, because the boundary values of 0 and $150,000 pro-

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-11
A new feature in SMART Vaccines 1.1, a Print command, summarizes the input (vaccine characteristics) and output information used in the analysis along with a time stamp for record.

vide a range of 150,000 and the actual value is two-thirds higher (worse). And if sufficient weight is placed on an attribute with such an outcome, the SMART Score can fall below 0. Again, the vertical axis of the graphical display dynamically adjusts to accommodate such a score.

A related case occurs when the SMART Score remains positive but has both positive and negative components. In this case, the graph shows the total score including both the positive and negative components in the sum. This is perfectly legitimate within multi-attribute utility theory, but to alert the user that such a case exists, the SMART Score graph for that candidate vaccine will show hatched bars rather than the standard solid color bars. In this situation, users should carefully attend to the actual values shown—both positive and negative contributions to the SMART Score—rather than just using the bar graph representation of the SMART Score to inform their decision making. Figure 3-12 shows a composite version of hypothetical vaccine candidates scoring above 100, scoring below 0, and having both positive and negative components in the SMART Score.

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-12
A hypothetical case comparison of four vaccine candidates where an influenza vaccine candidate scores 106 and a tuberculosis vaccine scores –5 based on the weights and impact of the attributes. Furthermore, the tuberculosis vaccine is represented by a hatched bar instead of solid colors to indicate that both positive and negative components have contributed to its SMART Score.

Key Insights from the User Groups

In this section, the committee summarizes key lessons learned beginning with the broadest policy issues and then shifting to more narrow issues in application of SMART Vaccines to the settings of the three user groups, and the officials from the Mexican Ministry of Health who served as advisory consultants.

All of these users fully understood that they were using a preliminary and evolving version of SMART Vaccines and that their feedback was to be applied toward improving the product. As a consequence, none of them attempted to use the software for actual decision making, but rather used the occasion to explore the software, both for their potential future use and to assist in the Institute of Medicine’s (IOM’s) unique product development effort.

In none of the use case scenarios did the users actually develop their official sets of attributes to be used in vaccine evaluation or the formal

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

weights to be attached to those attributes. In general, technical support staff selected a set of attributes as a starting point for discussion within their respective units, with the choices of attributes and their weights often being modified during presentation to support the discussions with higher level decision makers who would eventually make actual policy decisions.

Similarly, as the committee explored the software applications with various users, it found areas where their data were either incomplete or inaccurate, further emphasizing that the results were not real decision support but rather familiarization with the SMART Vaccines tool and its potential uses.

In Table 3-1, the committee has summarized the key things it has learned in these interactions with early users of SMART Vaccines and how this information informed the changes built into SMART Vaccines 1.1. These lessons are categorized by the software’s functional aspects.

Fourth Use Case Scenario: Product Profile Design

SMART Vaccines can be used to explore the desirability of potential vaccines with different sets of attributes. This can be done by vaccine developers using their best approximation of the attributes and weights that the public health community might use, by the public health community directly, or perhaps through a collaboration between vaccine developers and other stakeholders.

To illustrate this approach to using SMART Vaccines 1.1—including new features not previously available in SMART Vaccines 1.0—the committee came up with three hypothetical vaccines for pneumococcal infection and used data from South Africa for the test case. The vaccines in this illustration are similar but not identical to actual vaccines, and the example considers their uses in populations where vaccination against pneumococcal disease is not necessarily recommended.6

The first hypothetical vaccine under consideration, named PS23, was a polysaccharide vaccine with purified polysaccharides from 23 serotypes of bacteria, which was similar but not identical to a commercially available 23-serotype vaccine. According to the Centers for Disease Control and Prevention, a commercially available 23-valent pneumococcal vaccine has shown effectiveness of 50 to 85 percent. This information was used as baseline information for the hypothetical vaccine under consideration,

____________

6 These comparisons by the committee are for purposes of demonstrating the vaccine sensitivity analysis feature in SMART Vaccines and should not be considered as contemplating actual vaccines or their uses.

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

TABLE 3-1
Insights from User Groups and the Committee’s Notes

Software Aspects General Summary and Notes
Specify

Demographics, Diseases, Vaccines
  • Special population data may be important, but even in a sophisticated setting, difficult to find. For Canada, northern populations matter for several reasons, including climatic and geographic considerations, extremely low population density, difficulty of transportation, and ethnic differences between the native populations and the larger “standard” populations across the country. But even in a data-rich environment such as Canada, users found that obtaining reliable data on these special populations was difficult.
  • The process of entering illness burden and vaccine attributes separately for males and females seemed redundant to users when both populations would be treated identically. SMART Vaccines was created to allow differential disease burden and vaccine programmatic targeting not only for different age groups but also separately for males and females, as would be appropriate, for example, for an HPV vaccine or potential vaccines against breast cancer or prostate cancer if such were to arise.
  • The process of entering data for health care treatment costs in SMART Vaccines 1.0 seemed overly cumbersome to many users, forcing them to find and enter highly detailed sub-categories of health care use (e.g., office visits, clinic visits, emergency room visits, hospitalizations) that were not necessarily appropriate for their setting. SMART Vaccines 1.1 therefore uses a much more streamlined process for acquiring treatment costs data, the details of which users can organize offline in their own useful spreadsheet formats and then enter the results in a much more simplified way.
  • SMART Vaccines was originally created to assist in the prioritization of development of new vaccines. Nevertheless, two users (New York State Department of Health and Mexico’s Ministry of Health) had the sole goal of exploring the desirability of deploying existing vaccines in their population, and most prominently a focus on selecting among competing vaccines for the same disease (e.g., influenza). From these experiences and other discussions that committee members and staff have had with industry experts, the committee believes that this application will attract considerable attention among future users, particularly those in lower-resource regions where they do not envision having a major impact on vaccine development priorities.
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

TABLE 3-1
Continued

Software Aspects General Summary and Notes
Specify

Demographics, Diseases, Vaccines
(continued)

     Some of these applications focused not on the comparison of two or more different vaccines but on an even narrower question—given their chosen attributes and weights, which subsets of their population provided the best potential vaccination targets? For example, such questions might consider whether vaccination should focus more on infants and children, the elderly, or the working adult populations.
     The committee’s experience with these test scenarios suggests that for an expanded use of the future versions of SMART Vaccines it would be better to allow more finely granulated age groups to characterize disease burden of the population at hand and to define (with the same fine granularity) the target populations for a vaccine program’s introduction or expansion.

Evaluate

Attributes, Weights, Priorities
  • The variety of attributes was perceived by the users to be an issue potentially creating the risk of double counting. For example, the “benefits women and children” attribute could be double counted if the disease burden data focused directly on women and children. Thus, they preferred to include the women and children attribute if there was special attention beyond that created by the patterns of disease burden.
         Because SMART Vaccines calculates costs and benefits by summing across the entire affected population, it does not add any special emphasis for a vaccine that prevents—as an example—only a childhood disease such as chicken pox. The software adds up benefits only across the childhood population in such an instance and may appear to have relatively low benefit in such attributes as “reduction of incident cases” because the childhood population is a relatively small proportion of the total population. This would pertain, for example, with a disease that affected all ages such as influenza.
         To account for this, users may wish to specify a particular attention paid to children by including that attribute in their evaluation set and placing sufficient weight upon it to counter the effect of the particular vaccine helping only a fraction of the population (children, in this example).
         The risk of double counting may emerge with other measures as well. Because they are so similar, SMART Vaccines does not permit the use of both QALYs and DALYs. Because life-years are included in the calculations of both QALYs and DALYs, including QALYs (or DALYs) as well as “premature deaths averted” as attributes may create double counting.
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

TABLE 3-1
Continued

Software Aspects General Summary and Notes
Evaluate

Attributes, Weights, Priorities
(continued)
  • The committee learned that the users preferred to have the capability to modify assumptions about vaccine options dynamically during the evaluation phase, rather than having to go back to the Specify section that defined vaccines originally to alter presumed attributes of various vaccines. They preferred to see the changes in SMART Scores immediately in the Evaluate section of the software.
         SMART Vaccines 1.1 provides this capability. This version also moves the adjustment feature for the “likelihood of licensure within 10 years” (from what was a separate page in SMART Vaccines 1.0) to the same page where all other vaccine attributes are defined. This capability now appears as a separate radio button on the Evaluate page and opens up a dialog box where the user can directly modify vaccine attributes without repeating intermediate steps (e.g., selecting attributes to be used in the evaluation and assigning weights thereto).
Usability and Usefulness
  • Even within their established settings, the user groups had not yet developed a process to achieve a group-level consensus about the attributes and the weights to be attached to these.
  • The boundary values in SMART Vaccines matter in two ways. First, if they are too narrow, then the SMART Scores can go above 100, and the display in SMART Vaccines 1.0 did not accommodate this. SMART Vaccines 1.1 corrects this and allows SMART Scores to go above 100 or below 0.
         Second, when boundaries are too narrow (or too wide), the importance of an attribute is over (or under) emphasized. This arises because the multi-attribute utility model expects all attribute scores to have values between 0 and 100, and sets the weights accordingly. Within a reasonable range, allowing SMART Scores to go outside the 0 to 100 range deals with this issue, but there remains a more subtle issue if the boundaries are set so widely or narrowly that individual attribute have values that diverge too far from the 0 to 100 range anticipated by the multi-attribute utility model.
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

TABLE 3-1
Continued

Software Aspects General Summary and Notes
Usability and Usefulness
(continued)
  •      For example, if a boundary range is too wide by a factor of 10, then the component attribute scores for all candidate vaccines will shrink by a factor of 10 (compared with their scoring if the scores all fell into a 0 to 100 range). A user who intended to give that attribute 20 percent of the weight will in effect have assigned only 2 percent of the total weight to that attribute when the boundary range is too large by a factor of 10. The same thing occurs in reverse if the boundary values are set too narrowly. Consider again the effect of a 10-fold error in boundary setting. Some (or all) candidate vaccines will have attribute scores far in excess of 100, and that attribute will be over-represented in the final SMART Score by a factor of 10. In the extreme, it will swamp other attributes, even if assigned a very small weight (e.g., 1 or 2 percent).
         The committee has attended to this boundary setting problem in SMART Vaccines 1.1 as best it could with available data, but users are cautioned that these boundary value issues in general remain. At this stage of software development, boundary value recalibration must take place through recompilation of SMART Vaccines. Any time boundary values are recalibrated, all analyses must be repeated, because SMART Scores before and after the recalibration will not be commensurate.

  • Users groups—and other stakeholders—requested a method to save evaluation results at any point in the process. SMART Vaccines 1.1 includes a “print” button that shows both key states of the program (e.g., all vaccine attributes, the choices of the user for attributes to be used in the evaluation and the weights attached thereto, and the resulting SMART Scores for each vaccine candidate). These results are saved in PDF format (as named by the user) with a specific time and date stamp automatically supplied.
Decision Process
  • In no case did the users have access to (or were aware of) other software or decision aids that could carry out the types of analyses available in SMART Vaccines. In some cases, technical experts within the user groups’ organizations had written (or found access to) software that carried out sophisticated cost-effectiveness analysis on a single vaccine, but in no case did they know of or use software that allowed comparison across multiple vaccine targets, or that allowed specific inclusion of multiple programmatic attributes in the decision-support modeling. The multi-attribute capabilities of SMART Vaccines were (to the user groups’ perspective) unique. One user group described SMART Vaccines as “an amazing tool” to help support decision making.
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

TABLE 3-1
Continued

Software Aspects General Summary and Notes
Decision Process
(continued)
  • One of the user groups with significant technical expertise independently checked and verified outputs of calculations in SMART Vaccines, including cost-effectiveness ratios. In every case, their calculations matched those of the computer model. This gave them (and the committee) great confidence that the variables calculated within SMART Vaccines perform correctly. The committee has also carried out the same sort of calculation checks throughout the course of programming and testing of the software. However, no software program is devoid of bugs, and only repeated use and feedback to enhance the software system can deal with such issues over time.
  • In none of our use case scenarios did the user organization actually develop a set of attributes (and their weights) that would represent the group’s official metric for evaluation. Several of the user groups noted that they did not have an established process to carry this out. The committee believes that further research to study available methods to support the decision process would be desirable.

which was targeted for a population excluding infants. The hypothesized coverage and effectiveness rates, cost per dose, costs of administration, and developmental cost for this vaccine can be seen in Figure 3-13.

A second polysaccharide vaccine covering 30 serotypes—named PS30—was imagined with increased effectiveness rates but the same coverage rates as PS23 (see Figure 3-14). Because of the presumed additional complexity of creating a 30-serotype vaccine, the cost per dose was set higher than that for PS23, and the presumed developmental costs were set at the highest category—$1 billion or more. As with PS23, PS30 was also treated as a single-dose vaccine.

A third invented vaccine was a new conjugate vaccine that requires three doses to achieve the stated effectiveness, but with the potential to be deployed in all age groups. The assigned coverage and effectiveness rates can be seen in Figure 3-15.

Four attributes were selected for this demonstration to reflect a generic “public health” point of view: deaths averted, QALYs gained, direct cost savings, and cost-effectiveness ($/QALY). To minimize confounding changes, only these four attributes were used for the demonstration, and the

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-13
Characteristics of a hypothetical single dose 23-serotype vaccine (PS23) for pneumococcal infection in the South African population excluding infants.

ranks generated from the software’s rank-order centroid process were used without modification (see Figure 3-16). For all of these three candidates, the probability of licensure within 10 years was assumed to be 100 percent.

In the base case results, PS23 and PS30 polysaccharide vaccines scored 31 and 28, respectively, but the scores and the rank order would shift with small changes in any of the pertinent attributes (coverage, effectiveness, or costs). The PC conjugate vaccine invented for use by all ages had a SMART Score of –27. Figure 3-17 shows negative attribute values for net direct costs saved (i.e., it actually increases the total cost, including the vaccine program’s costs) and $/QALY, primarily because of the multiple-dose program and the costs per dose assumed in this scenario. Because of the positive attribute values for premature deaths averted per year and QALYs, the SMART Score for the PC vaccine is represented in a hatched bar.

To demonstrate the target product profile concept more fully, some of the key attributes were varied and the resulting changes in the scores of the hypothetical conjugate vaccine were observed. First, the expected coverage of the vaccine was increased to 80 percent. This is an external

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-14
Characteristics of a hypothetical single dose 30-serotype vaccine (PS30) for pneumococcal infection in the South African population excluding infants. The effectiveness of the vaccine, costs per dose, and the overall research and development and licensure costs are set to be substantially higher than those of the PS23 candidate.

factor beyond the core features of the product profile design, but it is still an important contributor to the vaccine characteristics. Holding everything else constant, this single change in the vaccine’s attributes shifted the SMART Score from –27 to –28 (see Figure 3-18). This decrease likely occurred because of the high cost per user associated with the triple-dosed vaccine.

In a second demonstration, increasing the potential length of immunity from 10 to 15 years while holding everything else constant produced a dramatic change in SMART Scores: from –27 to +4 (see Figure 3-19). This demonstrates that the length of immunity plays an integral role in a vaccine’s product profile design.

Further, by dropping the number of doses from three to two while maintaining the coverage (set at 80 percent), effectiveness (set at 80 percent), and length of immunity (set at 15 years), the cost per dose was reduced from $30 to $20. This brought the SMART Score for the conjugate vaccine to 35, surpassing the scores of the PS23 and PS30 polysaccharide vaccines (see Figure 3-20).

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-15
Characteristics of a hypothetical three-dose pneumococcal conjugate (PC) vaccine for use across all age groups in the South African population.

From a hypothetical South African decision maker’s perspective, this simulation demonstrated the following:

  • Although the PS30 vaccine has greater effectiveness than PS23—because it covers more serotypes of bacteria—the added costs offset those health gains, making the two nearly identical in the eyes of the hypothetical decision maker involved in this exercise.
  • The PC conjugate vaccine—in its original specification—does not provide as much value as either of the polysaccharide vaccines and would not be the vaccine of choice. But if the conjugate vaccine could be developed so that two doses provided the effectiveness originally presumed for the three dose vaccine, and if the cost per dose could be brought down to near $20 per dose, then PC becomes a stronger candidate for development compared with PS23 and PS30.

For additional analysis, one could further alter the product profile attributes of these vaccine candidates, making even greater use of the sen-

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-16
Attributes and weights selected using a traditional public health perspective (health and economic attributes alone) for comparing the hypothetical PS23, PS30, and PC vaccine candidates for pneumococcal infection in South Africa.

sitivity analysis capabilities in SMART Vaccines 1.1. The reader should keep in mind that the results depend on the choice of attributes and the weights assigned to them and that different preference settings could lead to completely different results. This sensitivity highlights the importance, when using SMART Vaccines, of agreeing on attributes and their weights at the beginning of any evaluation process rather than modifying those weights to achieve some preconceived result.

Just as in version 1.0, SMART Vaccines 1.1 provides detailed population data, including life-table information and average hourly wage rates, all of which are used in subsequent calculations for determining the effects of various vaccines (see Figure 3-3).7

____________

7 SMART Vaccines 1.1 eliminates a column that SMART Vaccines 1.0 contained where information was requested on Health Utilities Index 2 for age-specific determination of QALYs. That variable, used only in one attribute’s calculations, is not available except for few national populations (e.g., the British Commonwealth nations, Canada, and the United States).

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-17
Computed values and initial SMART Scores for the hypothetical PS23, PS30, and PC vaccine candidates. PS23 and PS30 scored 31 and 28, respectively, and the specific contributions of health (blue) and economic attributes (red) are displayed inside the bars. The PC vaccine scored –27, and the hatched bar indicates the influence of both positive (health) and negative (economic) values on the final score.

On the page requesting information on disease burden (see Figure 3-4), users enter population-specific information about the burden of various diseases of interest. Vaccines targeting these diseases will be available for later comparison and evaluation. For each disease of interest, users must enter two types of information: disease burden data and illness descriptors.

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-18
Changes to the SMART Score of the PC vaccine with an increase in coverage (a factor external to the product profile feature). The initial score of –27 dropped to –28, indicating that the additional costs associated with increasing the coverage outweighed the benefits for this vaccine.

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-19
Changes to the SMART Score of the PC vaccine caused by an increase in the length of immunity from 10 years to 15 years. The vaccine profile (sensitivity analysis) feature shows that this one product design improvement was able to elevate the score from –27 to +4.

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

image

FIGURE 3-20
Changes to the coverage (increased to 80 percent), effectiveness (increased to 80 percent), length of immunity (increased to 15 years), cost per dose (decreased to $20), and the number of doses (decreased from 3 to 2) dramatically increased the SMART Score of the PC vaccine candidate from an initial score of –27 to +35, thus surpassing the scores of PS23 (31) and PS30 (28) motivating the need for product profile changes. In this way the vaccine sensitivity analysis feature in SMART Vaccines permits the reverse engineering of product attributes for gaining comparative advantage.

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×

This page intentionally left blank.

Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 31
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 32
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 33
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 34
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 35
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 36
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 37
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 38
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 39
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 40
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 41
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 42
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 43
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 44
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 45
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 46
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 47
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 48
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 49
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 50
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 51
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 52
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 53
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 54
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 55
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 56
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 57
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 58
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 59
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 60
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 61
Suggested Citation:"3 Use Case Scenarios and Design Enhancements." Institute of Medicine and National Academy of Engineering. 2015. Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework. Washington, DC: The National Academies Press. doi: 10.17226/18763.
×
Page 62
Next: 4 Reflections and Looking Forward »
Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework Get This Book
×
Buy Paperback | $75.00 Buy Ebook | $59.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

SMART Vaccines - Strategic Multi-Attribute Ranking Tool for Vaccines - is a prioritization software tool developed by the Institute of Medicine that utilizes decision science and modeling to help inform choices among candidates for new vaccine development. A blueprint for this computer-based guide was presented in the 2012 report Ranking Vaccines: A Prioritization Framework: Phase I. The 2013 Phase II report refined a beta version of the model developed in the Phase I report.

Ranking Vaccines: Applications of a Prioritization Software Tool: Phase III: Use Case Studies and Data Framework extends this project by demonstrating the practical applications of SMART Vaccines through use case scenarios in partnership with the Public Health Agency of Canada, New York State Department of Health, and the Serum Institute of India. This report also explores a novel application of SMART Vaccines in determining new vaccine product profiles, and offers practical strategies for data synthesis and estimation to encourage the broader use of the software.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!