National Academies Press: OpenBook

Evidence-Based Practice for Public Health Emergency Preparedness and Response (2020)

Chapter:Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence

« Previous: 8 Improving and Expanding the Evidence Base for Public Health Emergency Preparedness and Response
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

A

Detailed Description of the Committee’s Methods for Formulating the Scope of the Reviews and Capturing the Evidence

The committee was charged with conducting a systematic review of the evidence base for selected public health emergency preparedness and response (PHEPR) practices from the 15 PHEPR Capabilities defined in the Centers for Disease Control and Prevention’s (CDC’s) Public Health Emergency Preparedness and Response Capabilities: National Standards for State, Local, Tribal, and Territorial Public Health (CDC, 2018). This appendix provides additional detail on some aspects of the committee’s evidence review methods, which are briefly described in Chapter 3. Specifically, this appendix details the committee’s review methods only to the point of data extraction and quality assessment of individual studies. The committee’s evidence synthesis and grading methodology is described in detail in Chapter 3. For examples of how the methods were applied to the review topics, see Chapters 47.

To develop its evidence review and evaluation methodology, the committee reviewed the standards, protocols, and best practices from several prominent guideline groups and their relevant publications (see Box A-1), although other sources were also consulted. Recognizing the complexity of PHEPR practices, the committee focused substantial effort up front on formulating the scope of the reviews and adapting review methods that would take into account practice and system complexity. Additional information regarding the complexity perspective that guided the committee’s approach to its task can be found in Chapter 3.

FORMULATING THE SCOPE OF THE REVIEWS

Selecting the Review Topics

In response to its charge, the committee developed a process for identifying which of the CDC PHEPR Capabilities would be the focus of its systematic literature reviews. Recognizing that the PHEPR Capabilities are broad, overarching topics, the committee gradually focused the scope of its reviews to specific PHEPR practices within the selected PHEPR Capabilities.

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

The evidence review topics were scoped through iterative stages with input from the literature, key stakeholders, and the committee’s expertise.

Preliminary Literature Review

To better understand the current evidence base for PHEPR practices, the committee conducted a preliminary literature review. This preliminary review helped formulate the scope of the final reviews and frame the key review questions that governed the committee’s systematic searches of the evidence. The committee focused its efforts on reviewing published scoping and literature reviews on the PHEPR literature (Abramson et al., 2007; Acosta et al., 2009; Challen et al., 2012; Khan et al., 2015; Savoia et al., 2017; Yeager et al., 2010) to get a sense of the research gaps in the field, as well as PHEPR practitioner and program assessments, when relevant, to understand the type of research that would be relevant and important to practitioners (CDC, 2016; Center for Public Health Systems and Services Research, 2018; Horney et al., 2017; Murthy et al., 2017; Siegfried et al., 2017).

Engaging with Stakeholders to Meet Practitioner Needs

To ensure that its report would be relevant and useful to key stakeholders, the committee appointed nine diverse state, local, tribal, and territorial (SLTT) PHEPR practitioners as consultants to advise on the systematic literature review process. The members of this group were suggested by practitioner associations, such as the National Association of County & City Health Officials (NACCHO) and the Association of State and Territorial Health Officials (ASTHO), and selected based on such criteria as jurisdiction type and demographics, size

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

of public health agency, individual tenure and experience, number of emergencies during tenure, the Federal Emergency Management Agency (FEMA) region, and state governance, as well as meeting conflict-of-interest requirements. The committee engaged with these PHEPR practitioner consultants throughout the review process on the refinement of its conceptual approach and the selection, development, and refinement of review topics, and solicited their feedback on the review findings and the committee’s recommendations. The PHEPR practitioner consultants provided real-world input and assisted in focusing the review topics on issues relevant to decision makers.

Selection Criteria and Selected Review Topics

In making decisions about the scope of its reviews, the committee, which included individuals with diverse expertise in evidence review and evaluation methodologies and the PHEPR subject matter (see Appendix F for biosketches of the committee members), considered criteria adapted from The Community Guide (Zaza et al., 2000) and the U.S. Preventive Services Task Force (USPSTF, 2015). These criteria are described in Box A-2.

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Selected review topics

The committee determined that, to develop an evidence review and evaluation methodology and demonstrate its feasibility, the task of selecting review topics needed to be approached from a “proof-of-concept” perspective. Therefore, for the initial development of its methodology for generating recommendations for evidence-based PHEPR practices, the committee applied the selection criteria discussed in Box A-2 and narrowed the scope of its reviews to four PHEPR Capabilities: Community Preparedness, Emergency Operations Coordination, Information Sharing, and Non-Pharmaceutical Interventions. In consultation with the PHEPR practitioner consultant group, the committee further focused its reviews on four topic areas within each of these four PHEPR Capabilities:

  • engaging with and training community-based partners to improve the outcomes of at-risk populations after public health emergencies (falls under Capability 1, Community Preparedness);
  • activating a public health emergency operations center (Capability 3, Emergency Operations Coordination);
  • communicating public health alerts and guidance with technical audiences during a public health emergency (Capability 6, Information Sharing); and
  • implementing quarantine to reduce or stop the spread of a contagious disease (Capability 11, Non-Pharmaceutical Interventions).

The process of focusing the selected broad PHEPR Capabilities into more precise topics was a considerable challenge because the Capabilities are standards, not practices; thus, to achieve each Capability requires many different practices, which are not always evident. To guide the selection process, the committee developed a comprehensive list of potential PHEPR practices by breaking down the functions and tasks within the PHEPR Capabilities into more manageable topics.

Selection justification

It is important to note that the committee’s process for selecting review topics was systematic and based on classifying the topics on a number of key dimensions and criteria. The committee acknowledges that a different group might have chosen a different set of topics. Given that the committee was able to conduct reviews for only a very small subset of PHEPR practices, it sought to inform future priorities for review topics by commissioning a scoping review and evidence map to examine and describe the extent and the nature of research conducted on practices within all of the CDC PHEPR Capabilities (see Chapter 2 and Appendix D). The discussion below describes how the committee applied the criteria listed in Box A-2 to select review topics.

Relevance and importance to national health security

In thinking more broadly about the strategies, priorities, strengths, and weaknesses in PHEPR for the nation, the committee reviewed the 2018 National Health Security Preparedness Index;1 Trust for America’s Health 2017 Ready or Not?: Protecting the Public’s Health from Diseases, Disasters, and Bioterrorism Report;2 CDC’s Public Health Preparedness and Response 2018 National Snapshot;3 CDC’s Public Health Emergency Preparedness (PHEP) Cooperative Agreement funding announcement;4 and the Assistant Secretary for Preparedness and Response’s

___________________

1 See https://nhspi.org (accessed June 18, 2020).

2 See https://www.tfah.org/report-details/ready-or-not-2017 (accessed June 18, 2020).

3 See https://www.cdc.gov/cpr/pubs-links/2018/documents/2018_Preparedness_Report.pdf (accessed June 18, 2020).

4 See https://www.cdc.gov/cpr/readiness/phep.htm (accessed June 18, 2020).

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

(ASPR’s) National Health Security Strategy.5 The committee deliberated as to where a review and evaluation of the evidence might advance the capacity of the nation to prepare for and respond to disasters and public health emergencies.

Current needs among key stakeholders

In considering the current needs for research to support practices among key stakeholders, including practitioners and policy makers, the committee reviewed the research domains and questions deemed a priority by practitioners in Siegfried et al. (2017) and the findings from the 2016 practitioner assessment of the CDC PHEPR Capabilities (CDC, 2016). Siegfried and colleagues’ findings provided crucial insight into where PHEPR practitioners perceived knowledge gaps and where future research studies would provide the information they need to enhance their capacity. One key finding was that the research question within the community resilience domain related to appropriate methods and procedures for identifying and mapping at-risk populations received the highest importance rating from the practitioners surveyed. As described later in this appendix, the committee leveraged this important work in conducting a structured priority-setting activity to identify priority topics for future systematic evidence reviews (refer to the section below on “Prioritization of Future Systematic Evidence Review Topics”).

The committee also considered the input offered by CDC in its remarks to the committee at the outset of this study, as well as input from the PHEPR practitioner consultants. In the 2016 practitioner assessment of the CDC PHEPR Capabilities, it was noted that such Capabilities as Mass Care, Fatality Management, Community Recovery, and Medical Surge are often the responsibilities of agencies other than public health (CDC, 2016). In its remarks, CDC also noted that Volunteer Management is less critical for public health than other PHEPR Capabilities and that it should be of lower priority in the allocation of resources for review (Carbone, 2018). Public Health Laboratory Testing and Public Health Surveillance and Epidemiological Investigation are considered well-established public health functions (Carbone, 2018; CDC, 2016). Furthermore, the evidence supporting Emergency Public Information and Warning, Responder Safety and Health, and Medical Materiel Management and Distribution is derived primarily from disciplines other than public health. Therefore, the committee decided not to focus its efforts and resources on reviewing these Capabilities because an aim of the study was to develop PHEPR-specific methods.

Potential to affect PHEPR practice

Siegfried and colleagues (2017) note that knowledge gaps identified by the practice community may result from insufficient dissemination, ineffective implementation, or a lack of existing research. Abramson and colleagues (2007) state that it is fundamental for potential research questions to question assumptions (e.g., asking whether the way in which response systems have been organized using an incident command system is effective). The committee deliberated about those areas in which a review and evaluation of the evidence would have the highest potential to increase the implementation of effective practices and phase out widely used but less effective practices.

Methodological diversity

The committee approached its reviews as a proof of concept for its methodology for generating recommendations for evidence-based PHEPR practices. Therefore, the committee assigned this criterion more weight than the others to ensure that the methodology it developed would be flexible enough to accommodate the range of PHEPR practices that may be reviewed in the future. Through an initial review of the literature

___________________

5 See https://www.phe.gov/Preparedness/planning/authority/nhss/Documents/NHSS-Strategy-508.pdf (accessed June 18, 2020).

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

and its members’ expertise, the committee considered how the type of research required to support evidence-based practice recommendations might vary across the CDC PHEPR Capabilities. The type of research that falls within the Community Preparedness and Information Sharing Capabilities aligns more with traditional research designs, such as randomized controlled trials (RCTs) and quasi-experimental studies, whereas the type of research that falls within Emergency Operations Coordination aligns more with noncomparative studies and experience-based evidence, including case reports and after action reports (AARs), as well as organizational theory, systems, and processes. The committee was also interested in developing a methodology that would accommodate modeling studies and qualitative evidence, and the research within the Non-Pharmaceutical Interventions Capability includes a considerable number of such studies.

Additionally, the committee considered characteristics of the PHEPR practices within these four Capabilities that might differ in ways important to interpreting the evidence. For example, the Emergency Operations Coordination Capability is inward facing, with the aim of supporting the ongoing response effort, while the Non-Pharmaceutical Interventions Capability is outward facing, with the aim of helping the public. Likewise, practices within Information Sharing may need to be implemented more quickly than those within Community Preparedness in order to be effective. These characteristics, defined by the committee as classification dimensions (listed in Figure 3-1 in Chapter 3) were examined for each PHEPR Capability to aid in the selection of practices that were diverse with respect to those characteristics.

Developing the Analytic Frameworks and Key Review Questions

Once the four review topics had been selected, the committee focused on further describing and constraining the scope of its reviews through the development of a detailed analytic framework and set of key review questions for each topic. The analytic frameworks and key review questions for each of the four review topics can be found in Chapters 47 and Appendixes B1B4.

Analytic Frameworks

The purpose of developing analytic frameworks is to present clearly in visual format the causal pathway and interactions between a practice and its components, populations, and outcomes of interest (i.e., a logic model). As described in Chapter 3, the analytic frameworks also facilitated a mixed-method approach to the committee’s systematic reviews by serving as a construct that enabled integration of the findings from separate syntheses of quantitative, qualitative, and case report and AAR evidence.

The committee developed analytic frameworks using its members’ experiential knowledge, feedback from the PHEPR practitioner consultants, and the available research. The primary results of this process are focused on the identification of the outcomes of a practice, its postulated harms, and key factors to examine for potential effect modification and applicability of the results to other contexts (e.g., populations, settings).

For the purposes of this review, the committee adapted The Community Guide approach for developing analytic frameworks (Briss et al., 2000). The committee used the analytic frameworks for the four practices to conceptualize their relationship to outcomes of interest. In addition to health outcomes (e.g., reduced morbidity and mortality), other outcomes of interest included intermediate outcomes (e.g., knowledge, behaviors), as well as system- and process-level outcomes (e.g., accelerated recovery and cohesive and effective operations,

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

respectively). These frameworks were not simple, linear models, and represented both direct pathways to outcomes (e.g., a study that directly assessed the impact of an intervention on the outcomes of interest), and indirect pathways (e.g., the intermediate steps or intermediate outcomes that led to an effect on the final outcomes of interest).

The committee had to make explicit judgments about the extent to which PHEPR practices are grouped together and considered in the same body of evidence because PHEPR practices are carried out in different settings and populations, and often implemented in different ways. If practices are defined very narrowly, there may not be sufficient evidence to evaluate effectiveness. However, when practices are grouped together, the available evidence may be heterogeneous. While that possibility poses a different kind of challenge for analyzing the evidence, it also makes it possible to assess generalizability and consistency across different contexts. Thus, the committee discussed the degree to which the grouping of PHEPR practices was needed to achieve a balance between sufficiency and heterogeneity of evidence. Similarly, the committee had to make judgments about how to define outcomes for its evidence synthesis, given the significant variation in the measures used in the studies included in its review. For example, rather than evaluating preparedness behaviors related to stockpiling supplies and developing a family communication plan separately, these outcome measures were grouped together into an outcome category of preparedness behavior.

Key Review Questions

As the committee generated an initial list of review questions for each review topic, it became clear that beyond questions about a practice’s effectiveness (i.e., what works), it was also important to consider questions about how and why it works, for whom, and under what circumstances. The initial question lists were then reviewed and prioritized in collaboration with the PHEPR practitioner consultants. The committee drafted an overarching question for each of the four topics, which was further broken down into several sub-questions that were addressed in the review as well (see Chapters 47 and Appendixes B1B4). These sets of key review questions specified the logic and scope of the review of each topic and were critical in guiding the literature searches, data extraction, and evidence analyses.

The committee then applied the PICOTS (population, intervention/phenomena of interest, comparators, outcomes, timing, and setting) question formulation framework (Butler et al., 2017). The committee added timing and setting to the traditional PICO in an effort to capture important contextual factors. The PICOTS framework helped specify the committee’s search parameters and define the inclusion and exclusion criteria.

Throughout the process of framing questions, the committee returned to the issues of complexity and the need to answer questions that go beyond the effectiveness of a practice. These questions also relate to practitioners’ needs for recommendations that are practicable and evidence that can help them make informed decisions about response activities.

Topic Refinement: Iterative Process

When addressing such complex topics, finalizing analytic frameworks and key review questions a priori is often not a suitable approach. Therefore, the committee refined the analytic frameworks and key review questions iteratively as it explored the evidence and engaged with stakeholders.

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

LITERATURE SEARCH

Multicomponent Approach to Searching the Evidence

Recognizing that evaluations of PHEPR practices may be published not only as intervention research studies but also as organizational reports, white papers, or program evaluations, including AARs, the committee adopted a multicomponent approach to searching the evidence. This approach included searching bibliographic databases and gray literature sources, as well as issuing a call for reports.

A professional librarian worked closely with the committee to plan the literature search strategies, ensure the appropriate translation of the key review questions into relevant and accurate terms, and conduct the searches so as to identify relevant research. Specific details regarding the search strategies and article selection process for each of the four review topics can be found at the end of this appendix.

Bibliographic Database Search

The committee conducted a series of searches in four databases—PubMed, Scopus, Medline (Ovid), and Embase (Ovid)—between December 2017 and January 2019 to identify peer-reviewed literature for the four selected PHEPR practices. The committee applied a date limit of 2001 to the present, limited the literature to the English language, and excluded editorials from the search results. The first search, conducted in December 2017, captured 14 of the 15 CDC PHEPR Capabilities (excluding Community Recovery, which is out of scope) and leveraged search strategies from existing scoping reviews. The searches conducted in July 2018 and December 2018 (and updated in June 2019) focused on the selected four review topics. In January 2019, the committee conducted two expanded searches on Information Sharing and Emergency Operations Coordination in subject-specific databases. The complete search syntax and search terms for each topic can be found at the end of this appendix.

Gray Literature Search

The committee identified gray literature published by relevant domestic and international organizations and agencies. These entities included the Association of Public Health Laboratories, ASPR, ASTHO, CDC, the Center for Health Security, the Council of State and Territorial Epidemiologists, the European Centre for Disease Prevention and Control, the Disaster Information Management Research Center at the National Library of Medicine in the National Institutes of Health (NIH), the U.S. Department of Homeland Security, FEMA, the U.S. Government Accountability Office, NACCHO, the National Center for Disaster Medicine and Public Health (NCDMPH), Preparedness and Emergency Response Research Centers (PERRCs), Public Health Canada, Public Health England, RAND Corporation, and the World Health Organization. In addition, the committee obtained 370 AARs published from 2009 to 2019 from the Homeland Security Digital Library.6

Call for Reports

In addition to online searching, the committee proactively solicited reports, both published and unpublished, through a request for documents via internal listservs at the National Academies and external mechanisms. An online request was published on the webpage for

___________________

6 See https://www.hsdl.org/c (accessed May 26, 2020).

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

this study,7 and the National Academies’ Board on Health Sciences Policy distributed the call for reports through the Forum on Medical and Public Health Preparedness for Disasters and Emergencies and the Disaster Science Action Collaborative. Staff contacted CDC, the study sponsor, for document suggestions, and also asked the agency to disseminate the announcement to its networks, particularly the former PERRC and Preparedness and Emergency Response Learning Centers networks. Additionally, staff sent targeted emails to PHEPR practitioner associations (e.g., NACCHO and ASTHO) and disaster science organizations (e.g., NIH’s Disaster Research Response system, NCDMPH, and the Association of Schools and Programs of Public Health). Submissions were accepted through March 8, 2019. This proved to be an effective way of collecting AARs, theses, and white papers.

Article Selection Method

Selecting which articles to include in the committee’s reviews was a multistep process that involved developing inclusion and exclusion criteria, conducting an initial screening of titles and abstracts, and retrieving and reviewing selected full-text articles. Deciding which articles were relevant to the analytic frameworks and key review questions required significant judgment and thorough documentation. Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowcharts for each review topic can be found in Chapters 47.

Inclusion and Exclusion Criteria

The committee developed specific inclusion and exclusion criteria based on the PICOTS framework for each of the four review topics. Generally, the committee did not exclude based on study design, lack of comparison groups, or lack of explicit outcomes (i.e., it included articles that describe lessons learned or present conclusions). Articles were excluded if they were editorials, opinion pieces, or commentaries with no indication of empirical evidence. A common set of inclusion and exclusion criteria was used for screening articles (see Box A-3).

Initial Screening and Full-Text Review

One staff member conservatively screened titles and abstracts for relevance to the four topic areas. Additional articles were identified for inclusion in the review through reference mining.

The next step was to review the full-text articles that had passed the first screen. Prior to that review, the process was pilot tested on a sample of articles, with screeners participating in a calibration training call. Two individuals, one committee member and one staff, then worked independently to review all selected full-text articles against the inclusion and exclusion criteria (see Box A-3). Discrepant articles were adjudicated primarily by the committee chair and in some instances by staff, and committee members were given the opportunity to review and object if necessary to the decisions made during the adjudication process. Articles on studies that used a clear research method were categorized as Tier 1, and articles that did not, including case reports, as Tier 2; both tiers were selected for extraction of key findings. Articles that were excluded but contained information that could provide background information for the review were categorized as background and kept for potential use at a later date. The committee used EndNote to manage its references and maintained in real time a detailed account of study selections and decisions.

___________________

7 See https://www.nationalacademies.org/our-work/evidence-based-practices-for-public-health-emergencypreparedness-and-response-assessment-of-and-recommendations-for-the-field (accessed May 26, 2020).

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

DATA EXTRACTION AND STUDY QUALITY ASSESSMENT

Studies captured in the literature searches for each review topic were placed in six categories:

  • quantitative comparative studies,
  • quantitative noncomparative (single-group) studies of specific interventions,
  • surveys (descriptive only),
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
  • simulation (and related) models,
  • qualitative research studies, and
  • AARs and case reports.

Mixed-method studies (having both quantitative and qualitative components) were included in both the quantitative and qualitative study categories, as appropriate. Data extraction and the quality assessment process were tailored as necessary to these categories of study types.

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

Data extraction and quality assessment for individual quantitative studies were performed by the Center for Evidence Synthesis in Health at Brown University, an evidence-based practice center (EPC) that conducts reviews for the Agency for Healthcare Research and Quality and others. Data extraction and quality assessment for the qualitative studies were performed by a commissioned team at Wayne State University. Prioritization of case reports and AARs for inclusion in the systematic reviews and data extraction for these reports were conducted by a PHEPR expert in evaluation at Columbia University. The evaluation and extraction of findings from selected modeling studies were performed by a modeling expert at Stanford University, as described further in Chapter 3.

Data Extraction

Data extracted from each study included the description of the practice being reviewed, elements needed to make determinations about the effect of the practice, and contextual

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

elements that would contribute to an assessment of the applicability of its results to other contexts (e.g., populations, settings). Extracted data elements for quantitative studies are listed in Box A-4.

Qualitative studies, AARs, and case reports were coded in accordance with the framework synthesis method (Barnett-Page and Thomas, 2009; Pope et al., 2000), as described in Chapter 3. The codebook for extracting study characteristics and findings was developed in consultation with the committee and National Academies staff. Training sessions for the use of the codebook were conducted with the research team, and a pilot test of the codebook portion on extracting study characteristics and findings was conducted to refine the process.

Individual Study Quality Assessment

The quality assessment methodology was determined based on study design. Many standardized tools are available for assessing quality or risk of bias (RoB), each with its own

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

merits and shortcomings, and new tools continue to be developed. Described here is the approach taken by the committee and the groups commissioned to assess study quality and RoB; however, different tools and methods could reasonably be applied in future PHEPR evidence reviews.

Quantitative Studies

For quantitative comparative studies, an assessment tool was developed by the Brown University EPC8 by drawing selected RoB domains from existing tools, including the Cochrane Risk of Bias version 2.0 tool (Higgins et al., 2019), Cochrane’s suggested risk-of-bias criteria for Effective Practice and Organisation of Care reviews (Cochrane, 2017), and the Cochrane Risk of Bias in Non-Randomized Studies of Interventions (ROBINS-I) tool (Sterne et al., 2016). The selection of RoB domains reflected the dual goals of adequately addressing important potential methodological concerns and being mindful of the resources and time available for assessing the methodological quality of studies with a wide range of potential study designs. The final set of domains and their assessment criteria are as follows:

  • Study population (eligibility criteria): Was the included sample prespecified, clearly specified, defined, and uniformly applied? Low RoB if yes; High RoB if no. This domain is consistent across outcomes.
  • Allocation concealment (and randomization method): For RCTs, was there a problem with the randomization method or allocation concealment? High RoB if yes; Low RoB if explicitly no problem; Unclear RoB if insufficient reporting to judge. For nonrandomized comparative studies (NRCSs) of different interventions, High RoB unless analytic methods were used to account adequately for inherent baseline differences in compared groups or if it is otherwise reasonable to assume that compared groups are sufficiently similar. If pre-post study (of a single group), then “None.” This domain is consistent across outcomes.
  • Comparator group: Was the comparator group chosen from the same population, with the same general eligibility criteria, as the intervention group? For RCTs, Low RoB. For NRCSs, there is overlap between this assessment and the assessment of “Allocation” (see above). If pre-post study (of a single group), Low RoB (unless there is an indication that groups differed pre- and postintervention). This domain is consistent across outcomes.
  • Sample size: Is there a justification of the sample size or power/analysis per outcome? High RoB if no; Low RoB if yes (and the sample size was reached) or if the analysis was statistically significant. This domain may differ for each outcome.
  • Loss to follow-up: Was there high loss to follow-up, arbitrarily set at 20 percent, or was there unequal loss to follow-up between groups? This criterion is largely based on comparisons between enrolled (or randomized) individuals and the numbers analyzed. High RoB if yes; Low RoB if no. This domain may differ for each outcome.
  • Outcome measurement or ascertainment bias: Was there a problem with how each outcome was measured? High RoB if unvalidated subjective outcome. For studies comparing different interventions, includes whether an outcome was measured differently in the different intervention groups. This domain may differ for each outcome.
  • Group similarity at baseline: Were the groups (intervention and comparator) similar at baseline? If similar, Low RoB. If there was a (nonminor) difference, was the difference

___________________

8 See Appendix C.

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
  • statistically accounted for for each outcome? Judgment of whether a difference was “nonminor” depended on both statistical and clinical significance. Unclear RoB only if baseline descriptions were omitted or were too sparse to evaluate for possible differences. If pre-post study (of a single group), Low RoB (unless there is an indication that groups differed pre- and postintervention). This domain may differ for each outcome (based primarily on whether adequate statistical adjustment was conducted).
  • Outcome assessor blinding: Regardless of study design, was the outcome assessor blinded, or were methods used to minimize biased outcome assessment? “Hard” outcomes (unambiguous, potentially like death) or outcomes based on objective measurements (e.g., laboratory measurements or government records, such as number quarantined) generally qualified as Low RoB, as did outcomes that were explicitly blinded. Other outcomes from observational studies were assumed to have High RoB unless otherwise indicated. Self-reported outcomes were typically High RoB unless the participants were blinded to their intervention. This domain may differ for each outcome.
  • Group differences and confounders: Did the analyses account for potential group differences or confounders, for example, by multivariable adjustment or propensity score analysis? For RCTs, Low RoB was assumed absent a suggestion of a lack of similarity between groups (despite randomization). For NRCSs, regardless of whether groups were similar at baseline, High RoB if there was no adjustment for potential differences or if adjustment was made only for something minor or insufficient (e.g., only sex across disparate populations). For pre-post studies, Low RoB (unless there was an indication that groups differed pre- and postintervention). This domain may differ for each outcome.
  • Other important limitations per data extractor or as reported by study authors. This domain may differ for each outcome.

Each outcome of each quantitative comparative study was evaluated for all of the above domains. Then an overall assessment of the study (or outcome) methodology (rated as good, moderate, or poor) was made based on the judgment of the evaluators after considering the various bias domains, which were weighted differently for different study designs. Each study (and outcome) was assessed for methodological quality by the Brown team’s senior researcher and was reviewed by at least one other experienced team member and was altered in discussion as needed.

The Brown University EPC developed and applied a separate tool for the assessment of descriptive surveys, drawing on published methods (Bennett et al., 2010; Davids and Roman, 2014). Descriptive surveys were assessed using the following domains and assessment criteria:

  • Adequacy of survey tool development: Low RoB: A priori methodology with group development and pretesting reported that survey has been validated and/or found reliable. High RoB: Lack of structured methodology for developing questions, single person or group developed, and/or no outside input or pilot, field, or pretesting of questions (or prior use). Unclear RoB: No or incomplete description of development process.
  • Study population eligibility criteria prespecified and uniformly applied: Low RoB: Explicitly reported, clear, and no major deviations from protocol. High RoB: Not prespecified or major deviation from protocol. Unclear RoB: Not reported whether prespecified or deviation.
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
  • Adequacy and appropriateness of polling/sampling methodology: Low RoB: Everyone who met criteria (universe, census); probability sampling (e.g., random selection of telephone, email, and text of population with high access to these technologies); other unbiased sampling of population of interest. High RoB: Problems such as that sampling was likely biased (e.g., texting may miss individuals of low socioeconomic status or those difficult to reach), nonprobability sample (e.g., for focus group, convenience sample); if sample of general population, no attempt to capture difficult-to-reach individuals (e.g., those with no phone, email). Unclear RoB: Not adequately described.
  • Respondents nonrepresentative of the target population: Low RoB: Respondents representative of target population and not different from nonrespondents. High RoB: Explicitly nonrepresentative; respondents differed from nonrespondents or target population. Unclear RoB: No description of target population or nonrespondents (and not High RoB).
  • Percentage who responded: The actual response rate, without a judgment of its adequacy.
  • Information on margin of error reported: Low RoB: If margin-of-error calculations made and reported, the reported values were extracted. Unclear RoB: No information on margin-of-error calculations. (While margin of error is a concept related to precision and not bias, the same terminology [High, Low, Unclear] was used for clarity and consistency.)

Qualitative Studies

Quality assessment for qualitative studies was undertaken as a component of a qualitative evidence synthesis commissioned to Wayne State University. The qualitative studies meeting the inclusion criteria for each review topic were appraised individually using the Critical Appraisal Skills Programme (CASP, 2018) checklist, which is applicable to assessing qualitative research. Areas of appraisal by CASP include appropriateness of qualitative methodology, data collection, relationship between research and participants, ethics, rigor of data analysis, clarity of findings, and value of research. Each area was assessed using “yes,” “no,” or “can’t tell.” In line with the Grading of Recommendations Assessment, Development and Evaluation Confidence in the Evidence from Reviews of Qualitative Research (GRADE-CERQual) approach for assessing methodological limitations (Lewin et al., 2018), each study received a final overall quality rating of “no or very minor concerns” (no significant methodological limitations), “minor concerns” (minor methodological limitations not impacting the credibility and validity of findings), “moderate concerns” (some methodological limitations likely to impact the credibility and validity of findings), or “serious concerns” (serious methodological limitation impacting the credibility and validity of findings).

After Action Reports

An appraisal tool for evaluating the methodological rigor of AARs was published in 2018 (ECDC, 2018) and was applied by a commissioned PHEPR expert to the AARs included in the committee’s analyses. The 11-item tool is designed to assist with the systematic documentation of methods used in AARs, compare validity, and potentially inform best practices for a standard template. The tool includes the following criteria:

  • prolonged engagement with the subject of inquiry,
  • use of theory,
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
  • data selection,
  • information sampling,
  • multiple data sources,
  • triangulation,
  • negative case analysis,
  • peer debriefing and support,
  • respondent validation,
  • clear report of methods of data collection and analysis (audit trail), and
  • depth and insight.

PRIORITIZATION OF FUTURE SYSTEMATIC EVIDENCE REVIEW TOPICS

The committee engaged with a second diverse group of 10 PHEPR practitioners, also representing SLTT agencies, in an open session to identify knowledge gaps that matter to practitioners and assess the relative priority, from their perspective, of potential evidence review topics encompassed within the CDC PHEPR Capabilities. This practitioner engagement activity was conducted after the committee’s four evidence review topics had been selected and therefore did not inform the selection process. The activity was intended to inform the identification of priorities for future PHEPR evidence reviews, as well as future research needs, to address critical knowledge gaps in PHEPR practice. For this prioritization activity, the committee modified an existing group judgment process originally developed to assess the appropriateness of medical procedures, but since modified for many other uses (i.e., modified Delphi-like process) (Shekelle, 2004; Sloss et al., 2000). This process involved the selection of this group of SLTT PHEPR practitioners, a review of the literature, and multiple rounds of group voting with feedback between rounds. This process was led by an experienced moderator.

This second group of PHEPR practitioners was identified through a process similar to that used to select the PHEPR practitioner consultants. The list of PHEPR topics included in this activity was adapted from an existing list of 44 research questions deemed by PHEPR practitioners to be priority areas (Siegfried et al., 2017). Building from that list, the committee combined several research questions into one topic area, removed several research questions that did not lend themselves to a review of the evidence, and added several questions that the committee derived from a review of the literature. This process resulted in 39 topic areas across the six PHEPR domains identified by Siegfried and colleagues: community resilience, incident management, information management, countermeasures and mitigation, surge management, and biosurveillance.

In January 2019, the committee engaged with these PHEPR practitioners in an open session in a virtual premeeting for the first round of voting and at an in-person meeting for the second round of voting. All 10 PHEPR practitioners participated in both rounds. The committee used Sli. Do, an online polling software, to conduct this activity.

The PHEPR practitioners were asked to rank the importance of the 39 topic areas on the committee’s list by rating each on a 5-point Likert rating scale—highest priority, high priority, moderate priority, low priority, and not a priority—as well as to provide any other comments or suggestions. They were provided with the following guidance with regard to rating the priority of topics:

These ratings should reflect the need for a systematic review of the evidence, not necessarily the importance of the practice. All of the topics are certainly important, but rating everything as the highest priority will not be helpful. Furthermore, the evidence for some practices with

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

very high importance to PHEPR may be well established, and a review of the evidence for those topics may be less valuable than a review of other topics for which there are important knowledge gaps.

Results from the first round (virtual premeeting) were compiled for the second round (in-person meeting), during which the PHEPR practitioners received feedback on the results from the first round, discussed those results, and revoted if doing so was warranted. The outcome of this process was a set of PHEPR review topics classified into the five Likert scale categories: highest priority, high priority, moderate priority, low priority, and not a priority (see Box A-5). This information, along with the published literature, aided the committee in identifying priorities for future evidence reviews, as well as future research needs, to address critical knowledge gaps in PHEPR practice.

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

DATABASE SEARCH STRATEGIES

Initial searches were run on December 3, 2018, covering 2001–December 2018. Final update searches were performed on June 5, 2019, covering December 2018–June 2019.

Community Preparedness

Search Parameters:

Date: 2001–Present

Language: English

Document Type: Exclude commentaries, editorials, letters, and notes

Databases: Medline (Ovid), Embase (Ovid), Scopus

Search Syntax:

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

Medline (Ovid):

Search No. Syntax Results
1 ((natural adj (disaster? or hazard?)) or (hurricane? or flood$ or typhoon? or earthquake$ or fire? or cyclon$ or heatwave? or freezing or ((ice or snow or lightning) adj storm?) or blizzard? or “heat wave” or (extreme adj (temperature? or heat or cold)) or tsunami? or “tidal wave”)).tw. 75,331
2 (“mass migration*” or SARS or ebola or smallpox or plague or measles).tw. 46,567
3 (firesetting or arson or explosion? or bomb$ or outbreak? or refugee* or (explo$ adj device?) or blackout? or brownout? or ((power or equipment) adj (loss or failure)) or radioactive or radiation or (nuclear adj (disaster or meltdown or catastrophe or fail$))).tw. 427,623
4 (((chemical or biological) adj warfare) or riot$ or influenza or flu or (civil adj (disorder? or defense or unrest))).tw. 89,156
5 cyclonic storms/ or droughts/ or floods/ or tidal waves/ or tsunami/ or snow/ or rain/ or avalanches/ or volcanic eruptions/ or earthquakes/ or landslides/ or fires/ or ice/ or tornadoes/ or extreme cold/ or extreme heat/ or lightning/ or cold temperature/ or hot temperature/ or wind/ or firesetting behavior/ or explosions/ or blackout/ or equipment failure/ or radioactive hazard release/ or influenza/ or refugees/ or riots/ or civil disorders/ or civil defense/ or communicable diseases/ or Severe Acute Respiratory Syndrome/ or SARS VIRUS/ or HEMORRHAGIC FEVER, EBOLA/ or MEASLES/ or SMALLPOX/ or PLAGUE/ 341,253
6 or/1–5 859,584
7 ((“public health” adj (practice or administration or incident* or emergenc*)) or “preventive medicine”).tw. or “public health”.ti,ab. 171,417
8 public health/ or preventive medicine/ or public health practice/ or public health administration/ 101,802
9 “health department*”.tw. 8,009
10 ((public or local or state or tribal or territorial or multi) adj health adj (department* or agenc*or jurisdiction)).ti,ab. 3,961
11 or/7–10 232,661
12 6 and 11 21,249
13 (epidemic? or pandemic? or terroris$ or bioterroris$ or “mass casualt*”).tw. 105,665
14 epidemics/ or pandemics/ or disease outbreaks/ or disasters/ or emergencies/ or mass casualty incidents/ or terrorism/ or bioterrorism/ or chemical terrorism/ or “september 11 terrorist attacks”/ 152,746
15 ((emergency adj (prepare* or response* or management or incident*)) or (disaster adj (plan$ or prepare* or mitigation or recover* or cycle or medicine or resilienc* or incident* or response* or management* or readiness))).tw. 8,725
16 emergency preparedness/ or emergency response/ or emergency management/ or disaster planning/ or disaster medicine/ or disaster resilience/ 15,465
17 or/13–16 230,361
18 12 or 17 241,051
19 Vulnerable Populations/ or Minority Groups/ or “Sexual and Gender Minorities”/ or “Emigrants and Immigrants”/ or “Transients and Migrants”/ or African Americans/ or Hispanic Americans/ or Indians, North American/ or Asian Americans/ or Poverty/ or Socioeconomic Factors/ or “Aged, 80 and over”/ or Aged/ or INFANT/ or INFANT, NEWBORN/ 4,051,798
20 Pregnant Women/ or DIABETES MELLITUS/ or Heart Diseases/ or DEAF-BLIND DISORDERS/ or BLINDNESS/ or Deafness/ or Persons With Hearing Impairments/ or Vision Disorders/ or Disabled Persons/ or Animal Assisted Therapy/ or Self-Help Devices/ or Autistic Disorder/ 312,323
21 DEMENTIA/ or ANXIETY/ or ANXIETY DISORDERS/ or “Transportation of Patients”/ or Homeless Persons/ or Rural Population/ or ELECTRICITY/ or CHILD/ or Health Literacy/ or Educational Status/ or Communication Barriers/ 1,835,230
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Search No. Syntax Results
22 Alcoholism/ or Substance-Related Disorders/ or Mental Disorders/ or Stress Disorders, Post-Traumatic/ or Depressive Disorder/ or SCHIZOPHRENIA/ or Psychotic Disorders/ 486,844
23 (vulnerable adj (population* or person* or individual*)).tw. 7,372
24 (“at risk” adj (individual* or population* or person*)).tw. 4,905
25 “functional need*”.tw. 327
26 vulnerability.tw. 40,340
27 (elderly or disabled).tw. 216,777
28 ((racial or ethnic or sexual or gender) adj minorit*).tw. 10,036
29 (“older adult*” or pediatric* or child*).tw. 1,288,450
30 (limited adj (english or language)).tw. 801
31 (migrant* or “low income” or “under resourced” or tribal or “dialysis patient*” or “electricity dependent” or “medically vulnerable”).tw. 56,111
32 (“pregnant wom*n” or “pre existing condition*” or “chronic condition*” or diabetes or “heart disease*” or blind* or deaf* or “assistive device*” or “service animal*” or “personal assistance service provider*” or autism or dementia or anxiety or “transportation need*” or homeless* or “lift equipped” or “oxygen tank*”).tw. 1,108,388
33 “language barrier*”.tw. 1,435
34 ((low or limited) adj “health literacy”).tw. 894
35 (“mental health disorder*” or “group home patient*” or “substance abuse disorder*”).tw. 2,989
36 (alcoholism or “substance disorder*” or “mental disorder*” or “stress disorder*” or PTSD or depression or “depressive disorder*” or schizophrenia or “psychotic disorder*”).tw. 405,450
37 or/19–36 6,603,197
38 Community Health Planning/ or Community-Institutional Relations/ or Community Participation/ or Community Health Services/ or Community Networks/ or Community-Based Participatory Research/ 66,327
39 Home Care Services/ or Hospices/ or Home Care Services/ or Hospice Care/ or Assisted Living Facilities/ or Skilled Nursing Facilities/ or Home Nursing/ or Local Government/ 55,665
40 Emergency Shelter/ or Voluntary Health Agencies/ or Red Cross/ 6,403
41 (community adj4 (engagement or outreach or preparedness or partner* or partnership* or train* or toolkit or education or plan*)).tw. 20,161
42 (community adj2 (stakeholder* or spokesperson* or spokespeople or gathering* or venue*)).tw. 848
43 “town hall meeting*”.tw. 58
44 “faith based organization*”.tw. 213
45 (“animal service agenc*” or “childcare organization*” or “chronic disease program*” or “communicable disease program*” or “community coalition*” or “emergency management agenc*” or “emergency medical service*” or “environmental health agenc*” or “fire department*” or “fire and rescue” or “health care coalition*” or “health care organization*” or “health care system*” or “health care provider*” or “infection control program*” or “housing authorit*” or “shelter* authorit*” or “human service provider*” or “immunization program*” or “jurisdictional strategic advisory council*” or “law enforcement” or “media organization*” or “local media” or “metal health provider*” or “behavioral health provider*” or “public health preparedness program*” or “school agenc*” or “education agenc*” or “social service*” or “state office of aging” or “surveillance program*” or “volunteer organization*”).tw. 88,794
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Search No. Syntax Results
46 “community based participatory research”.tw. 1,707
47 (“C MIST” or “communication medical independence supervision transportation”). tw. 0
48 (sovi or “social vulnerability index”).tw. 50
49 (“at risk” adj (database* or registr* or map*)).tw. 3
50 “social capital”.tw. 2,396
51 ((medicare or medicaid or “health measure*”) adj data).tw. 1,375
52 “empower initiative”.tw. 0
53 (“emergency shelter*” or NGO* or “red cross” or “local business*”).tw. 6,186
54 (“home care” or “hospice care” or hospice* or “home hospice” or “assisted living facilit*” or “skilled care facilit*”).tw. 25,222
55 or/38–54 239,867
56 18 and 37 and 55 4,047
57 limit 56 to (english language and yr=”2001–Current”) 2,857
58 limit 57 to (comment or editorial or letter) 38
59 57 not 58 2,819

Embase (Ovid):

Search No. Syntax Results
1 ((natural adj (disaster? or hazard?)) or (hurricane? or flood$ or typhoon? or earthquake$ or fire? or cyclon$ or heatwave? or freezing or ((ice or snow or lightning) adj storm?) or blizzard? or “heat wave” or (extreme adj (temperature? or heat or cold)) or tsunami? or “tidal wave”)).tw. 87,057
2 (“mass migration*” or SARS or ebola or smallpox or plague or measles).tw. 38,502
3 (firesetting or arson or explosion? or bomb$ or outbreak? or refugee* or (explo$ adj device?) or blackout? or brownout? or ((power or equipment) adj (loss or failure)) or radioactive or radiation or (nuclear adj (disaster or meltdown or catastrophe or fail$))).tw. 468,844
4 (((chemical or biological) adj warfare) or riot$ or influenza or flu or (civil adj (disorder? or defense or unrest))).tw. 93,578
5 hurricane/ or drought/ or flooding/ or tsunami/ or snow/ or rain/ or ice/ or avalanche/ or volcano/ or earthquake/ or landslide/ or fire/ or tornado/ or cold/ or heat/ or lightning/ or wind/ or arson/ or explosion/ or device failure/ or nuclear accident/ or civil disorder/ or influenza/ or communicable disease/ or severe acute respiratory syndrome/ or Ebola hemorrhagic fever/ or measles/ or smallpox/ or Smallpox virus/ or plague/ or refugee/ 228,653
6 or/1–5 800,537
7 ((“public health” adj (practice or administration or incident* or emergenc*)) or “preventive medicine”).tw. or “public health”.ti,ab. 213,782
8 “health department*”.tw. 7,711
9 ((public or local or state or tribal or territorial or multi) adj health adj (department* or agenc*or jurisdiction)).ti,ab. 3,708
10 public health/ or preventive medicine/ or public health service/ 191,406
11 or/7–10 330,666
12 6 and 11 30,320
13 (epidemic? or pandemic? or terroris$ or bioterroris$ or “mass casualt*”).tw. 114,929
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Search No. Syntax Results
14 ((emergency adj (prepare* or response* or management or incident*)) or (disaster adj (plan$ or prepare* or mitigation or recover* or cycle or medicine or resilienc* or incident* or response* or management* or readiness))).tw. 10,779
15 epidemic/ or pandemic/ or disaster/ or mass disaster/ or nuclear terrorism/ or terrorism/ or chemical terrorism/ or bioterrorism/ or disaster planning/ or disaster planning/ or disaster medicine/ 121,985
16 or/13–15 191,448
17 12 or 16 208,880
18 vulnerable population/ or minority group/ or “sexual and gender minority”/ or undocumented immigrant/ or immigrant/ or migrant/ or African American/ or Hispanic/ or American Indian/ or Asian American/ or poverty/ or socioeconomics/ or very elderly/ or infant/ or newborn/ 995,819
19 pregnant woman/ or diabetes mellitus/ or heart disease/ or deafblindness/ or blindness/ or hearing impairment/ or visual disorder/ or disabled person/ or animal assisted therapy/ or self help device/ or autism/ or mental deficiency/ 728,645
20 dementia/ or anxiety/ or anxiety disorder/ or patient transport/ or homeless person/ or homeless man/ or homeless youth/ or homeless woman/ or rural population/ or electricity/ or health literacy/ or educational status/ or communication barrier/ 430,145
21 alcoholism/ or substance abuse/ or mental disease/ or alcoholism/ or drug abuse/ or drug dependence/ or addiction/ or schizophrenia/ or psychosis/ 468,019
22 (vulnerable adj (population* or person* or individual*)).tw. 11,533
23 (“at risk” adj (individual* or population* or person*)).tw. 7,977
24 “functional need*”.tw. 400
25 vulnerability.tw. 54,524
26 (elderly or disabled).tw. 267,455
27 ((racial or ethnic or sexual or gender) adj minorit*).tw. 13,879
28 (“older adult*” or pediatric* or child*).tw. 1,422,928
29 (limited adj (english or language)).tw. 1,192
30 (migrant* or “low income” or “under resourced” or tribal or “dialysis patient*” or “electricity dependent” or “medically vulnerable”).tw. 68,197
31 (“pregnant wom*n” or “pre existing condition*” or “chronic condition*” or diabetes or “heart disease*” or blind* or deaf* or “assistive device*” or “service animal*” or “personal assistance service provider*” or autism or dementia or anxiety or “transportation need*” or homeless* or “lift equipped” or “oxygen tank*”).tw. 1,513,411
32 “language barrier*”.tw. 2,429
33 ((low or limited) adj “health literacy”).tw. 1,636
34 (“mental health disorder*” or “group home patient*” or “substance abuse disorder*”).tw. 5,015
35 (alcoholism or “substance disorder*” or “mental disorder*” or “stress disorder*” or PTSD or depression or “depressive disorder*” or schizophrenia or “psychotic disorder*”).tw. 518,753
36 or/18–35 4,505,232
37 community care/ or community program/ or community/ or emergency shelter/ or red cross/ or home care/ or hospice/ or hospice care/ or home for the aged/ or nursing home/ or assisted living facility/ 210,785
38 (community adj4 (engagement or outreach or preparedness or partner* or partnership* or train* or toolkit or education or plan*)).tw. 25,460
39 (community adj2 (stakeholder* or spokesperson* or spokespeople or gathering* or venue*)).tw. 1,371
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Search No. Syntax Results
40 “town hall meeting*”.tw. 81
41 “faith based organization*”.tw. 307
42 (“animal service agenc*” or “childcare organization*” or “chronic disease program*” or “communicable disease program*” or “community coalition*” or “emergency management agenc*” or “emergency medical service*” or “environmental health agenc*” or “fire department*” or “fire and rescue” or “health care coalition*” or “health care organization*” or “health care system*” or “health care provider*” or “infection control program*” or “housing authorit*” or “shelter* authorit*” or “human service provider*” or “immunization program*” or “jurisdictional strategic advisory council*” or “law enforcement” or “media organization*” or “local media” or “metal health provider*” or “behavioral health provider*” or “public health preparedness program*” or “school agenc*” or “education agenc*” or “social service*” or “state office of aging” or “surveillance program*” or “volunteer organization*”).tw. 112,450
43 “community based participatory research”.tw. 2,259
44 (“C MIST” or “communication medical independence supervision transportation”). tw. 0
45 (sovi or “social vulnerability index”).tw. 67
46 (“at risk” adj (database* or registr* or map*)).tw. 7
47 “social capital”.tw. 3,029
48 ((medicare or medicaid or “health measure*”) adj data).tw. 2,567
49 “empower initiative”.tw. 0
50 (“emergency shelter*” or NGO* or “red cross” or “local business*”).tw. 8,621
51 (“home care” or “hospice care” or hospice* or “home hospice” or “assisted living facilit*” or “skilled care facilit*”).tw. 29,249
52 or/37–51 349,425
53 17 and 36 and 52 5,222
54 limit 53 to (english language and yr=”2001–Current”) 4,712
55 limit 54 to (editorial or letter or note) 187
56 54 not 55 4,525

Scopus:

TITLE-ABS-KEY(((natural W/1 (disaster* or hazard*)) or hurricane* or flood* or typhoon* or earthquake* or fire* or cyclon* or heatwave* or freezing or ((ice or snow or lightning) W/1 storm*) or blizzard* or “heat wave” or (extreme W/1 (temperature* or heat or cold)) or tsunami* or drought* or “tidal wave” or epidemic* or pandemic* or terrorism or bioterrorism or “mass casualt*” or (firesetting or arson or explosion? or bomb* or (explo* W/1 device*) or blackout* or tornado* or brownout* or ((power or equipment) W/1 (loss or failure)) or radioactive or radiation or (nuclear W/1 (disaster or meltdown or catastrophe or fail*)) or (refugee* or “mass migration*” or SARS or ebola or smallpox or plague or measles or riot* or influenza or “communicable disease*” or ((chemical or biological) W/1 warefare) or (civil W/1 (disorder* or defense or unrest)))) AND ((“public health” W/1 (practice or administration)) or “preventive medicine” or “health department” or ((public or local or state or tribal or territorial or multi) W/1 health W/1 (department* or agenc* or jurisdiction)))) OR ((emergency W/1 (preparedness or response or management)) or (disaster W/1 (plan* or preparedness or mitigation or recovery or cycle or medicine or resilience or readiness or ready))) AND (“vulnerable population*” or “minority group*” or {sexual and gender minorities} or “disabled person*” or immigrant* or emigrant* or transient* or refugee* or migrant* or “african ameri-

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

can*” or black or “hispanic American*” or hispanic or latin* OR indian* or “native american*” or “asian american*” or Asian* or poverty or “socioeconomic factor*” or (“at risk” w/1 (individual* or population*)) or “functional need*” or (vulnerable w/1 (individual* or population*)) or vulnerability or elderly or disabled or ((racial or ethnic or sexual or gender) w/1 minorit*) or “older adult” or pediatric* or child* or (limited w/1 (English or language)) or migrant* or immigrant* or “low income” or “under resourced” or tribal or “dialysis patient*” or “electricity dependent” or “medically vulnerable” or infant* or newborn* or {aged 80 and over} or “pregnant wom*n” or “pre existing condition*” or “chronic condition*” or diabetes or “heart disease*” or blind* or deaf* or “assistive device*” or “service animal*” or “personal assistance service provider*” or autism or dementia or anxiety or “transportation need*” or homeless* or “lift equipped” or “oxygen tank*” or “language barrier*” or ((low or limited) W/1 “health literacy”) or “mental health disorder*” or “group home patient*” or “substance abuse disorder*” or alcoholism or “substance disorder*” or “mental disorder*” or “stress disorder*” or PTSD or depression or “depressive disorder*” or schizophrenia or “psychotic disorder*”) AND (“community preparedness” OR “community partners” OR “community partnerships” OR “community engagement” OR “Community Health Planning” OR “Community Networks” OR “Community-Institutional Relations” or “cooperative behavior” or “risk assessment*” or “social capital” or “social support” or {community based participatory research} or (community w/2 (engagement or outreach or preparedness or partner* or partnership* or train* or toolkit or education or plan*)) or toolkit* or (“at risk” w/1 (database* or registr* or map*)) or “social capital” or CMIST OR “communication medical independence supervision transportation” or sovi or “social vulnerability index” or (“health measure*” w/1 data) or “empower initiative” or (community W/2 (stakeholder* or spokesperson* or spokespeople or gathering* or venue*)) or “town hall meeting*” or “faith based organization*” or “animal service agenc*” or “childcare organization*” or “chronic disease program*” or “communicable disease program*” or “community coalition*” or “emergency management agenc*” or “emergency medical service*” or “environmental health agenc*” or “fire department*” or {fire and rescue} or “health care coalition*” or “health care organization*” or “health care system*” or “health care provider*” or “infection control program*” or “housing authorit*” or “shelter* authorit*” or “human service provider*” or “immunization program*” or “jurisdictional strategic advisory council*” or “law enforcement” or “media organization*” or “local media” or “metal health provider*” or “behavioral health provider*” or “public health preparedness program*” or “school agenc*” or “education agenc*” or “social service*” or “state office of aging” or “surveillance program*” or “volunteer organization*” or “emergency shelter*” or NGO* or “red cross” or “local business*” or “home care” or “hospice care” or hospice* or “home hospice” or “assisted living facilit*” or “skilled care facilit*”)) AND PUBYEAR AFT 2000

Exclude: Note, editorials, letters, and notes

Results: 3,406

Emergency Operations Coordination

Search Parameters:

Date: 2001–Present

Language: English

Document Type: Exclude commentaries, editorials, letters, and notes

Databases: Medline (Ovid), Embase (Ovid), Scopus

Search Syntax:

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

Medline (Ovid):

Search No. Syntax Results
1 ((natural adj (disaster? or hazard?)) or (hurricane? or flood$ or typhoon? or earthquake$ or fire? or cyclon$ or heatwave? or freezing or ((ice or snow or lightning) adj storm?) or blizzard? or “heat wave” or (extreme adj (temperature? or heat or cold)) or tsunami? or “tidal wave”)).tw. 75,331
2 (“mass migration*” or SARS or ebola or smallpox or plague or measles).tw. 46,567
3 (firesetting or arson or explosion? or bomb$ or outbreak? or refugee* or (explo$ adj device?) or blackout? or brownout? or ((power or equipment) adj (loss or failure)) or radioactive or radiation or (nuclear adj (disaster or meltdown or catastrophe or fail$))).tw. 427,623
4 (((chemical or biological) adj warfare) or riot$ or influenza or flu or (civil adj (disorder? or defense or unrest))).tw. 89,156
5 cyclonic storms/ or droughts/ or floods/ or tidal waves/ or tsunami/ or snow/ or rain/ or avalanches/ or volcanic eruptions/ or earthquakes/ or landslides/ or fires/ or ice/ or tornadoes/ or extreme cold/ or extreme heat/ or lightning/ or cold temperature/ or hot temperature/ or wind/ or firesetting behavior/ or explosions/ or blackout/ or equipment failure/ or radioactive hazard release/ or influenza/ or refugees/ or riots/ or civil disorders/ or civil defense/ or communicable diseases/ or Severe Acute Respiratory Syndrome/ or SARS VIRUS/ or HEMORRHAGIC FEVER, EBOLA/ or MEASLES/ or SMALLPOX/ or PLAGUE/ 341,253
6 or/1–5 859,584
7 ((“public health” adj (practice or administration or incident* or emergenc*)) or “preventive medicine”).tw. or “public health”.ti,ab. 171,417
8 public health/ or preventive medicine/ or public health practice/ or public health administration/ 101,802
9 “health department*”.tw. 8,009
10 ((public or local or state or tribal or territorial or multi) adj health adj (department* or agenc*or jurisdiction)).ti,ab. 3,961
11 or/7–10 232,661
12 6 and 11 21,249
13 (epidemic? or pandemic? or terroris$ or bioterroris$ or “mass casualt*”).tw. 105,665
14 epidemics/ or pandemics/ or disease outbreaks/ or disasters/ or emergencies/ or mass casualty incidents/ or terrorism/ or bioterrorism/ or chemical terrorism/ or “september 11 terrorist attacks”/ 152,746
15 ((emergency adj (prepare* or response* or management or incident*)) or (disaster adj (plan$ or prepare* or mitigation or recover* or cycle or medicine or resilienc* or incident* or response* or management* or readiness))).tw. 8,725
16 emergency preparedness/ or emergency response/ or emergency management/ or disaster planning/ or disaster medicine/ or disaster resilience/ 15,465
17 or/13–16 230,361
18 12 or 17 241,051
19 “emergency operations coordination”.tw. 4
20 “operation* center*”.tw. 152
21 “emergency operations center*”.tw. 67
22 “public health emergency operations center*”.tw. 3
23 “national incident management system*”.tw. 35
24 “incident management”.tw. 219
25 Safety Management/ 19,133
26 “safety management”.tw. 818
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Search No. Syntax Results
27 “emergency operation*”.tw. 3,470
28 “unified command”.tw. 12
29 “incident command”.tw. 213
30 “incident management team*”.tw. 16
31 “situational awareness”.tw. 459
32 Cooperative Behavior/ 40,680
33 “management infrastructure”.tw. 101
34 “public health coordination*”.tw. 7
35 “threshold criteria”.tw. 291
36 “situational awareness”.tw. 459
37 (essential adj (function* or personnel*)).tw. 4,635
38 Public Health Practice/st [Standards] 612
39 “delineation of service*”.tw. 2
40 ((tabletop or functional or “full scale” or trigger* or activat* or hypothetical or “stand up” or assessment) adj2 (drill or mobilize or mobilization or deploy* or exercise* or scenario* or incident* or event* or plan* or procedure* or protocol* or policy or policies)).tw. 28,387
41 “ESF-8”.tw. 4
42 “incident management system*”.tw. 78
43 “public health emergency operation*”.tw. 6
44 “disaster* operation* center*”.tw. 2
45 or/19–44 97,811
46 18 and 45 3,700
47 limit 46 to (english language and yr=”2001–Current”) 2,791
48 limit 47 to (comment or editorial or letter) 116
49 47 not 48 2,675

Embase (Ovid):

Search No. Syntax Results
1 ((natural adj (disaster? or hazard?)) or (hurricane? or flood$ or typhoon? or earthquake$ or fire? or cyclon$ or heatwave? or freezing or ((ice or snow or lightning) adj storm?) or blizzard? or “heat wave” or (extreme adj (temperature? or heat or cold)) or tsunami? or “tidal wave”)).tw. 106,875
2 (“mass migration*” or SARS or ebola or smallpox or plague or measles).tw. 50,215
3 (firesetting or arson or explosion? or bomb$ or outbreak? or refugee* or (explo$ adj device?) or blackout? or brownout? or ((power or equipment) adj (loss or failure)) or radioactive or radiation or (nuclear adj (disaster or meltdown or catastrophe or fail$))).tw. 602,188
4 (((chemical or biological) adj warfare) or riot$ or influenza or flu or (civil adj (disorder? or defense or unrest))).tw. 111,443
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Search No. Syntax Results
5 hurricane/ or drought/ or flooding/ or tsunami/ or snow/ or rain/ or ice/ or avalanche/ or volcano/ or earthquake/ or landslide/ or fire/ or tornado/ or cold/ or heat/ or lightning/ or wind/ or arson/ or explosion/ or device failure/ or nuclear accident/ or civil disorder/ or influenza/ or communicable disease/ or severe acute respiratory syndrome/ or Ebola hemorrhagic fever/ or measles/ or smallpox/ or Smallpox virus/ or plague/ or refugee/ 274,496
6 or/1–5 1,010,732
7 ((“public health” adj (practice or administration or incident* or emergenc*)) or “preventive medicine”).tw. or “public health”.ti,ab. 238,311
8 “health department*”.tw. 9,525
9 ((public or local or state or tribal or territorial or multi) adj health adj (department* or agenc*or jurisdiction)).ti,ab. 4,481
10 public health/ or preventive medicine/ or public health service/ 231,842
11 or/7–10 387,664
12 6 and 11 32,828
13 (epidemic? or pandemic? or terroris$ or bioterroris$ or “mass casualt*”).tw. 132,027
14 ((emergency adj (prepare* or response* or management or incident*)) or (disaster adj (plan$ or prepare* or mitigation or recover* or cycle or medicine or resilienc* or incident* or response* or management* or readiness))).tw. 12,084
15 epidemic/ or pandemic/ or disaster/ or mass disaster/ or nuclear terrorism/ or terrorism/ or chemical terrorism/ or bioterrorism/ or disaster planning/ or disaster planning/ or disaster medicine/ 138,280
16 or/13–15 221,710
17 12 or 16 241,171
18 “emergency operations coordination”.tw. 6
19 “operation* center*”.tw. 207
20 “emergency operations center*”.tw. 78
21 “public health emergency operations center*”.tw. 3
22 “national incident management system*”.tw. 38
23 “incident management”.tw. 330
24 “safety management”.tw. 1,585
25 “emergency operation*”.tw. 4,746
26 “unified command”.tw. 19
27 “incident command”.tw. 271
28 “incident management team*”.tw. 25
29 “situational awareness”.tw. 810
30 “management infrastructure”.tw. 153
31 “public health coordination*”.tw. 7
32 “threshold criteria”.tw. 436
33 “situational awareness”.tw. 810
34 (essential adj (function* or personnel*)).tw. 5,885
35 “delineation of service*”.tw. 6
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Search No. Syntax Results
36 ((tabletop or functional or “full scale” or trigger* or activat* or hypothetical or “stand up” or assessment) adj2 (drill or mobilize or mobilization or deploy* or exercise* or scenario* or incident* or event* or plan* or procedure* or protocol* or policy or policies)).tw. 43,159
37 “ESF-8”.tw. 5
38 “incident management system*”.tw. 113
39 “public health emergency operation*”.tw. 6
40 “disaster* operation* center*”.tw. 2
41 cooperation/ 40,559
42 or/18–41 97,713
43 17 and 42 2,404
44 limit 43 to (english language and yr=”2001–Current”) 2,141
45 limit 44 to (editorial or letter or note) 156
46 44 not 45 1,985

Scopus:

TITLE-ABS-KEY(((natural W/1 (disaster* or hazard*)) or hurricane* or flood* or typhoon* or earthquake* or fire* or cyclon* or heatwave* or freezing or ((ice or snow or lightning) W/1 storm*) or blizzard* or “heat wave” or (extreme W/1 (temperature* or heat or cold)) or tsunami* or drought* or “tidal wave” or epidemic* or pandemic* or terrorism or bioterrorism or “mass casualt*” or (firesetting or arson or explosion? or bomb* or (explo* W/1 device*) or blackout* or tornado* or brownout* or ((power or equipment) W/1 (loss or failure)) or radioactive or radiation or (nuclear W/1 (disaster or meltdown or catastrophe or fail*)) or (refugee* or “mass migration*” or SARS or ebola or smallpox or plague or measles or riot* or influenza or “communicable disease*” or ((chemical or biological) W/1 warefare) or (civil W/1 (disorder* or defense or unrest)))) AND ((“public health” W/1 (practice or administration)) or “preventive medicine” or “health department” or ((public or local or state or tribal or territorial or multi) W/1 health W/1 (department* or agenc* or jurisdiction)))) OR ((emergency W/1 (preparedness or response or management)) or (disaster W/1 (plan* or preparedness or mitigation or recovery or cycle or medicine or resilience or readiness or ready))) AND PUBYEAR AFT 2000

Exclude: Editorials, letters, and notes

Results: 2,813

Information Sharing

Search Parameters:

Date: 2001–Present

Language: English

Document Type: Exclude commentaries, editorials, letters, and notes

Databases: Medline (Ovid), Embase (Ovid), Scopus

Search Syntax:

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

Medline (Ovid):

Search No. Syntax Results
1 ((natural adj (disaster? or hazard?)) or (hurricane? or flood$ or typhoon? or earthquake$ or fire? or cyclon$ or heatwave? or freezing or ((ice or snow or lightning) adj storm?) or blizzard? or “heat wave” or (extreme adj (temperature? or heat or cold)) or tsunami? or “tidal wave”)).tw. 75,331
2 (“mass migration*” or SARS or ebola or smallpox or plague or measles).tw. 46,567
3 (firesetting or arson or explosion? or bomb$ or outbreak? or refugee* or (explo$ adj device?) or blackout? or brownout? or ((power or equipment) adj (loss or failure)) or radioactive or radiation or (nuclear adj (disaster or meltdown or catastrophe or fail$))).tw. 427,623
4 (((chemical or biological) adj warfare) or riot$ or influenza or flu or (civil adj (disorder? or defense or unrest))).tw. 89,156
5 cyclonic storms/ or droughts/ or floods/ or tidal waves/ or tsunami/ or snow/ or rain/ or avalanches/ or volcanic eruptions/ or earthquakes/ or landslides/ or fires/ or ice/ or tornadoes/ or extreme cold/ or extreme heat/ or lightning/ or cold temperature/ or hot temperature/ or wind/ or firesetting behavior/ or explosions/ or blackout/ or equipment failure/ or radioactive hazard release/ or influenza/ or refugees/ or riots/ or civil disorders/ or civil defense/ or communicable diseases/ or Severe Acute Respiratory Syndrome/ or SARS VIRUS/ or HEMORRHAGIC FEVER, EBOLA/ or MEASLES/ or SMALLPOX/ or PLAGUE/ 341,253
6 or/1–5 859,584
7 ((“public health” adj (practice or administration or incident* or emergenc*)) or “preventive medicine”).tw. or “public health”.ti,ab. 171,417
8 public health/ or preventive medicine/ or public health practice/ or public health administration/ 101,802
9 “health department*”.tw. 8,009
10 ((public or local or state or tribal or territorial or multi) adj health adj (department* or agenc*or jurisdiction)).ti,ab. 3,961
11 or/7–10 232,661
12 6 and 11 21,249
13 (epidemic? or pandemic? or terroris$ or bioterroris$ or “mass casualt*”).tw. 105,665
14 epidemics/ or pandemics/ or disease outbreaks/ or disasters/ or emergencies/ or mass casualty incidents/ or terrorism/ or bioterrorism/ or chemical terrorism/ or “september 11 terrorist attacks”/ 152,746
15 ((emergency adj (prepare* or response* or management or incident*)) or (disaster adj (plan$ or prepare* or mitigation or recover* or cycle or medicine or resilienc* or incident* or response* or management* or readiness))).tw. 8,725
16 emergency preparedness/ or emergency response/ or emergency management/ or disaster planning/ or disaster medicine/ or disaster resilience/ 15,465
17 or/13–16 230,361
18 12 or 17 241,051
19 Health Personnel/ 35,245
20 Medical Staff/ or Public Health Administration/ 17,520
21 Emergency Responders/ or Emergency Medical Technicians/ 6,006
22 Physicians/ or Pharmacists/ or NURSES/ or Dental Staff/ or Nursing Staff/ or Local Government/ or “Coroners and Medical Examiners”/ 156,181
23 (physician* or doctor* or nurse* or responder* or pharmacist* or “health worker*”).tw. 671,000
24 “technical audience*”.tw. 9
25 “health department*”.tw. 8,009
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Search No. Syntax Results
26 “public health agenc*”.tw. 1,750
27 (provider* or “homeless service provider*” or veterinar* or “environmental health provider*”).tw. 161,383
28 “Hazardous Materials Response Teams”.tw. 1
29 (“hazardous material*” adj4 (responder* or team*)).tw. 25
30 clinician*.tw. 162,346
31 (coroner* or “medical examiner*”).tw. 4,641
32 ((federal or state* or local or tribal or territorial) adj government*).tw. 8,525
33 “private sector”.tw. 6,136
34 (“first responder*” or “Program for Monitoring Emerging Diseases” or “Top Officials Three Exercise”).tw. 1,476
35 or/19–34 1,061,369
36 Emergency Medical Service Communication Systems/ 1,725
37 Electronic Mail/ 2,469
38 Text Messaging/ 2,062
39 Online Systems/ or Internet/ or Social Media/ 77,266
40 COMMUNICATION/ 77,527
41 Information Systems/ or Health Information Systems/ or “Surveys and Questionnaires”/ or Public Health Informatics/ 431,884
42 Telefacsimile/ 239
43 Telecommunications/ 4,696
44 Computer Communication Networks/ 13,375
45 “health alert network*”.tw. 8
46 “public health alert*”.tw. 30
47 “public health messag*”.tw. 655
48 “emergency alert*”.tw. 34
49 (“joint information” adj (center* or system*)).tw. 2
50 “communication system*”.tw. 3,984
51 (warning* or notification* or messag* or dissemination).tw. 117,401
52 ((crisis or risk) adj communication*).tw. 1,958
53 “communication channel*”.tw. 1,044
54 “community health information exchange”.tw. 1
55 (email* or “text messag*” or “conference call*” or “provider access line*” or website* or “guidance document*” or “threshold criteria” or webinar* or webex or webcast* or “new technolog*” or “proprietary technolog*” or “bi directional information” or “data exchange*”).tw. 44,575
56 (“communication plan*” or “communication protocol*”).tw. 474
57 (alert* adj (activation or trigger*)).tw. 53
58 ((activation or trigger*) adj alert*).tw. 56
59 telefacsimile.tw. 10
60 or/36–59 724,798
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Search No. Syntax Results
61 18 and 35 and 60 3,860
62 limit 61 to (english language and yr=”2001–Current”) 3,055
63 limit 62 to (comment or editorial or letter) 31
64 62 not 63 3,024

Embase (Ovid):

Search No. Syntax Results
1 ((natural adj (disaster? or hazard?)) or (hurricane? or flood$ or typhoon? or earthquake$ or fire? or cyclon$ or heatwave? or freezing or ((ice or snow or lightning) adj storm?) or blizzard? or “heat wave” or (extreme adj (temperature? or heat or cold)) or tsunami? or “tidal wave”)).tw. 106,875
2 (“mass migration*” or SARS or ebola or smallpox or plague or measles).tw. 50,215
3 (firesetting or arson or explosion? or bomb$ or outbreak? or refugee* or (explo$ adj device?) or blackout? or brownout? or ((power or equipment) adj (loss or failure)) or radioactive or radiation or (nuclear adj (disaster or meltdown or catastrophe or fail$))).tw. 602,188
4 (((chemical or biological) adj warfare) or riot$ or influenza or flu or (civil adj (disorder? or defense or unrest))).tw. 111,443
5 hurricane/ or drought/ or flooding/ or tsunami/ or snow/ or rain/ or ice/ or avalanche/ or volcano/ or earthquake/ or landslide/ or fire/ or tornado/ or cold/ or heat/ or lightning/ or wind/ or arson/ or explosion/ or device failure/ or nuclear accident/ or civil disorder/ or influenza/ or communicable disease/ or severe acute respiratory syndrome/ or Ebola hemorrhagic fever/ or measles/ or smallpox/ or Smallpox virus/ or plague/ or refugee/ 274,496
6 or/1–5 1,010,732
7 ((“public health” adj (practice or administration or incident* or emergenc*)) or “preventive medicine”).tw. or “public health”.ti,ab. 238,311
8 “health department*”.tw. 9,525
9 ((public or local or state or tribal or territorial or multi) adj health adj (department* or agenc*or jurisdiction)).ti,ab. 4,481
10 public health/ or preventive medicine/ or public health service/ 231,842
11 or/7–10 387,664
12 6 and 11 32,828
13 (epidemic? or pandemic? or terroris$ or bioterroris$ or “mass casualt*”).tw. 132,027
14 ((emergency adj (prepare* or response* or management or incident*)) or (disaster adj (plan$ or prepare* or mitigation or recover* or cycle or medicine or resilienc* or incident* or response* or management* or readiness))).tw. 12,084
15 epidemic/ or pandemic/ or disaster/ or mass disaster/ or nuclear terrorism/ or terrorism/ or chemical terrorism/ or bioterrorism/ or disaster planning/ or disaster planning/ or disaster medicine/ 138,280
16 or/13–15 221,710
17 12 or 16 241,171
18 health care personnel/ 140,648
19 medical staff/ 33,299
20 public health service/ 60,866
21 rescue personnel/ 7,179
22 physician assistant/ or emergency physician/ or hospital physician/ or physician/ 273,932
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Search No. Syntax Results
23 pharmacist/ 68,847
24 nurse/ 108,515
25 dentist/ 21,686
26 coroner/ 2,341
27 (physician* or doctor* or nurse* or responder* or pharmacist* or “health worker*”).tw. 1,004,960
28 “technical audience*”.tw. 18
29 “health department*”.tw. 9,525
30 “public health agenc*”.tw. 2,263
31 (provider* or “homeless service provider*” or veterinar* or “environmental health provider*”).tw. 250,619
32 “Hazardous Materials Response Teams”.tw. 1
33 (“hazardous material*” adj4 (responder* or team*)).tw. 30
34 clinician*.tw. 270,866
35 (coroner* or “medical examiner*”).tw. 5,765
36 ((federal or state* or local or tribal or territorial) adj government*).tw. 11,742
37 “private sector”.tw. 8,311
38 (“first responder*” or “Program for Monitoring Emerging Diseases” or “Top Officials Three Exercise”).tw. 2,370
39 or/18–38 1,726,738
40 e-mail/ 17,690
41 mobile phone/ or text messaging/ or reminder system/ 19,052
42 Internet/ 100,103
43 online system/ 23,460
44 data base/ 229,820
45 social media/ 13,229
46 information system/ 35,460
47 fax/ 718
48 telecommunication/ 22,813
49 computer network/ 13,665
50 “health alert network*”.tw. 10
51 “public health alert*”.tw. 45
52 “public health messag*”.tw. 947
53 “emergency alert*”.tw. 56
54 (“joint information” adj (center* or system*)).tw. 7
55 “communication system*”.tw. 5,606
56 (warning* or notification* or messag* or dissemination).tw. 175,676
57 ((crisis or risk) adj communication*).tw. 2,765
58 “communication channel*”.tw. 1,535
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Search No. Syntax Results
59 “community health information exchange”.tw. 2
60 (email* or “text messag*” or “conference call*” or “provider access line*” or website* or “guidance document*” or “threshold criteria” or webinar* or webex or webcast* or “new technolog*” or “proprietary technolog*” or “bi directional information” or “data exchange*”).tw. 85,158
61 (“communication plan*” or “communication protocol*”).tw. 818
62 (alert* adj (activation or trigger*)).tw. 130
63 ((activation or trigger*) adj alert*).tw. 114
64 telefacsimile.tw. 10
65 or/40–64 659,896
66 17 and 39 and 65 3,673
67 limit 66 to (english language and yr=”2001–Current”) 3,230
68 limit 67 to (editorial or letter or note) 186
69 67 not 68 3,044

Scopus:

TITLE-ABS-KEY(((natural W/1 (disaster* or hazard*)) or hurricane* or flood* or typhoon* or earthquake* or fire* or cyclon* or heatwave* or freezing or ((ice or snow or lightning) W/1 storm*) or blizzard* or “heat wave” or (extreme W/1 (temperature* or heat or cold)) or tsunami* or drought* or “tidal wave” or epidemic* or pandemic* or terrorism or bioterrorism or “mass casualt*” or (firesetting or arson or explosion? or bomb* or (explo* W/1 device*) or blackout* or tornado* or brownout* or ((power or equipment) W/1 (loss or failure)) or radioactive or radiation or (nuclear W/1 (disaster or meltdown or catastrophe or fail*)) or (refugee* or “mass migration*” or SARS or ebola or smallpox or plague or measles or riot* or influenza or “communicable disease*” or ((chemical or biological) W/1 warefare) or (civil W/1 (disorder* or defense or unrest)))) AND ((“public health” W/1 (practice or administration)) or “preventive medicine” or “health department” or ((public or local or state or tribal or territorial or multi) W/1 health W/1 (department* or agenc* or jurisdiction)))) OR ((emergency W/1 (preparedness or response or management)) or (disaster W/1 (plan* or preparedness or mitigation or recovery or cycle or medicine or resilience or readiness or ready))) AND (physician* or doctor* or nurse* or responder* or pharmacist* or “health worker*” or “technical audience*” or “health department*” or “public health agenc*” or provider* or “homeless service provider*” or veterinarian* or “environmental health provider*” or “Hazardous Materials Response Teams” or (“hazardous material*” W/4 (responder* or team*)) or clinician* or coroner* or “medical examiner*” or ((federal or state* or local or tribal or territorial) W/1 government*) or “private sector” or “first responder*” or “Program for Monitoring Emerging Diseases” or “Top Officials Three Exercise”) AND (“health alert network*” or “public health alert*” or “public health messag*” or “emergency alert*” or (“joint information” W/1 (center* or system*)) or “communication system*” or warning* or notification* or “community health information exchange” or messag* or dissemination or ((crisis or risk) W/1 communication*) or “communication channel*” or email* or “text messag*” or “conference call*” or “provider access line*” or website* or “guidance document*” or “threshold criteria” or webinar* or webex or webcast* or “new technolog*” or “proprietary technolog*” or “bi directional information” or “data exchange*” or “communication plan*” or “communication protocol*” or (alert* W/1 (activation or trigger*)) or ((activation or trigger*) W/1 alert*) or telefacsimile)) AND PUBYEAR > 2000

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

Exclude: Editorials, letters, and notes

Results: 1,198

Non-Pharmaceutical Interventions: Quarantine

Search Parameters:

Date: 2001–Present

Language: English

Document Type: Exclude commentaries, editorials, letters, and notes

Databases: Medline (Ovid), Embase (Ovid), Scopus

Search Syntax:

Medline (Ovid):

Search No. Syntax Results
1 ((natural adj (disaster? or hazard?)) or (hurricane? or flood$ or typhoon? or earthquake$ or fire? or cyclon$ or heatwave? or freezing or ((ice or snow or lightning) adj storm?) or blizzard? or “heat wave” or (extreme adj (temperature? or heat or cold)) or tsunami? or “tidal wave”)).tw. 75,331
2 (“mass migration*” or SARS or ebola or smallpox or plague or measles).tw. 46,567
3 (firesetting or arson or explosion? or bomb$ or outbreak? or refugee* or (explo$ adj device?) or blackout? or brownout? or ((power or equipment) adj (loss or failure)) or radioactive or radiation or (nuclear adj (disaster or meltdown or catastrophe or fail$))).tw. 427,623
4 (((chemical or biological) adj warfare) or riot$ or influenza or flu or (civil adj (disorder? or defense or unrest))).tw. 89,156
5 cyclonic storms/ or droughts/ or floods/ or tidal waves/ or tsunami/ or snow/ or rain/ or avalanches/ or volcanic eruptions/ or earthquakes/ or landslides/ or fires/ or ice/ or tornadoes/ or extreme cold/ or extreme heat/ or lightning/ or cold temperature/ or hot temperature/ or wind/ or firesetting behavior/ or explosions/ or blackout/ or equipment failure/ or radioactive hazard release/ or influenza/ or refugees/ or riots/ or civil disorders/ or civil defense/ or communicable diseases/ or Severe Acute Respiratory Syndrome/ or SARS VIRUS/ or HEMORRHAGIC FEVER, EBOLA/ or MEASLES/ or SMALLPOX/ or PLAGUE/ 341,253
6 or/1–5 859,584
7 ((“public health” adj (practice or administration or incident* or emergenc*)) or “preventive medicine”).tw. or “public health”.ti,ab. 171,417
8 public health/ or preventive medicine/ or public health practice/ or public health administration/ 101,802
9 “health department*”.tw. 8,009
10 ((public or local or state or tribal or territorial or multi) adj health adj (department* or agenc*or jurisdiction)).ti,ab. 3,961
11 or/7–10 232,661
12 6 and 11 21,249
13 (epidemic? or pandemic? or terroris$ or bioterroris$ or “mass casualt*”).tw. 105,665
14 epidemics/ or pandemics/ or disease outbreaks/ or disasters/ or emergencies/ or mass casualty incidents/ or terrorism/ or bioterrorism/ or chemical terrorism/ or “september 11 terrorist attacks”/ 152,746
15 ((emergency adj (prepare* or response* or management or incident*)) or (disaster adj (plan$ or prepare* or mitigation or recover* or cycle or medicine or resilienc* or incident* or response* or management* or readiness))).tw. 8,725
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Search No. Syntax Results
16 emergency preparedness/ or emergency response/ or emergency management/ or disaster planning/ or disaster medicine/ or disaster resilience/ 15,465
17 or/13–16 230,361
18 12 or 17 241,051
19 “nonpharmaceutical intervention*”.tw. 78
20 QUARANTINE/ 2,031
21 Social Distance/ 2,465
22 (quarantine* or “social distanc*”).tw. 4,091
23 “self isolation”.tw. 56
24 “voluntary quarantine*”.tw. 6
25 “involuntary quarantine*”.tw. 2
26 “home quarantine*”.tw. 21
27 “hospital quarantine*”.tw. 3
28 ((adherence or compliance) adj4 (quarantine* or “control measure*”)).tw. 176
29 “restricted movement*”.tw. 316
30 (separation adj4 expose*).tw. 107
31 or/19–30 7,962
32 18 and 31 1,589
33 32 1,589
34 limit 33 to (english language and yr=”2001–Current”) 1,167
35 limit 34 to (comment or editorial or letter) 48
36 34 not 35 1,119

Embase (Ovid):

Search No. Syntax Results
1 ((natural adj (disaster? or hazard?)) or (hurricane? or flood$ or typhoon? or earthquake$ or fire? or cyclon$ or heatwave? or freezing or ((ice or snow or lightning) adj storm?) or blizzard? or “heat wave” or (extreme adj (temperature? or heat or cold)) or tsunami? or “tidal wave”)).tw. 106,875
2 (“mass migration*” or SARS or ebola or smallpox or plague or measles).tw. 50,215
3 (firesetting or arson or explosion? or bomb$ or outbreak? or refugee* or (explo$ adj device?) or blackout? or brownout? or ((power or equipment) adj (loss or failure)) or radioactive or radiation or (nuclear adj (disaster or meltdown or catastrophe or fail$))).tw. 602,188
4 (((chemical or biological) adj warfare) or riot$ or influenza or flu or (civil adj (disorder? or defense or unrest))).tw. 111,443
5 hurricane/ or drought/ or flooding/ or tsunami/ or snow/ or rain/ or ice/ or avalanche/ or volcano/ or earthquake/ or landslide/ or fire/ or tornado/ or cold/ or heat/ or lightning/ or wind/ or arson/ or explosion/ or device failure/ or nuclear accident/ or civil disorder/ or influenza/ or communicable disease/ or severe acute respiratory syndrome/ or Ebola hemorrhagic fever/ or measles/ or smallpox/ or Smallpox virus/ or plague/ or refugee/ 274,496
6 or/1–5 1,010,732
7 ((“public health” adj (practice or administration or incident* or emergenc*)) or “preventive medicine”).tw. or “public health”.ti,ab. 238,311
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Search No. Syntax Results
8 “health department*”.tw. 9,525
9 ((public or local or state or tribal or territorial or multi) adj health adj (department* or agenc*or jurisdiction)).ti,ab. 4,481
10 public health/ or preventive medicine/ or public health service/ 231,842
11 or/7–10 387,664
12 6 and 11 32,828
13 (epidemic? or pandemic? or terroris$ or bioterroris$ or “mass casualt*”).tw. 132,027
14 ((emergency adj (prepare* or response* or management or incident*)) or (disaster adj (plan$ or prepare* or mitigation or recover* or cycle or medicine or resilienc* or incident* or response* or management* or readiness))).tw. 12,084
15 epidemic/ or pandemic/ or disaster/ or mass disaster/ or nuclear terrorism/ or terrorism/ or chemical terrorism/ or bioterrorism/ or disaster planning/ or disaster planning/ or disaster medicine/ 138,280
16 or/13–15 221,710
17 12 or 16 241,171
18 “nonpharmaceutical intervention*”.tw. 111
19 quarantine/ 431
20 social distance/ 2,047
21 (quarantine* or “social distanc*”).tw. 5,139
22 “self isolation”.tw. 82
23 “voluntary quarantine*”.tw. 6
24 “involuntary quarantine*”.tw. 3
25 “home quarantine*”.tw. 25
26 “hospital quarantine*”.tw. 6
27 ((adherence or compliance) adj4 (quarantine* or “control measure*”)).tw. 247
28 “restricted movement*”.tw. 501
29 (separation adj4 expose*).tw. 168
30 or/18–29 7,740
31 17 and 30 1,441
32 limit 31 to (english language and yr=”2001–Current”) 1,165
33 limit 32 to (editorial or letter or note) 49
34 32 not 33 1,116

Scopus:

TITLE-ABS-KEY(((natural W/1 (disaster* or hazard*)) or hurricane* or flood* or typhoon* or earthquake* or fire* or cyclon* or heatwave* or freezing or ((ice or snow or lightning) W/1 storm*) or blizzard* or “heat wave” or (extreme W/1 (temperature* or heat or cold)) or tsunami* or drought* or “tidal wave” or epidemic* or pandemic* or terrorism or bioterrorism or “mass casualt*” or (firesetting or arson or explosion? or bomb* or (explo* W/1 device*) or blackout* or tornado* or brownout* or ((power or equipment) W/1 (loss or failure)) or radioactive or radiation or (nuclear W/1 (disaster or meltdown or catastrophe or fail*)) or (refugee* or “mass migration*” or SARS or ebola or smallpox or plague or measles or riot* or influenza or “communicable disease*” or ((chemical or biological) W/1 warefare) or (civil

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

W/1 (disorder* or defense or unrest)))) AND ((“public health” W/1 (practice or administration)) or “preventive medicine” or “health department” or ((public or local or state or tribal or territorial or multi) W/1 health W/1 (department* or agenc* or jurisdiction)))) OR ((emergency W/1 (preparedness or response or management)) or (disaster W/1 (plan* or preparedness or mitigation or recovery or cycle or medicine or resilience or readiness or ready))) AND (“nonpharmaceutical intervention*” or “non-pharmaceutical intervention*” or quarantine* or “social distance*” or “self isolation*” or “voluntary quarantine*” or “involuntary quarantine*” or “home quarantine*” or “hospital quarantine*” or ((adherence or compliance) W/4 (quarantine* or “control measure*”)) or “restricted movement*” or (separation W/4 expos*))) AND PUBYEAR AFT 2000

Exclude: Editorials, letters, and notes

Results: 245

REFERENCES

Abramson, D. M., S. S. Morse, A. L. Garrett, and I. Redlener. 2007. Public health disaster research: Surveying the field, defining its future. Disaster Medicine and Public Health Preparedness 1(1):57–62.

Acosta, J. D., C. Nelson, E. B. Beckjord, S. R. Shelton, E. Murphy, K. L. Leuschner, and J. Wasserman. 2009. A national agenda for public health systems research on emergency preparedness. Santa Monica, CA: RAND Health.

Barnett-Page, E., and J. Thomas. 2009. Methods for the synthesis of qualitative research: A critical review. BMC Medical Research Methodology 9(1):59.

Bennett, C., S. Khangura, J. C. Brehaut, I. D. Graham, D. Moher, B. K. Potter, and J. M. Grimshaw. 2010. Reporting guidelines for survey research: An analysis of published guidance and reporting practices. PLOS Medicine 8(8):e1001069.

Briss, P. A., S. Zaza, M. Pappaioanou, J. Fielding, L. K. Wright-De Aguero, B. I. Truman, D. P. Hopkins, P. Dolan Mullen, R. S. Thompson, S. H. Woolf, V. G. Carande-Kulis, L. Andersin, A. R. Hinman, D. V. McQueen, S. M. Teutsch, J. R. Harris, and The Task Force on Community Preventive Services. 2000. Developing an evidence-based guide to community preventive services—Methods. American Journal of Preventive Medicine 18(1S):35–43.

Butler, M., R. A. Epstein, A. Totten, E. P. Whitlock, M. T. Ansari, L. J. Damschroder, E. Balk, E. B. Bass, N. D. Berkman, S. Hempel, S. Iyer, K. Schoelles, and J.-M. Guise. 2017. AHRQ series on complex intervention systematic reviews—Paper 3: Adapting frameworks to develop protocols. Journal of Clinical Epidemiology 90:19–27.

Carbone, E. G. 2018 (unpublished). Charge to committee: CDC talking points. Atlanta, GA: Centers for Disease Control and Prevention.

CASP (Critical Appraisal Skills Programme). 2018. CASP qualitative checklist. http://casp-uk.net/casp-tools-checklists (accessed July 2, 2019).

CDC (Centers for Disease Control and Prevention). 2016. Assessment of the public health preparedness capabilities: National standards for state and local planning final report. Atlanta, GA: Centers for Disease Control and Prevention. https://www.cdc.gov/cpr/readiness/00_docs/PHEP-Final-Report_508_9_20_16.pdf (accessed March 26, 2020).

CDC. 2018. Public health emergency preparedness and response capabilities: National standards for state, local, tribal, and territorial public health. Atlanta, GA: Centers for Disease Control and Prevention. https://www.cdc.gov/cpr/readiness/00_docs/CDC_PreparednesResponseCapabilities_OctOcto2018_Final_508.pdf (accessed March 26, 2020).

Center for Public Health Systems and Services Research. 2018. National health security preparedness index 2018 realease summary of key findings. Lexington, KY: Center for Public Health Systems and Services Research. https://nhspi.org/wp-content/uploads/2018/04/2018-Key-Findings.pdf (accessed March 26, 2020).

Challen, K., A. C. Lee, A. Booth, P. Gardois, H. B. Woods, and S. W. Goodacre. 2012. Where is the evidence for emergency planning? A scoping review. BMC Public Health 12(542).

Cochrane. 2017. Suggested risk of bias criteria for EPOC reviews. https://epoc.cochrane.org/sites/epoc.cochrane.org/files/public/uploads/Resources-for-authors2017/suggested_risk_of_bias_criteria_for_epoc_reviews.pdf (accessed April 20, 2020).

Davids, E. L., and N. V. Roman. 2014. A systematic review of the relationship between parenting styles and children’s physical activity. African Journal for Physical, Health Education, Recreation and Dance 228–246.

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×

ECDC (European Centre for Disease Prevention and Control). 2018. Best practice recommendations for conducting after-action reviews to enhance public health preparedness. Solna, Sweden. https://www.ecdc.europa.eu/sites/default/files/documents/public-health-preparedness-best-practice-recommendations.pdf (accessed March 26, 2020).

Higgins, J. P. T., J. Savovic, M. J. Page, J. Sterne, and ROB2 Development Group. 2019. Revised Cochrane risk-of-bias tool for randomized trials (ROB2). https://sites.google.com/site/riskofbiastool/welcome/rob-2-0-tool/current-version-of-rob-2 (accessed March 26, 2020).

Horney, J. A., E. G. Carbone, M. Lynch, Z. J. Wang, T. Jones, and D. A. Rose. 2017. How health department contextual factors affect public health preparedness (PHP) and perceptions of the 15 PHP Capabilities. American Journal of Public Health 107(S2):S153–S160.

Khan, Y., G. Fazli, B. Henry, E. de Villa, C. Tsamis, M. Grant, and B. Schwartz. 2015. The evidence base of primary research in public health emergency preparedness: A scoping review and stakeholder consultation. BMC Public Health 15(432).

Lewin, S., A. Booth, C. Glenton, H. Munthe-Kaas, A. Rashidian, M. Wainwright, M. A. Bohren, O. Tuncalp, C. J. Colvin, R. Garside, B. Carlsen, E. V. Langlois, and J. Noyes. 2018. Applying GRADE-CERQual to qualitative evidence synthesis findings: Introduction to the series. Implementation Science 13(Suppl 1):2.

Murthy, B. P., N. M. Molinari, T. T. LeBlanc, S. J. Vagi, and R. N. Avchen. 2017. Progress in public health emergency preparedness: United States, 2001–2016. American Journal of Public Health 107(S2):S180–S185.

Pope, C., S. Ziebland, and N. Mays. 2000. Qualitative research in health care: Analysing qualitative data. BMJ (Clinical Research) 320(7227):114–116.

Savoia, E., L. Lin, D. Bernard, N. Klein, L. P. James, and S. Guicciardi. 2017. Public health system research in public health emergency preparedness in the United States (2009–2015): Actionable knowledge base. American Journal of Public Health 107(S2):e1–e6.

Shekelle, P. 2004. The appropriateness method. Medical Decision Making 24(2):228–231.

Siegfried, A. L., E. G. Carbone, M. B. Meit, M. J. Kennedy, H. Yusuf, and E. B. Kahn. 2017. Identifying and prioritizing information needs and research priorities of public health emergency preparedness and response practitioners. Disaster Medicine and Public Health Preparedness 11(5):552–561.

Sloss, E. M., D. H. Solomon, P. G. Shekelle, R. T. Young, D. Saliba, C. H. MacLean, L. Z. Rubenstein, J. F. Schnelle, C. J. Kamberg, and N. S. Wenger. 2000. Selecting target conditions for quality of care improvement in vulnerable older adults. Journal of the American Geriatrics Society 48(4):363–369.

Sterne, J. A., M. A. Hernan, B. C. Reeves, J. Savovic, N. D. Berkman, M. Viswanathan, D. Henry, D. G. Altman, M. T. Ansari, I. Boutron, J. R. Carpenter, A. W. Chan, R. Churchill, J. J. Deeks, A. Hrobjartsson, J. Kirkham, P. Juni, Y. K. Loke, T. D. Pigott, C. R. Ramsay, D. Regidor, H. R. Rothstein, L. Sandhu, P. L. Santaguida, H. J. Schunemann, B. Shea, I. Shrier, P. Tugwell, L. Turner, J. C. Valentine, H. Waddington, E. Waters, G. A. Wells, P. F. Whiting, and J. P. Higgins. 2016. ROBINS-I: A tool for assessing risk of bias in non-randomised studies of interventions. BMJ 355:i4919.

USPSTF (U.S. Preventive Services Task Force). 2015. U.S. Preventive Services Task Force procedure manual. https://www.uspreventiveservicestaskforce.org/Page/Name/procedure-manual (accessed March 4, 2020).

Yeager, V. A., N. Menachemi, L. C. McCormick, and P. M. Ginter. 2010. The nature of the public health emergency preparedness literature 2000–2008: A quantitative analysis. Journal of Public Health Management and Practice 16(5):441–449.

Zaza, S., R. S. Lawrence, C. S. Mahan, M. Fullilove, D. Fleming, G. J. Isham, and M. Pappaioanou. 2000. Scope and organization of the Guide to Community Preventive Services: The task force on community preventive services. American Journal of Preventive Medicine 18(1 Suppl):27–34.

Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page251
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page252
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page253
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page254
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page255
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page256
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page257
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page258
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page259
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page260
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page261
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page262
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page263
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page264
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page265
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page266
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page267
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page268
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page269
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page270
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page271
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page272
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page273
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page274
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page275
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page276
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page277
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page278
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page279
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page280
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page281
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page282
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page283
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page284
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page285
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page286
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page287
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page288
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page289
Suggested Citation:"Appendix A: Detailed Description of the Committee's Methods for Formulating the Scope of the Reviews and Capturing the Evidence." National Academies of Sciences, Engineering, and Medicine. 2020. Evidence-Based Practice for Public Health Emergency Preparedness and Response. Washington, DC: The National Academies Press. doi: 10.17226/25650.
×
Page290
Next: Appendix B: Mixed-Method Reviews of Selected Topics »
Evidence-Based Practice for Public Health Emergency Preparedness and Response Get This Book
×
Buy Paperback | $140.00 Buy Ebook | $109.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

When communities face complex public health emergencies, state local, tribal, and territorial public health agencies must make difficult decisions regarding how to effectively respond. The public health emergency preparedness and response (PHEPR) system, with its multifaceted mission to prevent, protect against, quickly respond to, and recover from public health emergencies, is inherently complex and encompasses policies, organizations, and programs. Since the events of September 11, 2001, the United States has invested billions of dollars and immeasurable amounts of human capital to develop and enhance public health emergency preparedness and infrastructure to respond to a wide range of public health threats, including infectious diseases, natural disasters, and chemical, biological, radiological, and nuclear events. Despite the investments in research and the growing body of empirical literature on a range of preparedness and response capabilities and functions, there has been no national-level, comprehensive review and grading of evidence for public health emergency preparedness and response practices comparable to those utilized in medicine and other public health fields.

Evidence-Based Practice for Public Health Emergency Preparedness and Response reviews the state of the evidence on PHEPR practices and the improvements necessary to move the field forward and to strengthen the PHEPR system. This publication evaluates PHEPR evidence to understand the balance of benefits and harms of PHEPR practices, with a focus on four main areas of PHEPR: engagement with and training of community-based partners to improve the outcomes of at-risk populations after public health emergencies; activation of a public health emergency operations center; communication of public health alerts and guidance to technical audiences during a public health emergency; and implementation of quarantine to reduce the spread of contagious illness.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!