1

Introduction and Overview¹

During public health emergencies (PHEs) involving chemical, biological, radiological, or nuclear (CBRN) threats or emerging infectious diseases, medical countermeasures (MCMs) (e.g., drugs, vaccines, devices) may need to be dispensed or administered to affected populations to help mitigate the human health impact of the threat. The optimal MCMs determined for use during an emergency might be U.S. Food and Drug Administration (FDA) approved but used in unapproved ways (e.g., in a new age group or against a new agent); FDA approved using animal models because human efficacy testing is not ethical or feasible; or not yet FDA approved for any situation. Current medical product surveillance systems are typically passive systems that were not designed to provide information in the time frame necessitated by emergencies.

As part of the United States’ scientific and research preparedness enterprise, there is an imperative to go “beyond the last mile” of MCM dispensing and administration to build and maintain a national capability to monitor and assess the use of MCMs (e.g., safety, compliance, clinical benefit) after they have been dispensed during PHEs. This need was first recognized in 2010 by then-Secretary of the U.S. Department of Health and

___________________

¹ This workshop was organized by an independent planning committee whose role was limited to identification of topics and speakers. This Proceedings of a Workshop was prepared by the rapporteurs as a factual summary of the presentations and discussions that took place at the workshop. Statements, recommendations, and opinions expressed are those of individual presenters and participants, and are not necessarily endorsed or verified by the National Academies of Sciences, Engineering, and Medicine, or the Health and Medicine Division, and they should not be construed as reflecting any group consensus.

Human Services (HHS) Kathleen Sebelius,² whose recognition led to the development of the Public Health Emergency Medical Countermeasures Enterprise (PHEMCE) Action Plan for Developing an Enhanced National Capability for Monitoring and Assessing Medical Countermeasures During Public Health Emergencies³ in 2013. The PHEMCE action plan helped to identify the current core capabilities for MCM monitoring and assessment: data collection and reporting (i.e., surveillance systems), receipt and management of information, analysis of information, regulatory decision making, and communication.

In 2014, HHS Assistant Secretary for Preparedness and Response (ASPR) leadership established an MCM monitoring and assessment integrated program team (IPT) to develop a comprehensive, PHEMCE-wide coordinated capability to monitor and assess MCM use through data collection and analysis during and after an emergency to enable assessment and decision making during present and future PHEs. The IPT is actively working on this issue and has identified, engaged, and partnered with numerous stakeholders to advance its goals. IPT membership includes FDA, the U.S. Centers for Disease Control and Prevention (CDC), ASPR, the U.S. Department of Defense, the U.S. Department of Veterans Affairs, and the National Institutes of Health (NIH). Because of its representation on IPT and concomitant role in regulating MCMs, FDA has been an instrumental partner in working to develop the needed national capability for monitoring and assessing MCMs.

Looking forward, stakeholders could consider strategies to bridge the data collection gap between distributing and monitoring and surveilling MCMs during a PHE. Addressing this issue could better inform ongoing and future PHE responses, including the ability to use such information for clinical and regulatory decision making in near real time. This information could also advance efforts to protect and promote public health, particularly in light of the fact that there may be very limited human safety and efficacy data available for some MCMs prior to their use during PHEs. Boris Lushniak, dean and professor, University of Maryland School of Public Health, remarked that successfully developing and advancing a national MCM monitoring and assessment capability will require engagement with and by stakeholders from diverse sectors, including but not limited to local, state, tribal, and federal public health and emergency response agencies (including those involved in MCM planning and response); academia;

___________________

²The Public Health Emergency Medical Countermeasures Enterprise Review: Transforming the Enterprise to Meet Long-Range National Needs. 2010. Washington, DC: U.S. Department of Health and Human Services. https://www.medicalcountermeasures.gov/media/1138/mcmreviewfinalcover-508.pdf (accessed September 22, 2017).

³ See https://www.phe.gov/Preparedness/mcm/phemce/Documents/2015-PHEMCE-SIP.pdf (accessed September 12, 2017).

clinical trial networks; contract research organizations; health information technology; electronic health records (EHRs); big data; drug sponsors; and health care–related professional organizations (e.g., medical, public health, and health care organizations and boards).

WORKSHOP BACKGROUND AND OBJECTIVES

To further the discussion on the need for a more robust national capability for monitoring and assessing MCMs following their distribution and use during PHEs, the Board on Health Sciences Policy of the National Academies of Sciences, Engineering, and Medicine (the National Academies) hosted a 2-day public workshop, Building a National Capability to Monitor and Assess Medical Countermeasure Use in Response to Public Health Emergencies.⁴ The workshop, sponsored by FDA, was held on June 6–7, 2017, in Washington, DC. The statement of task for the workshop can be found in Box 1-1. The workshop was organized into five panel sessions (see Appendix B for the workshop agenda), which were followed by periods of open discussion with workshop participants facilitated by Laura Runnels of LAR Consulting.

ORGANIZATION OF THIS PROCEEDINGS OF A WORKSHOP

This Proceedings of a Workshop was prepared by the rapporteurs as a factual summary of what occurred at the workshop. Statements, recommendations, and opinions expressed are those of individual workshop participants and have not been endorsed or verified by the National Academies, and they should not be construed as reflecting any group consensus. The workshop was webcast live and the slide presentations and videos are archived on the meeting website.⁵

As previously noted, the workshop was organized into five panel sessions. However, to best reflect the workshop discussions and themes that arose over the 2 days, the rapporteurs for this proceedings chose to structure the discussions conceptually as follows:

• The final section of this chapter provides workshop sponsor FDA’s perspective on MCM monitoring and assessment during PHEs, which set the stage for workshop discussions.

___________________

⁴ See https://www.phe.gov/Preparedness/mcm/phemce/Documents/2015-PHEMCE-SIP.pdf (accessed September 12, 2017).

⁵ See http://nationalacademies.org/hmd/Activities/PublicHealth/MedicalCounterMeasures/2017-JUNE-06.aspx (accessed September 22, 2017).

• Chapter 2 introduces the concepts of electronic health data, big data, operations for response, and clinical networks and explains how each of these terms was defined for the purposes of the workshop. This chapter also provides stakeholder perspectives from federal, state, and local health agencies; health care; the pharmaceutical industry; and academia.
• Chapter 3 summarizes workshop discussions that characterized data needs, sources, and collection methodologies for stakeholder decision making, with consideration of topics such as leveraging

• existing datasets, disseminating data and information, and unique challenges for data collection.
• Chapter 4 provides an overview of workshop discussions that centered on considerations for conducting rapid clinical research on MCMs during a PHE. Workshop speakers and panelists contemplated how operations for threat response could be adapted and how the current clinical trial infrastructure could be leveraged to facilitate MCM monitoring and assessment in PHEs, with discussion of topics such as data and information sharing, protocol design, and governance.
• Chapter 5 summarizes both the perspectives of federal stakeholders on key workshop takeaways, including barriers and opportunities for promoting monitoring and assessment of MCM use within their respective agencies, and reflections from individual workshop participants.

VIEWPOINT FROM THE U.S. FOOD AND DRUG ADMINISTRATION ON MCM MONITORING AND ASSESSMENT DURING A PHE

Rear Admiral Carmen Maher, acting assistant commissioner for counterterrorism policy and acting director of the Office of Counterterrorism and Emerging Threats at FDA, set the stage for the workshop by providing an FDA perspective on monitoring and assessment of MCMs. The pace of potential threats has increased since the 2001 anthrax attack, she noted, including several infectious disease threats that have emerged or reemerged in the past 5 years, such as Ebola virus, Zika virus, Middle East respiratory syndrome coronavirus, highly pathogenic avian influenza, enterovirus D68, measles, and H7N9 influenza. In response to these threats, the MCM field has advanced significantly in the last 15 years. For example, a comprehensive framework⁶ was developed by stakeholders to rapidly identify the requisite MCM for a given threat and to help facilitate the development, manufacturing, stockpiling, and deployment of those MCMs. More recently, Maher noted, efforts have focused on how to conduct advanced product development in the midst of responding to a PHE (e.g., during the recent Ebola virus and Zika virus outbreaks). As is the case with any new medical product, when assessing MCMs, FDA must balance the potential benefits and risks of a product in light of the available scientific data, the public health needs, and the regulatory framework. When monitoring and assessing MCMs, however, all aspects of this process occur within a com-

___________________

⁶ See https://www.phe.gov/Preparedness/mcm/phemce/Documents/2015-PHEMCE-SIP.pdf (accessed September 12, 2017).

pressed timeframe. Furthermore, FDA must be cognizant of accessing the information needed to fulfill his goal without disrupting on-the-ground responses, said Maher.

Role of FDA Across the MCM Product Life Cycle

Maher discussed key elements of FDA’s role in the development, review, and distribution of MCMs and the need to go beyond the last mile to monitor and assess them for safety and effectiveness. The traditional medical product life cycle is a tightly controlled, iterative process, Maher said, and decisions are made by FDA at each developmental stage regarding whether a product advances (see Figure 1-1). FDA has several legal and regulatory mechanisms that can be applied during the development and review of MCMs to accelerate medical product approval and availability, such as the Animal Rule, Emergency Use Authorization (EUA), Expanded Access provisions, and various authorities provided under Investigational New Drug Application and Investigational Device Exemption pathways.⁷ After a product is approved, FDA has a variety of mechanisms for post-market

___________________

⁷ More information about FDA’s legal and regulatory MCM authority is available at https://www.fda.gov/EmergencyPreparedness/Counterterrorism/MedicalCountermeasures/MCMLegalRegulatoryandPolicyFramework/default.htm (accessed August 23, 2017).

surveillance, including passive and active surveillance systems and voluntary and mandatory reporting. Maher reiterated that most of the capabilities for post-market monitoring of medical products were not designed for use in PHEs. Still, FDA is currently reliant on those systems as there is no comprehensive strategy or framework for monitoring and assessment of MCM use during a PHE.

Medical products typically proceed through the preclinical and clinical developmental steps outlined in Figure 1-1 with the end goal of being approved and marketed. In recent responses to PHEs, however, FDA has been faced with making approval decisions farther left along the pathway than ever before (i.e., closer to the discovery and preclinical phases), with information on safety and effectiveness being collected after the product has been made available for use in the target population, said Maher. In some cases, such as a mass casualty event, she added, such a product might have been made widely available. Maher highlighted several potential key challenges in developing MCMs, including the following:

• The affected population is only available for clinical research during a PHE, so traditional clinical trials are not feasible.
• Products may have been approved under the Animal Rule,⁸ so the first opportunity to gather data on MCM efficacy and safety in the affected population might be during a PHE.
• Post-marketing commitments or requirements may accompany the MCM for which the requisite data may be difficult to collect during a PHE.
• No MCMs are available for the threat at hand, so an early research and development product that could potentially address the threat must be identified and a determination made as to if, and how quickly, it could be developed.
• Human efficacy tests might be deemed unethical, further complicating the planning and conduct of clinical trials in a PHE.

Despite these challenges, and although regulatory decisions might be made based on limited data, Maher emphasized that such decisions are carefully considered by FDA and are made based on informed benefit–risk calculations using sound scientific data. Because information is often limited, however, it is essential to collect as much robust, credible data as possible during product use in a PHE to complete the safety and efficacy profile, she said.

___________________

⁸ For more information about the Animal Rule, see https://www.fda.gov/emergencypreparedness/counterterrorism/medicalcountermeasures/mcmregulatoryscience/ucm391604.htm (accessed August 23, 2017).

TABLE 1-1 Differences Between FDA Product Assessment in a PHE Versus Traditional Research and Development

SOURCE: Maher presentation, June 6, 2017.

Maher highlighted some of the key differences between assessing a new medical product developed via a traditional research and development pathway and an MCM developed in anticipation of or in response to a PHE (see Table 1-1). The intent in a PHE is to mitigate and respond, Maher reiterated, and decisions are made quickly. PHEs are often unpredictable in terms of their timing, location, and severity, even PHEs that have been planned for with preparedness activities. Maher also noted that during a PHE there is often simultaneous administration of multiple products to large numbers of the general public. Furthermore, during an emergency response, there is little or no tracking or monitoring of products that have been used as many MCMs are dispensed on scene by first responders or at mass points of dispensing (PODs). In contrast, in the traditional research and development pathway, a single product is studied in specific populations in a very controlled clinical setting.

Looking Beyond the Last Mile in the MCM Product Life Cycle

Within the federal government, emphasis has traditionally been placed on the dispensing and distribution of MCMs—commonly referred to as the “last mile” of the MCM product life cycle—rather than on their monitoring and assessment. The focus of this workshop, Maher said, is what happens beyond the last mile. Although tools and capabilities are available with which to monitor and assess medical products, these tools were not originally developed with the purpose of rapidly collecting, analyzing, and gathering data in the midst of PHEs or using that information in real-time to inform decisions on the use of that product within present (and future)

PHEs. Importantly, Maher said, data are needed that can show if the countermeasure is not performing as expected, or if there are unanticipated adverse events. Maher challenged workshop participants to consider how and where such information about MCM performance could be captured and rapidly assessed without disrupting the ongoing emergency response. To move beyond the last mile and integrate monitoring and assessment into the MCM life cycle, FDA is reviewing its current capabilities and identifying opportunities in four main areas: electronic health data, unstructured/big data, operations for response, and clinical networks.

There has been much progress to date, Maher said, including a host of PHEMCE, Department of Defense, and other initiatives that are under way or in development (see Box 1-2). Over the past 10 years, stakeholders within PHEMCE have made significant progress in coordinating response efforts in PHEs, with a mutual recognition of the contributions each stakeholder can bring to bear. She noted that managing and prioritizing differing commitments and priorities among stakeholders in a way that provides for a coordinated, rapid response are critical to these efforts. However, Maher noted, this coordination has been more successful in development and distribution of MCMs than in gathering the requisite data for monitoring and assessing MCMs.

Finally, Maher reemphasized that FDA does not intend to create a new system of monitoring and assessing medical products. Rather, their goal is to build a PHEMCE monitoring and assessment component into existing infrastructure. For example, how could current EHR capabilities or hand-held device capabilities be used to collect information on MCM product performance in a real-world setting? How could clinical trial networks be linked? What roles could machine learning, social media, crowd sourcing, and smart technology play? Ultimately, Maher concluded, how do we leverage and coordinate all the many elements and partners during a PHE response to collect safety and efficacy data about MCMs?

OVERVIEW OF CROSS-CUTTING TOPICS HIGHLIGHTED DURING PRESENTATIONS AND DISCUSSIONS⁹

Various themes emerged across multiple panel sessions and discussions as workshop participants considered the various aspects of monitoring and assessing MCMs in PHEs. The themes and opportunities highlighted below,

___________________

⁹ The rapporteurs’ summary of the main topics and recurring themes is drawn from the presentations, discussions, and summary remarks by the workshop panelists and participants. Items on this list should not be construed as reflecting any consensus of the workshop participants or any endorsement by the National Academies of Sciences, Engineering, and Medicine.

drawn from individual panelist remarks and open discussions, are discussed further throughout this proceedings.

Benefits and Risks of MCMs

A key stage-setting question considered by workshop participants was whether an MCM deployed during a PHE is providing benefit (i.e., acting

as expected), or causing harm. Discussions at the workshop centered on performing research and collecting data to inform the benefit–risk profile of MCMs, which may be deployed during PHEs at varying developmental stages (i.e., an investigational product with limited preclinical data, a product approved under the Animal Rule, or an FDA-approved product that may have post-marketing commitments and requirements) and with varying levels of safety and efficacy data. Because PHEs dictate a rapid timeline for

collecting data on safety and efficacy to inform MCM benefit–risk profiles, several workshop participants noted that discussions should consider how existing infrastructure and data sources could be leveraged and adapted for this setting.

Clinical Development for MCMs Versus Traditional Medical Product Development

As various workshop participants noted, a PHE might be the only opportunity to collect clinical safety and efficacy data on MCMs in humans. However, the uncertainty of when and how an MCM will be used makes it difficult to prospectively craft protocol designs and determine elements of clinical research that will be adaptable to a broad range of situations. Topics raised throughout the workshop related to clinical research on MCM use included elements of predefined study protocols, predefined variables for collecting data, and pre-identification of clinical trial sites. Elizabeth Higgs, global health science advisor for the Division of Clinical Research at the National Institute of Allergy and Infectious Diseases (NIAID) at NIH, noted that clinical research on MCMs requires increased efficiency, expediency, and adaptability, relative to routine clinical development. In light of these requirements, however, stakeholders must ensure that data collection efforts are as robust as possible and that rigor is not sacrificed at the expense of being faster and more adaptive in collecting data and/or implementing trials, she said.

Question-Driven Data Collection

A common theme throughout workshop discussions was that data collection should be driven by the questions that need to be answered in order to inform MCM use. However, data also inform the development of questions, and issues can arise when questions are driven by the availability of data, rather than by what needs to be known. Understanding the bidirectionality between crafting the right questions and understanding the availability and applicability of data sources to answer those questions is critical to monitoring and assessment efforts.

Potential Challenges to Identifying Data Needs and Data Collection

Several workshop participants noted that it is often not possible to prospectively determine what data need to be collected on MCM use during PHEs. Anita Patel, senior advisor and lead in the Pandemic Medical Care and Countermeasures Influenza Coordination Unit of the National Center for Immunization and Respiratory Diseases at CDC, noted that although

baseline data are needed for any PHE, depending on the specifics of the PHE—for example, the severity of the threat and known benefit–risk profile of the MCM—additional categories of information may be required. Furthermore, she said, different data users need different types of data, and data sources used will shift as data needs evolve over the course of an event.

In considering how best to collect data, several key themes arose, including understanding who the end users of the data are; determining where the best physical locations for collecting data are (i.e., in the field or in inpatient or outpatient facilities); how to best align data collection with timing and sequences of PHEs; identifying missing or unknown data; integration of data from different sources into an evolving response structure; and leveraging machine learning and artificial intelligence, as well as innovative modeling, for collecting and interpreting health data. Some of these concepts are discussed in this section, as well as throughout the proceedings.

Data Gaps and Equity

In consideration of what data are needed and the potential sources for gathering these data, individual participants emphasized that stakeholders should also be cognizant of what data are not being collected and how these missing data affect information-gathering efforts. For example, which patient populations are underrepresented during PHEs, how could this underrepresentation bias the data, and how could this issue be addressed in future data collection efforts? At a more granular level, participants noted that certain types of data are currently missing that would benefit MCM monitoring and assessment efforts, including medical histories and symptomatology.

It was emphasized that stakeholders would benefit from increasing their understanding of data collection methods and potential biases in data samples. Individual participants highlighted the need to recognize the differences between data that are collected for a specific purpose (following a predefined methodology) and data that are collected and analyzed retrospectively, and to interpret findings accordingly.

Data Standardization, Integration, and Interoperability

The health care system in the United States is complex and heterogeneous, and there is no single unified system with common standards for collecting and sharing data, said Eva Lee, director of the Center for Operations Research in Medicine and Health Care at the H. Milton Stuart School of Industrial and System Engineering at the Georgia Institute of Technology. Even local health departments within a state or individual hospitals

under the same network face challenges sharing information. At the present time, the electronic health data sources available to stakeholders are not completely interoperable, so the current task is to assemble a patchwork of data sources into a comprehensive surveillance system for MCMs that can be rapidly activated and cover a large portion of the population. Looking forward, several workshop participants emphasized that stakeholders should work toward interoperability between EHR systems and between EHR and other data systems. Integration of data could occur at various time points: early (at data input), intermediate (mapping multiple sources to a common data model), or late (after separate analysis).

Legal, propriety, and governance barriers present challenges to interoperability of systems, however, and it will take effort and good will to find ways to make these systems work together for purposes other than that for which they were primarily designed and in a way that remains consistent with their original missions. Issues to be considered and surmounted include data standardization and careful consideration of patient privacy and data security.

Data Privacy and Security

There is a clear need to balance patient privacy and data security with sharing of information; however, it was acknowledged by individual workshop participants that the Health Insurance Portability and Accountability Act of 1996 (HIPAA) is often subject to misinterpretation, overinterpretation, or is used as a reason not to share information. Furthermore, it was noted that there is often confusion among institutional review boards (IRBs) about when sharing of information is considered research and when it is considered public health practice. Several participants discussed the need to raise awareness that public health practice is not human subjects research and does not require IRB review.

Existing Data Sources, Datasets, and Clinical Trial Infrastructure

FDA noted at the outset of the workshop that its goal is not to create a new system to monitor and assess MCMs, but to build a PHEMCE monitoring and assessment component into existing infrastructure. However, most existing data sources are part of the traditional care delivery setting, and MCM dispensing often takes place in non-traditional settings. Therefore, tools routinely used for medical product surveillance settings are not immediately amenable to the rapid collection, aggregation, and analysis needed to enable real-time use of data for decision making in a PHE. Consideration could be paid as to how these data and tools could be adapted for this setting.

Throughout the workshop, individual participants highlighted some of the many existing data sources that could be leveraged for monitoring and assessing the outcomes of MCM use, including point-of-care patient narratives, EHRs, government surveillance and tracking systems, big data, emergency medical services (EMS), pharmacy databases, social media, and mobile health applications. Challenges, limitations, and the potential for these opportunities were discussed.

The use of existing clinical trial networks will be important to achieving rapid connectivity throughout the MCM monitoring and assessment enterprise, noted individual workshop participants. For example, in a PHE, predesigned protocols could be sent to predetermined, experienced clinical sites, and existing “networks of networks” could be adapted for this purpose.

Workforce and Training

The field of data science requires a broad range of skill sets and specific subject-matter expertise in the health sciences. No single individual has all the skills and expertise needed to be able to address any given problem. Participants discussed training programs that could enable more effective use of data systems and incorporating data science in undergraduate and graduate education to develop a pipeline of individuals who have data science and informatics competencies to complement their domain knowledge. Participants also remarked that an introduction to clinical research techniques could be incorporated into preparedness training for PHE responders to provide them with a broader perspective and increased familiarity with performing data collection on MCM use in the context of a PHE.

Leadership and Administrative Preparedness

Moving forward, leadership and responsibility around MCM monitoring and assessment during PHEs are needed to promote coordination of efforts, said Higgs. These efforts are generally more developed at the international than at the domestic level, she added. Similarly, remarked Ray Barishansky, Deputy Secretary of Health, Pennsylvania Department of Health, although public health preparedness and response has become quite robust over the past 15 years, there is an “administrative preparedness” angle to MCM monitoring and assessment that is often not as robust as it could be. At the state and local levels, he said, efforts can be decentralized, and there can be confusion around who the primary authority is during a response, including in areas such as data sharing.

Continuous Testing of Critical Infrastructure Systems

The importance of routine testing and exercising of systems to develop proficiency among users and establish the capacity of these systems was a common theme among workshop participants. Incorporating awareness of practices, policies, and regulations into everyday health practice could serve to ensure their understanding and identify risks and concerns before a crisis occurs.

Effective Communication Among Stakeholders

Individual participants called for better communication across stakeholders, including better processes for disseminating changes in policy and guidance from federal agencies down to local health departments, as well as ways to communicate information and questions back upstream. In consideration of collecting the best data possible, several participants observed that researchers could do better at articulating the value of data collection to health systems administrators, including explaining why they should share data from their systems and similarly articulating the value of public health preparedness effort to policy makers (see Policy and Regulation [Dis] Incentives in Chapter 3).

In the context of communicating to the public during a PHE, data are needed by local authorities to enable timely risk communication to affected populations. Alison Levy, emergency operations manager for Public Health–Seattle and King County, Washington, noted that cross-jurisdictional coordination is essential, as an uncoordinated message across a region could confuse the public on the necessary actions and damage public trust. In a similar vein, participants also discussed the importance of timely return of findings from data analysis to stakeholders, including research participants.

Funding

It was observed by individual workshop participants that medium, small, and rural health agencies are willing to participate in data collection, but they often lack the funding, staffing, and technology to do so. This gap could be addressed so that smaller health jurisdictions can collect and report their data and understand what the data mean for them on the local level. In support of this effort, potential costs of data collection and funding sources for these data collection efforts could be determined before a PHE occurs. Bringing the PHEMCE perspective to stakeholder activities that are already being funded to collect data could be a solution to implement cost savings through coordination of efforts.