BioWatch and Public Health Surveillance: Evaluating Systems for the Early Detection of Biological Threats: Abbreviated Version (2011)

The committee was tasked with evaluating the relative merits and current and potential capabilities of the BioWatch monitoring system (Generation 2 and Generation 3) to detect bioterrorist attacks and other biothreats. This chapter provides (1) an evaluation of the current BioWatch system; (2) an assessment of plans for the next generation of environmental monitoring technology; and (3) recommendations for improving the program’s operation and processes for further development.

In its evaluation, the committee considered the entire BioWatch “system,” as introduced in Chapter 1—the technology to collect and test air samples, the associated laboratory assays, the additional information gathering needed to confirm and characterize an incident, operational guidance, interagency and risk communication, response planning, and the personnel to support these operations. It also looked at the BioWatch program—the programmatic activity managed and funded by the Office of Health Affairs (OHA) in the Department of Homeland Security (DHS) and carried out through work with other federal agencies and state and local partners.

For the BioWatch system to effectively detect a terrorist attack and prevent illness and death, a variety of conditions must be met. Some of these conditions are clearly independent of the performance of the BioWatch system (e.g., whether people are exposed to the biological agent, or whether the agent is susceptible to medical countermeasures), but they are necessary conditions for prevention of illness and death:

1. An airborne bioterrorist attack must occur in a place where the BioWatch system is deployed and with a biological agent that BioWatch air samplers and laboratory test methods can detect.

2. Atmospheric conditions must disseminate the material in a way that exposes people to a viable pathogenic biological agent.

3. Atmospheric conditions must disseminate the material in the direction of the BioWatch air samplers, resulting in sufficient quantities reaching their filters to allow detection.

4. The samplers must function as anticipated; samples must be collected, managed, and tested appropriately to allow for detection of the genetic signature of the biological agent; and positive results must be reported to public health authorities in time to allow for any assessment that must be made before a decision regarding mass prophylaxis can be reached.

5. Public health and other authorities must collaborate effectively in conducting an assessment of the likelihood that the positive test indicates a bioterrorist attack and a population health risk, based on the mix of factors that may shape such assessments (see Chapter 2); and, taken together, the correct conclusion must be made that mass dispensing is indicated.

6. Infection with the biological agent can be successfully prevented or treated with the drugs or other medical countermeasures that could be used in the mass prophylaxis program.

7. State or local public health systems must be able to deliver mass prophylaxis in time for its benefits to be realized; depending on the disease-causing agent and the size of the exposed population, this may involve providing prophylaxis within 2–3 days to several million people.

As noted in Chapter 1, evaluation of the likelihood of an airborne biological attack that could be detected by the BioWatch system is beyond the committee’s scope, as is an evaluation of the ability of public health officials to deliver mass prophylaxis in a limited time period. Here, the committee evaluates elements of the BioWatch system that are necessary, if not sufficient, for the system to be effective, including the air samplers and their placement, the samplers’ capture of targeted organisms, the laboratory assays, information reporting, event characterization, and public health decision making.

The committee considered information provided by DHS regarding the separate components of the BioWatch system, as well as information relevant to the operational capabilities of the components working together.

There are two types of BioWatch air samplers: the Portable Sampling Unit (PSU) and the Dry Filter Unit (DFU). Information about the performance of these devices is critical for a realistic assessment of system performance. The committee examined the available evidence provided by DHS, but some of the information that the committee would have liked to review has not been collected. The committee also considered information about the number and placement of air samplers in BioWatch jurisdictions and the analytic tool that was developed by the Los Alamos National Laboratory and used to guide their placement for the Generation 2 deployment. The committee considered the siting methodology applied in the Generation 2 deployment to be an improvement over the original approach, but it saw areas where better data and modeling approaches for complex urban environments would be valuable.

Since 2003, millions of BioWatch samples have been analyzed. Positive laboratory findings have led to BioWatch Actionable Results (BARs) being declared in a few dozen instances, none of which has been shown to be the result of the intentional release of a biological agent. DHS does not consider these BARs to be “false positives” because the target DNA was actually detected by the specified assays.¹ The committee’s view is that from an operational perspective these detections can indeed be considered “BAR false positives,” because the detections were not the result of a bioterrorist attack.

The laboratory assays used to detect the presence of genetic material from biological agents of interest need to be both specific and sensitive: that is, the assay must have a high probability of detecting the biological agents that BioWatch targets and have a low probability of reporting the presence of genetic material from other organisms as “detections.” Several agents of concern, and their close genetic relatives, exist naturally in water or soil (Kuske, 2005; Kuske et al., 2006). Distinguishing natural background levels of endemic agents from a bioterrorism threat and separating genetic “near neighbors” from the target agent present technical challenges. In addition, the potential exists for newly emerging or “engineered” threats to be used in an airborne biological attack. Ideally, BioWatch probes and assays could

be rapidly updated as newly emerging or engineered threats are recognized and as risk-management analyses indicate a need.

An assay that is not sufficiently sensitive can give false negative results, if, for example, an airborne threat were present but was undetected during analysis. This “missed detection” probability is a parameter that is essential for evaluating the effectiveness of the system. In addition, the PCR assay panels employed on the existing analysis platforms need to be adequately compared, standardized, and validated in pursuit of optimal performance across the BioWatch system. A close collaboration is needed between DHS and the Department of Health and Human Services (HHS) to do so.

Collection efficiency, efficiency of extraction, and subsequent sample dilution are all important factors in assessing a detection system’s sensitivity. Members of the committee had the opportunity to review information about the BioWatch Generation 2 sensitivity levels, but the committee did not have adequate test and validation data to evaluate the ostensible performance of the BioWatch technology. However, as discussed earlier, the experience available from BioWatch Generation 2’s deployment to date provides real-world data to consider: numerous BARs for certain agents, and no alarms (and no apparent misses) for other BioWatch target pathogens.

To the extent it is well understood for a particular biological agent, the consideration of infectious dose (i.e., the minimum amount of a biological agent necessary to cause infection in a host) may help in the interpretation of BioWatch results. However, uncertainties about infectious doses for some agents, differences among pathogens in the size of their infectious doses, and uncertainties about the concentrations and dispersion of an aerosolized agent used in an attack mean that the committee does not recommend that consideration of infectious dose be a factor in distinguishing between an attack and natural background. To the extent that the analysis of BioWatch samples can relate the amount of genetic material present in a sample to infectious dose, the information is likely to aid public health officials in planning a response. But BioWatch collectors and assays should be sensitive enough to detect small quantities of genetic material for a targeted pathogen, because the presence of small quantities at one collector may be related to the presence of much larger quantities elsewhere or, perhaps, a failed attack that requires further investigation.

Test and evaluation (T&E) processes provide an opportunity to assess a system’s actual performance against the stated requirements and specifications, to assess technical maturity, and to provide assurance that the system will effectively perform its mission. Developmental T&E (DT&E) focuses on the technological and engineering aspects of the system, and operational

T&E (OT&E) evaluates a system’s ability to perform as intended in its operating environment, including consideration of end users’ capabilities and proficiencies. OT&E can identify critical issues in system operations (e.g., failure, inaccuracies, subsystem incompatibility, and cost) that need to be resolved to ensure that the system performs to stated capabilities. OT&E can also provide the opportunity to empirically and realistically compare the costs and benefits of next-generation technology against currently fielded systems. It provides the opportunity to consider otherwise-unforeseen barriers to accurate and reliable detection and to make corresponding system improvements before full-scale deployment.

At Dugway Proving Ground, it is possible for an entire sampling device or its components to be challenged with simulants² or live agents in chambers or to undergo whole-system testing with simulants in open-air tunnels. DT&E at Dugway allows a designer to assess the performance of a system in a relatively “pristine” environment, without consideration of confounding factors (e.g., weather, pollution, humidity, buildings or other physical structures, and people and animals and pathogens associated with them) that may affect the performance of a system in an urban environment.³ The BASIS technology that was adapted for BioWatch underwent DT&E at Dugway, where it was challenged with live agents inside a sealed chamber, but the results from the tests were not provided to the committee.

With only limited information available from test and evaluation studies, the committee could not appropriately evaluate whether the BioWatch system meets user needs.

DHS has direct programmatic responsibility for the BioWatch air samplers, laboratory assays, and the reporting of assay results by its contractor staff to local public health officials. DHS’s role after the declaration of a BAR, along with that of HHS/CDC and other federal partners, is to support

their state and local partners who are responsible for public health decision making. Having the information tools necessary to make post-BAR decisions as rapidly as possible is likely to be crucial in being able to maximize the possible benefit from the BioWatch system’s detection of a bioterrorism attack.

Little or no empirical data were available to help the committee in its evaluation of the response aspect of the BioWatch system. One after-action report was provided to the committee (Lindley, 2009), but the chief source of information was testimony by state and local public health officials during the committee’s information-gathering meetings. (See Appendix A for a list of participants at these meetings.)

In its information gathering, the committee explored aspects of the current BioWatch response capability, including the coordination and collaboration between DHS and public health officials as well as the resources needed for response. The committee presents its findings and recommendations for improvements to the BioWatch system in this chapter. These improvements can be implemented immediately to increase the potential effectiveness of the existing system and need not await development and implementation of the next generation of BioWatch technology.

The BioWatch program appears to lack necessary coordination, communication, and collaboration among the several contributors at federal, state, and local levels that must be fully engaged for a functional system. The committee heard testimony about significant shortcomings in the earliest days of the BioWatch program in DHS’s inclusion of public health officials as partners and collaborators in developing plans and procedures for a response to BARs. At that time, the planning and concept of operations for the response to a BAR had not been well developed and did not appropriately take into account the central role of local officials. A lack of consensus about communication with local officials was illustrated by a 2005 instance in which tests of air samples indicated the presence on the National Mall in Washington, DC, of one of the biological agents monitored by the BioWatch system. The criteria for a BAR were not fully met, but local officials were informed of the detection 4 to 5 days after the event (Dvorak, 2005). Moreover, they learned of it from CDC rather than from DHS (Dvorak, 2005). Public health officials also told the committee that DHS had initially been reluctant to inform local and state officials about the locations of BioWatch air samplers within their own jurisdictions.

Further, DHS has not always provided technical information to public health officials about the sensitivity, specificity, and limits of detection of the laboratory assays that are the basis for declaring a BAR. As a result,

the laboratory directors in some jurisdictions, whose responsibility it is to report a BAR, have not had the means to evaluate the laboratory evidence concerning the presence of a biological agent (Personal communication, Scott Becker, Association of Public Health Laboratories, April 22, 2009). Although procedures for interpreting and responding to positive test results have improved, there remain serious gaps in effective communication and trust between the BioWatch program and public health personnel in BioWatch jurisdictions.

The seeming lack of trust by DHS in public health officials has been mirrored by an apparent lack of confidence in DHS and the BioWatch program by many local public health officials. They are cautious about the interpretation of BARs because of experience with the system to date, and some question the value of BioWatch.

The BioWatch system relies on prompt public health action, such as prophylaxis in response to a BAR, to be most effective in saving lives. Critical to the success of the BioWatch system is the confidence of local and state public health officials and decision makers in the system’s detection and assay technology, and in their ability to work as partners with DHS and other federal agencies in preparing for and possibly having to respond to a bioterrorist attack detected through BioWatch. Public health officials emphasized in their discussions with the committee that they are aware that they depend on the public’s trust to be able to motivate action and cooperation and avoid panic in life-threatening circumstances. These officials do not want to risk this trust by initiating potentially high-regret actions for a possible bioterrorist attack when a full-blown response is not warranted, as has been the case with the BARs to date. Some stated to the committee that they would be unlikely to administer prophylaxis on the basis of a BAR alone, waiting instead until clinical cases occur before taking that step.

The committee sees the need for DHS to strengthen its relationships with state and local health officials and other key responders through training, exercises, and information exchange. The annual BioWatch meeting provides one forum for information exchange between BioWatch jurisdictions and DHS and among the jurisdictions. However, this annual conference is not sufficient for such information sharing. In the past, DHS has provided limited opportunities for state and local input into the conference agenda or an active role in planning it. Going forward, DHS should use this conference as an opportunity to more actively involve local partners as part of forging a stronger partnership.

DHS has begun taking some important steps to test and validate BioWatch assays that should help build confidence in the assays. DHS can further improve upon this in continuing its efforts to strengthen relationships with its local and state partners. It needs to not only invite input but also respond to it, and it needs to facilitate more information sharing about

matters such as BARs so that the BioWatch jurisdictions can learn from others’ experiences. To the extent that such information sharing has been constrained by security concerns, DHS should help ensure that essential public health officials in each jurisdiction have appropriate security clearances. The training and exercises that DHS holds with BioWatch jurisdictions should also continue to be helpful in strengthening necessary skills and the relationships vital to effective action under critical circumstances following a BAR.

Several critical decisions need to be made in the event of a BAR: (1) Does it indicate a real bioterrorist attack? (2) If so, or if an attack cannot be ruled out, should public health officials begin to mobilize to start prophylaxis, and what population should be targeted initially? (3) Is it safe to allow persons to stay in the area around the site of the BAR, or should recommendations be made to evacuate or shelter in place? and (4) What information should be provided to the public?

Decisions such as these will need to be made very quickly, even though important information is likely to be limited. At the time of a BAR, the available information may include the site(s) of the air sampler(s) that produced positive assay results, weather conditions, and the biological agent detected. It may take 1 to 2 days to obtain additional data from environmental sampling and surveillance inputs. Initial decisions may need to be revised as additional information becomes available.

Information from law enforcement and national security sources is not typically a factor in public health decision making, but it will be critical if bioterrorism is suspected. While it is quite likely that a biological attack could occur without any meaningful intelligence warning, all relevant foreign and domestic intelligence that might possibly reflect on the likelihood of an attack that threatens human health should be immediately delivered to public health decision makers, along with any insights on how to evaluate or interpret such findings. The decision makers will need to know of any information such as thefts or diversions of agent stocks or known efforts by potential terrorists to culture organisms or develop means of dissemination.

One critical gap in current tools and technology is a means to determine the extent of contamination. The 2001 anthrax attacks demonstrated seri-

ous weaknesses in the environmental sampling approaches taken by federal agencies to determine the presence or absence of a biological agent. Because validated methods for collection of environmental samples and analysis were not used, there could be little confidence in the results (GAO, 2007). Draft guidance on environmental sampling following a BAR has been developed by DHS (2008c), and other agencies have also offered relevant guidance (CDC, 2002; EPA, 2008). Nevertheless, no validated sampling strategy and collection methods are currently available for use in situations in which contamination from a biothreat agent is suspected. Without such validated methodologies, states and localities lack much-needed means to determine the extent of contaminated areas or identify populations most likely to be exposed.

Another potential tool for characterizing an event that may be a bioterrorist attack is the use of dispersion modeling to assess where exposures to airborne pathogens may be likely.⁴ According to GAO (2008b), field testing and evaluations show that plume models developed by federal agencies specifically for tracking the atmospheric release of CBRN (chemical, biological, radiological, or nuclear) materials in urban areas have severe limitations and require additional field evaluation. Moreover, the deficiencies in environmental sampling, noted above, mean that it is unlikely to be helpful in refining the results produced by dispersion models.

DHS has supported the development of tools intended to help jurisdictions assess a biological event. For example, the Bioagent Event Reconstruction Tool (BERT) employs modeling to simulate what might happen during an airborne biological agent attack (Brown et al., 2006). It uses as inputs measurements from biological agent detectors and data from wind sensors. While BERT can estimate the amount of material released and offer possible locations for a release for certain kinds of scenarios, it is not applicable to other conditions. In addition, its plume models have important limitations (Brown et al., 2006). During TOPOFF exercises led by DHS, contradictory results provided by competing plume models led to confusion by first responders and decision makers.

The DHS Interagency Modeling and Atmospheric Assessment Center (IMAAC) at Lawrence Livermore National Laboratory has been designated as the focal point for coordinating and disseminating plume modeling products. However, GAO (2008b) concluded that IMAAC needs better

procedures for working with federal, state, and local agencies to deal with conflicting modeling results.

The available dispersion models can be used to generate hypotheses for testing and to inform analysis of release sites, but they are not designed to guide public health interventions. On the other hand, as made clear in other parts of this report, the use of epidemiological models, and models of responses to detection, alerts, and positive reports from syndromic surveillance, can and should be used to evaluate the current (and proposed) versions of BioWatch and related systems and technologies.

Beginning in 2005, DHS sponsored development of a decision support tool called the Biological Warning and Incident Characterization (BWIC) system, which was pilot tested in three jurisdictions. The aim of BWIC is to integrate databases and several modeling programs into a common system for an emergency management team. A major component is the application of plume models to indicate where a release is likely to have occurred and what populations and areas are likely to have been exposed. It also includes a situational awareness tool to keep decision makers informed of estimates from analysts using the modeling components (Argonne National Laboratory, 2008). As tested, BWIC also supports monitoring routine operations, such as daily filter collection from BioWatch air samplers and reports from laboratory analysis of filters. It also incorporates scenarios to facilitate exercises and planning. Decision support and situational awareness tools such as BWIC may well aid state and local decision making following a BAR, but further development to address limitations identified during pilot testing is needed.

The committee urges action to improve tools for environmental characterization of the nature and extent of aerosol releases of biological agents in order to inform public health decision makers following a BAR. DHS should work with HHS (CDC), EPA, and the states and localities to ensure that the public health and law enforcement authorities in BioWatch jurisdictions have the capabilities to rapidly confirm and characterize an event that results in a BAR (event reconstruction) via environmental testing and other methodologies (e.g., plume modeling) to rapidly determine the area affected in order to target control measures effectively. Support is needed for work to identify, develop, and validate methodologies for specimen collection, laboratory analyses, and interpretation of environmental data. Examples of potential approaches to improving tools for incident characterization are shown in Box 3-1.

As new methodologies for incident characterization and event reconstruction are developed and validated, DHS and its BioWatch partners should also be furthering the development of computer support systems that can capture information as events are unfolding to maximize support for informed decision making. These decision support tools should reflect evidence and experience gained from event characterization. Because experience with BARs is currently limited to the detection of certain naturally occurring organisms, it will be important to understand whether or how this experience can be applied to the decision making that would follow a BAR for other biological agents.

With the declaration of a BAR, the planned procedure is for local public health officials to immediately notify both DHS and CDC, and for CDC to “provide public health consultation, support, and guidance to BioWatch jurisdictions regarding disease surveillance, laboratory issues, and/or environmental and occupational health” (DHS, 2008c, p. 11). Particularly in light of declining public health budgets (discussed in Chapter 4), BioWatch jurisdictions may find themselves in need of additional expert advice or information-gathering resources in planning for or responding to a BAR. In testimony to the committee, some public health officials noted the shortage of staff to carry out routine public health activities and the challenges posed by the need for additional information gathering following a BAR.

As the nation’s lead public health agency, CDC has a broad role in the preparedness and planning activities for biological threats and other hazards and should be working with all BioWatch jurisdictions to prepare for the response to a BAR. Specific areas where additional CDC assistance may be most useful include developing and ensuring the availability of resources such as the following:

• improved environmental sampling methods and laboratory assays for characterizing an incident after a BAR,

• criteria for each bioterror agent to help guide the decisions on if and when to start prophylaxis,

• guidance for decisions regarding initial targeting of prophylaxis if the extent of exposure is not yet clear,

• guidance for decisions regarding evacuation versus sheltering in place for the area(s) surrounding where the BAR was detected,

• additional templates for risk communication messages to the public and health care providers, and

• guidance on environmental risk assessment and remediation.

The lack of such guidance may have a crippling effect on decision making, or even on a jurisdiction’s willingness to participate in the BioWatch program. According to its annual performance reports, DHS did not reach its target for deploying additional air samplers in indoor locations in FY2007 and FY2008, in part, because of “some jurisdictions’ reluctance to employ bioaerosol collectors in indoor venues” without sufficient guidance on responding to an indoor detection of a biological agent (DHS, 2008a, p. 41, 2009d). The committee suggests as a model the guidance for responding to alerts from an automated system to detect B. anthracis in the workplace (CDC, 2004). This general guidance for employers, health departments, emergency responders, health care providers, and hospitals reflects collaboration between CDC, the U.S. Postal Service, and state and local officials to establish specific guidelines to accompany the installation of the Postal Service’s Biohazard Detection System.⁵

In addition to guidance documents and related resources, the committee also sees a need for public health departments and other key local responders (e.g., law enforcement, emergency management, and environmental protection agencies) to conduct tabletop or functional exercises in collaboration with CDC and the BioWatch program at least annually to assess and improve their alert response capacities. BioWatch jurisdictions also have a need for on-the-ground assistance from subject-matter experts at the time of a BAR. This assistance could take the form of a team of trained experts, assembled jointly by the BioWatch program and CDC, who could be invited to come to the aid of local officials. This team could provide assistance for planning and conducting periodic exercises designed to evaluate state and local preparedness to respond to BARs, and it could also be available to provide immediate assistance following an actual BAR. Such assistance would include guidance regarding critical issues related to estimating the population potentially exposed, risk communication, the initiation of antimicrobial prophylaxis or other countermeasures, determining the scope of environmental contamination, and the requirements for environmental remediation. This resource should be integrated with the local incident command.

Since the initial deployment of BioWatch in 2003, numerous exercises have been held by BioWatch jurisdictions to plan and prepare for a bioterrorist attack that might be detected by the BioWatch system. There have

also been a few dozen BARs declared to date, giving the jurisdictions where they occurred firsthand experience in contemplating the critical information needed after a BAR. When shared, jurisdictions’ experiences with exercises and BARs could be helpful to other communities and to DHS and the other federal BioWatch partners.

The committee sees the creation, sharing, and synthesis of information from after-action reports (AARs) on BARS as important to the continuing improvement of the BioWatch system. They can offer insights into the capabilities of and desired improvements to planning, operation, and response for the BioWatch system. These reports can also help identify opportunities to improve the future architecture and technical specifications of the BioWatch system and ways to improve the efficiency and effectiveness of response.

There is precedent for requiring such reports: the Pandemic and All-Hazards Preparedness Act of 2006 (P.L. 109-417) requires that localities receiving preparedness funding under cooperative agreements with HHS provide reports regarding strengths and weaknesses identified through preparedness exercises or drills (Sec 201). To enhance the generalizability of findings and ensure that lessons learned from each event are maximized, AARs could use Homeland Security Exercise and Evaluation Program (HSEEP) concepts and the federal Target Capabilities List (TCL) and the Universal Task List (UTL) (DHS, 2007a,b).⁶ Given both the burden and importance of preparing AARs, the committee urges DHS to provide funding to support their preparation rather than having this cost borne by BioWatch jurisdictions. Examples of the types of information that such BAR AARs could include are shown in Box 3-2.

The BioWatch program and its federal partners, including CDC, should jointly institute a systematic process for capturing, reviewing, and sharing the insights and recommendations of local, state, and federal officials involved in operating the BioWatch system. These reviews should address such issues as experiences in fielding BioWatch equipment, financial and other impacts of BioWatch operations on local jurisdictions, and key findings noted in after-action reports for BioWatch alerts or exercises. The reviews should identify common gaps as well as best practices in planning or conducting responses to BARs. Updates on lessons learned should be a standing agenda topic for an external BioWatch advisory panel (see Recom-

mendation 7), which can identify issues requiring program improvement and ensure that its recommendations are incorporated in the program’s continuous quality improvement plan. Lessons learned should also inform capacity-building and technical assistance programs for state and local public health departments participating in BioWatch.

The introduction of the BioWatch system into major metropolitan areas across the country has generated a variety of additional responsibili-

ties and obligations for state and local health departments. Although the guidance from the BioWatch program has matured over time, it continues to leave local health officials with limited insight into determining the appropriate public health actions following a BAR. CDC is a source of some assistance to local decision makers, but the coordination between the two federal agencies has not always been optimal. The committee sees a need for both the BioWatch program in DHS and relevant offices in CDC to provide better and more coordinated guidance to BioWatch jurisdictions.

In making the following recommendation and others in this chapter, the committee emphasizes an important caveat: The recommendations are made based on the assumption that the BioWatch technologies can be shown to perform satisfactorily and to produce useful data. Because certain critical testing is still needed for Generation 2, and has not yet been performed for Generation 3, there is uncertainty regarding the performance of the BioWatch system.

While the response to a BAR demands resources and expertise from a BioWatch jurisdiction, the day-to-day operations of BioWatch also impose costs. DHS pays for personnel and equipment for the laboratory assays, but localities provide space and personnel time for planning, training, and exercises. The committee received only limited information to evaluate the scale of these costs (e.g., Downes, 2008), but the data gathered suggest that they are modest relative to annual BioWatch program costs (see additional discussion in Chapter 2). The committee recommends that these day-to-day costs be assumed by DHS as part of the costs of operating the BioWatch program.

Using an autonomous detection system for the next generation of BioWatch technology is a potential means to reduce operating costs and improve timeliness of results. Automating the analysis of air samples within the field devices would eliminate the need for manual collection of filters and their transport to a laboratory for analysis, and reduce the time between agent release and detection.

Replacement of Generation 2 BioWatch samplers with continuously operated automated air samplers capable of sample collection and preparation, analysis, and data transfer as proposed for Generation 3 could resolve some problems that exist with the current system. Key features of the two systems are shown in Table 3-1.

Under routine operation, the BioWatch system currently produces test results once per day for samples collected 10 to 34 hours earlier. This window accounts for a 24-hour collection period and the time for sample collection, transport to a laboratory, and laboratory analysis. For Generation 3, the program is seeking to acquire devices that will be able to collect and test samples in the field and report results within 4 to 6 hours of sample collection. In principle, faster and more frequent testing provides the opportunity for earlier detection of a biological agent and therefore an earlier decision to begin a public health response, which may result in a significant increase in lives saved over a Generation 2 time line.

In addition to deployment of the autonomous detectors, the Generation 3 plans include increasing the number of jurisdictions participating in the BioWatch program, the number of monitoring devices in the participating jurisdictions, and the number of devices in indoor locations. Currently, a few hundred air samplers are deployed (DHS, 2008a). Plans for Generation 3 include deploying additional units (DHS, 2008b,d). Anticipated costs for Generation 3 are discussed in Chapter 2.

Since 2003, DHS has been pursuing an automated detection capability for BioWatch, aimed at reducing the time needed to obtain test results to within 4 to 6 hours of sample collection (HSARPA, 2003). When the BioWatch program was run by DHS S&T, development began for a system called the Bioagent Autonomous Networked Detector (BAND), which was anticipated to serve as the Generation 3 technology. After OHA was given operational responsibility for BioWatch, it announced plans to deploy an interim system—Generation 2.5—while final testing and acquisition of BAND units were completed for Generation 3 (Hooks, 2008b). DHS planned the interim system, referred to as the Autonomous Pathogen Detection System (APDS), because completion of work on BAND was progressing more slowly than expected, in part, because OHA issued revised specifications for Generation 3 in January 2008 (GAO, 2008a).

In fall 2008, DHS cancelled plans for Generation 2.5 because of delays in validation and production, and it revised plans for Generation 3. The change in DHS’s strategy for Generation 3 development and deployment has resulted in a new acquisition process, reflected in the release of a request for information, a request for proposals, and an operational requirements document (DHS, 2008b, 2009b,c). The specifications for the Generation 3 detection system call for a fully autonomous, networked biosensor that is capable of continuous monitoring (24 hours per day, 365 days per year) in indoor and outdoor environments for at least the pathogens currently included in the BioWatch program. Units are to collect and analyze samples at the sampling site and electronically report data on the results of the analysis of samples and of various self-tests (DHS, 2009c). Operational requirements have been set for the automated sample collection, preparation, analysis, preservation of samples, and waste handling.

Continued development of a program such as BioWatch is a long-term effort that must be approached more systematically than was possible in the program’s earliest days. Over many decades, the Department of Defense (DoD) has gained considerable experience in technology development and acquisition. It has developed an acquisition process that includes the generation of goals, measures to assess those goals, and test and evaluation methodologies to validate performance (Chadwick, 2007). Although DoD’s application of its system is not always successful (GAO, 2009), the committee sees the benefit of applying a similar systematic and structured approach for BioWatch. A formal acquisition plan for BioWatch would address factors such as costs (e.g., for operations, equipment, reagents, training, and response), technology update cycles, maintenance, and performance standards.

Development of a formal acquisition plan is particularly important because the BioWatch “user” community is almost exclusively external to DHS and composed primarily of state and local governmental entities (whereas DoD has the benefit of a defined set of users within the department). If a detailed plan for Generation 3 is shared with and incorporates input from end users, it may also encourage buy-in from the user community by displaying a level of rigor, detail, and partnership in planning and potential execution of the system that is currently lacking.

DHS plans to subject candidates for Generation 3 to T&E according to a Test and Evaluation Master Plan (TEMP) that was being finalized as this report was being written. To meet the proposed time line, potential

candidates must be at a Technology Readiness Level of 6 or higher⁷ to be accepted into the competition and must be deployable with 12 months (DHS, 2009a). Testing is scheduled to begin in mid-2009 and procurement in 2012. A number of participants will be involved in the process. A T&E workgroup is involved in developing and finalizing the TEMP. Units within DHS S&T will review the test plan and the test results for OHA. All stakeholders (i.e., CDC, EPA, FBI, state and local public health agencies) will be invited to participate in any or all parts of testing.

This proposed test plan demonstrates a welcome and substantive improvement in the BioWatch management approach, demonstrating an effort to apply more rigorous and systematic procedures to the development of BioWatch technology than were feasible during its initial hurried deployment. However, serious challenges remain in meeting the testing time line, overcoming technical risks, and providing for needed stakeholder involvement.

The T&E time line is too short. It does not allow for preproduction qualification tests or likely challenges to, or obstacles in, the development and testing process. The timing, duration, and location of the field test seem too limited to provide a basis for high confidence in intermediate technology decision points. The committee emphasizes that field testing needs to occur in various locations under a range of environmental conditions to which the detector will be subject under routine operation in order to gain representative results.

The technical risks of the acquisition and test plan for Generation 3 technology are too high. A first principle in technology acquisition is to assess technology maturity. It is unclear if DHS has adequately assessed the maturity of the technology needed for Generation 3 BioWatch. The committee saw no indication that DHS is planning to assess the environmental background levels of monitored pathogens in the BioWatch jurisdictions. This information could reveal whether environmental backgrounds will affect the performance of the biodetector assays. Also, the plan described to

the committee indicates that the information technology component of the system will be developed separately from the detector. This approach may produce obstacles in testing and challenges in integrating these components. In addition, the system’s automation component provides an opportunity for failure that could produce errors in sample analysis and electronic data exchange. Also of concern are the apparent plans to use equipment operators from the technology vendors or the national laboratories. The system should also be tested with representative operators from one or more BioWatch jurisdictions.

A requirement that will hamper development of the Generation 3 detector, but that has little operational relevance, is a false positive “rate” that is stated only as an exponential factor of 10, with no specification of the operational interval to which it applies (DHS, 2009c). In this form, it is difficult to interpret this performance requirement. But however it is interpreted, it represents an enormous T&E hurdle for obtaining statistically valid test data.

The assays that will be a critical feature of Generation 3 pose another technical challenge. New assays that are successful in the test panels are expected to be introduced into field testing for the Generation 3 system in mid-2009. Additionally, it appears that the requirements for sensitivity and selectivity are based on an estimation of capabilities of the existing analytical collection and assay technology, as opposed to being based on consideration of the infectious dose of the pathogens or other criteria that are based on the operational goals for the BioWatch system. The trade-offs between these two approaches for setting requirements are complex, with many uncertainties, but the latter needs to be taken into account in setting specifications for Generation 3.

An objective for Generation 3 is to increase the number of pathogen targets. This reinforces the need for validation of the assay component of the system. Moreover, the committee emphasizes that selection of biological agents for Generation 3 should be based on risk analyses and risk-management considerations to avoid inclusion of pathogens for which BioWatch is not an appropriate detection tool and unnecessary investment in related technology and assay development. Even if autonomous biological detectors are fielded, further laboratory analysis may be necessary to determine pathogen viability and antimicrobial sensitivity and for post-BAR environmental sampling.

The committee believes that the plans that DHS described for testing in an operational environment will not be sufficient to evaluate the efficacy of detector placement in a BioWatch jurisdiction. This experimental design issue is exacerbated by the fact that DHS does not appear to have a protocol for determining effective placement of detectors to ensure a high probability of sensing a bioterrorist attack. The committee urges that DHS make every

effort to arrange for the Generation 3 testing to include an experiment involving the release of simulant in a BioWatch jurisdiction to test and validate both siting models and system operation. Conducting such a test will require careful selection of appropriate simulants and close collaboration with a variety of officials in the jurisdiction. The operational testing may also need to consider the security of detector sites and appropriate tools (e.g., video monitoring) that can help assure local jurisdictions that a BAR is not the result of tampering or other interference with a detector.

DHS should consider whether, other than providing a more timely BAR, Generation 3’s automated process for generating a detection signal provides state and local public health officials with a more reliably “actionable” output than Generation 2’s manually detected genetic signature. In its development and selection of the Generation 3 system, DHS needs to be establishing with public health officials how the overall utility of Generation 3 outputs can be evaluated.

Stakeholder involvement is critical to the T&E process; unfortunately, the extent of stakeholder involvement described by DHS is too narrow. DHS indicates that it is reaching out to EPA, CDC, and FBI for very specific portions of the Generation 3 testing. It has also begun outreach efforts to some public health organizations. It appears, however, that state and local partners will be invited to observe the testing but will have no official role and no support from DHS to facilitate their participation.

To ensure that the goals of the test plan address public health needs, it is important that state and local public health officials be involved in the entire process. The absence of a formal means of involving these stakeholders gives the impression that DHS does not view their perspectives as a necessary and valuable component of the testing process. Failing to formally include (and fund the involvement of) state and local public health agencies risks missing the identification of faults that may be revealed during operational testing, such as problems in detector performance, shortcomings in operator training and operations protocols, or inadequacies in supporting agencies (e.g., public health laboratories or supply systems). The user community must have an active role as early as possible to have confidence in the system.

As the selection and deployment of Generation 3 move forward, DHS should conduct operational testing of BioWatch Generations 2 and 3 in representative jurisdictions, against operationally representative, open-air simulant releases using a method that might be employed by a terrorist in a limited number of sites (indoor and outdoor) and using representative operators and decision makers. This testing should be conducted before Generation 3 is widely deployed. Ideally, operational testing of Generation 2 should provide the extensive agent-specific performance characterization needed for development of Generation 3 requirements and to inform risk-management decisions in response to BioWatch results. Thorough operational testing of Generations 2 and 3 should include the development of readiness criteria that address the end user’s ability to interpret and use results from the system. The design of this testing should permit an evaluation of the entire system, including the following elements:

• the detector’s performance (the probability of a correct detection and identification within an acceptable time, rejection of background noise, and preservation of the sample to test for the viability and antimicrobial sensitivity of the detected pathogen), reliability, and supportability (setup, operation, maintenance, and repair);

• the effectiveness of the BioWatch training program for its operators;

• the ability of the BioWatch logistics system to provide adequate support to the operators and end users; and

• the suitability of BioWatch data to support low- and high-regret response decisions by local public health officials for the full spectrum of agents and bounding scenarios.

Operational test results should be evaluated against measures of effectiveness that are collaboratively developed by the BioWatch program office and the public health community. Possibilities include sensitivity and specificity, pathogens detected, timeliness, and coverage.

Given the technical risks that exist in the acquisition and test plan for Generation 3, DHS would benefit from an external technical review entity that could assess the technical risks of the candidate technologies and aid in the development of criteria for decision making at critical decision points. A subsequent recommendation (Recommendation 7) proposes such an advisory group.

If BioWatch Generation 3 technology can be successfully developed and deployed, it could facilitate greater population coverage, aid in achieving an earlier initiation of a public health response to an attack, and reduce per-unit operating costs by eliminating filter transport and manual analysis. However, it is not apparent that science and technology exist that can be engineered into a functional detector that meets, even marginally, the proposed performance requirements for Generation 3.

Development of a field-deployable autonomous biological aerosol threat detector that can meet appropriate performance requirements has been and continues to be a major challenge of biodefense. Over the past several decades, beginning with the DoD XM19 biodetection system that was pursued in the early 1980s, hundreds of millions of dollars have been spent in efforts to develop a functional biodetector, and the challenge remains. Yet the plans for the Generation 3 BioWatch system depend on the successful development and fielding of an automated biodetector system, both outdoors and indoors, within a relatively short period of time.

The technical challenges associated with developing a successful detector for the BioWatch program are great. Several deserve specific note.

• Distinguishing between target pathogens and near neighbors: It is estimated that only a small fraction of the microbial world has been identified, let alone cultured, characterized, and genetically sequenced. While the functional (phenotypic) effects of a human pathogen are dramatically different from a nonpathogenic near neighbor, their molecular signatures can be remarkably similar.

• Impact of the operational environment: The outdoor operational environment (e.g., temperature and humidity extremes) and the need for

a detector to function without being serviced for extended periods of time put a great engineering burden on the device.

• Coverage and cost trade-offs: The BioWatch program is emphasizing coverage of major metropolitan areas and within them, locations and special events that bring large numbers of people together. But within jurisdictions, population densities and distributions vary between workday and evening and workday and weekend. Achieving desired population coverage will require fielding many samplers and thus the costs of a system’s capital equipment and operations must be extremely low to be affordable.

• Uncertainty about the characteristics of airborne biological threats: The BioWatch program currently focuses on certain specific pathogens, but the scope of potential aerosol biothreats is great. Future bioterrorism threats may include engineered agents, and recent experience has demonstrated the threat from emerging respiratory pathogens (e.g., Legionnaires’ disease). Ideally, a biodetector should be readily upgradable to address novel naturally occurring and engineered biothreats.

These challenges to development of the Generation 3 BioWatch detector underscore the need for significant investments in early applied research leading to transformational innovations. Significant advances in several diverse scientific areas hold promise of converging to achieve development of the required biodetector. For example, developments in engineering research may permit creation of more robust and inexpensive detector platforms, and new communication systems and network architectures will support “smart” networks.

Opportunities also exist to refine the analysis of BioWatch air samples, but an important prerequisite is the characterization of endemic environmental organisms in and near BioWatch jurisdictions to establish background levels of both the pathogens of concern and near relatives that might cross-react in assays. With high-throughput sequencing technology rapidly filling in the genomic microbial tree of life, it is possible to refine the nucleic acid assays used in the BioWatch system to optimize their sensitivity and specificity for detection of the particular pathogens of interest. In addition, studies supported by several federal agencies are rapidly establishing relationships between organisms’ molecular information (e.g., genomics, proteomics) and their pathogenicity. With this knowledge, chemical analysis of aerosol samples may provide a better basis for judging the health hazard associated with a BAR.

Significant logistical challenges need to be resolved before full fielding of the Generation 3 system. Currently, the BioWatch program has a few

hundred samplers deployed (DHS, 2008a), but intends to deploy a greater number for Generation 3. The BioWatch program, in close coordination with HHS, has to ensure that the entire BioWatch system has sufficient resources. This includes not only the air sampling equipment and laboratory analysis, but resources for all associated maintenance and upgrade capabilities, doctrine and technical procedures, the initial and sustainment training base, laboratory facilities (and their safety systems), courier services, communications and computer systems, trained staff and leadership, and a fully resourced public health response capability. Even though it may be reasonable to assume that the BioWatch Generation 3 system will alleviate some of the current analytical and logistical burdens, it is equally reasonable to assume that its complex, automated detectors will require new, specialized technical and maintenance support and personnel with appropriate training to interpret the results it produces. The total system architecture must account for all of these system components, and program budgeting must be adequate to support all aspects of the system.

The availability of a Generation 3 autonomous detector has the potential to provide a major advance in capability and logistics of the BioWatch system. But DHS should heed the scale of the challenge, as signaled by the considerable investments by DoD and others—with only limited progress—toward the goal of an automated detector for bioaerosol threats. Further progress in the development and deployment of the Generation 3 system for BioWatch should benefit from a strong and effective collaboration within DHS between OHA and S&T. OHA should ensure that operational needs and concerns from BioWatch jurisdictions and federal BioWatch partners are brought to the development effort, while S&T should ensure that Generation 3 development responds to both technological opportunities and hazards.

The instruments and knowledge currently used in the BioWatch system have origins in and are built upon previous research in academic, industrial, and governmental institutions, supported by a wide range of agencies. For example, the APDS system that was originally proposed for use in a Genera-

tion 2.5 deployment contains components and concepts from flow injection and segmented flow analysis introduced originally for high-speed analysis of large sample populations (Bergamin et al., 1978; Ruzicka and Hansen, 1978, 1988; Ruzicka and Guebeli, 1991); microfluidic analysis system concepts introduced for multiplex analysis based on mixtures of functionalized polymer beads, each containing unique capture/receptor chemistry (Manz et al., 1992; Harrison et al., 1993; Jacobson et al., 1994a,b); fluorescence labeling to detect unique hybridization reactions which was a part of the genome sequencing chemistry (Hunkapiller et al., 1991); information technology systems; and nucleic acid amplification chemistry (Saiki et al., 1985). A foundation of support for research and the resulting advances in knowledge and technology are crucial to maintain the essential science base that will support development of next generation biothreat reduction capabilities.

A sustained research and development effort is needed to provide the scientific and technological knowledge required for effective and sustainable outdoor and indoor environmental biosurveillance in urban environments. The committee encourages DHS to work with other research funding agencies (e.g., the National Science Foundation, DoD, EPA, and HHS) to coordinate and leverage investments that will contribute to the development of less expensive and more capable environmental biosurveillance systems, including contributing to the knowledge and innovation required to engineer an autonomous environmental biodetection system.

In particular, DHS should collaborate with DoD and the National Institutes of Health to establish an applied research program to advance the state of science needed for development of an autonomous, field-deployable detector with capabilities to meet operational requirements for BioWatch Generation 3 and beyond. Research is also needed to improve the knowledge base for interpreting surveillance results, developing techniques for addressing unknown threats (i.e., emerging, re-emerging, and engineered biothreats), and applying environmental monitoring systems to surveillance for natural disease (e.g., monitoring for Legionella) as well as biodefense to make them more cost-effective.

DHS should also consider participation in the work being done by DoD, the National Institute of Allergy and Infectious Diseases in HHS, and the World Health Organization on host–pathogen interactions, surveillance, and epidemiologic research investigations and, as part of that effort, establish shared databases to consolidate information of value to the work of all the participants. Through research on host–pathogen interactions it may prove possible to identify genetic or molecular markers that signal virulence or antimicrobial resistance in pathogens or increased vulnerability in human hosts. This kind of information may make it possible to devise more informative detection techniques that would help public health officials maximize the effectiveness of their response plans.

A research program could consider including the BioWatch jurisdictions as study sites in a program to characterize the microbial ecology in urban areas. Objectives would include characterization of the geographic and temporal distribution of pathogen and near-neighbor populations in air and natural reservoirs (e.g., soil, lakes, and sanitary waste facilities). The BioWatch air samplers currently in use have capabilities that may be useful for such research. One priority should be full characterization of samples from BARs, but examination of the large number of other BioWatch samples may also be informative.

Results of the ongoing characterization effort and BAR follow-on studies would be included in a central database for use in the design of bioassay signatures and interpretation of BAR results. This central database could be shared with bioforensic databases, thereby bringing together knowledge derived from biosurveillance and bioforensic programs. Seroprevalence studies could help to understand current human exposures to these endemic organisms.⁸ To the extent that the currently deployed BioWatch system can contribute to such research efforts, it should be seen as a resource to inform and enlarge the science and technology base from which new generations of BioWatch technologies will be derived.

As with other enhancements to BioWatch proposed by the committee, the costs and benefits of various research and development activities have to be evaluated and prioritized against the range of demands on DHS and BioWatch program resources.

The preceding sections have described a series of needs and challenges facing DHS and the BioWatch program as it moves forward. In doing so, the program should be guided by strategic planning based upon program missions, goals, and objectives. The committee was very interested in whether DHS has documented overall programmatic goals, such as the number of cities (or people) the BioWatch program is intended to protect, the number of agents to detect, or the desired probabilities of detection of various kinds of aerosol releases. In response to a committee request, DHS (2008d) stated the goal noted in Chapter 2: “to establish and operate a bioaerosol monitoring capability to accurately detect the release of biological threat agents of greatest concern to the nation in locations that are at greatest risk of catastrophic consequences and to enable timely response and mitigation.” Although this goal provides a high-level focus for the program, it does not provide the detail needed, or identify the spectrum of capacities required, for development, operation, and continuous improvement of such a complex activity involving diverse stakeholder groups.

The relatively weak documentation and statement of the program’s goals—and associated performance metrics—may well be the result of the short time frame for the initial deployment of the BioWatch system and the infancy of DHS as a federal agency. Now, with several years of operational experience, a more mature program in a more mature agency requires more rigor in program planning and a better understanding of the goals for not only technological performance but overall system performance, including effective engagement of key partners in states and localities where the BioWatch system is deployed.

For a system to meet user needs, the users should participate in determining the goals and performance metrics. The BioWatch program management has apparently placed more emphasis on the “early detection” aspect of a bioaerosol monitoring capability than on “enabling timely response and mitigation.” Facilitating response and mitigation requires cooperation among DHS, public health, and other federal partners and the integration and assessment of many different types of information. The concerns of local

decision makers, who will have to act in response to a BAR, must be taken into account to validate performance measures and program scope, enhance confidence in BioWatch results, and achieve buy-in from key partners. State and local government officials, including public health representatives, should be involved in decisions regarding all stages of BioWatch planning, deployment, maintenance, and upgrades within their jurisdictions.

The committee recognizes that DHS does plan a more concerted effort to improve the relationship between the BioWatch program’s management and its federal, state, and local partners. DHS should produce a formal acknowledgement of this commitment and follow through with its implementation.

The BioWatch program needs both a stronger information base and better and more systematic input, analysis, and evaluation on an ongoing basis. The committee is encouraged by the more systematic and technically rigorous approach to planning and technical evaluation proposed for Generation 3. DHS should also take this opportunity to incorporate risk-assessment and risk-management approaches into its planning for the BioWatch program. Such an evaluation should include a reexamination of the agents to include in the monitoring program, the selection of jurisdictions, the location of air samplers or detectors within jurisdictions, and even the justification for the existence of the program. In the context of the program’s goals, analytical tools should be brought to bear to aid in decision making each time DHS considers a significant change regarding a technology or process.

In particular, DHS should make use of its biennial Bioterrorism Risk Assessment (BTRA) in assessing the BioWatch system. But the concerns about the 2006 BTRA that were noted in a recent NRC report should be resolved (NRC, 2008b). Using an improved BTRA, DHS should model scenarios and assess risks, and apply this information in its decision making about, and continuous evaluation of, the BioWatch program. Indeed, DHS appears to have applied these principles in its initial decision making in planning for the demonstration of the APDS (Hooks, 2008a).

Such an approach might include an examination of the most effective applications of the BioWatch system. DHS might, for example, consider whether special event monitoring would be more effective than constant monitoring, or whether focusing on transportation hubs would be better than attempting comprehensive coverage of localities. Any plans to add biological agents to the set to be monitored by the BioWatch system should include an assessment of whether analytical challenges will be encountered because of natural background levels of these biological agent signatures or the presence of other interferents. The committee did not receive evidence that DHS has factored such analyses into most decision making for the BioWatch program.

Finally, just as the committee advocates rigorous analysis of the costs and benefits of existing plans for BioWatch 2 and 3, it also advocates that any further work based on the recommendations in this report, or through other research and development efforts, be subjected to thorough scrutiny of the associated costs and benefits.

In the committee’s view, the BioWatch program has suffered from too little input from outside experts who can bring broader perspectives to bear. Obtaining advice from an independent panel, consisting of members with a mix of technical and operational expertise, would enhance the BioWatch program. Such an external advisory panel would provide specific analyses and recommendations on the BioWatch program’s technology upgrades, planning, and operations. Panel members could also receive, and relay back to their constituencies, information about pertinent research and development activities.

The advisory panel should bring to DHS expertise in epidemiology, environmental health, public health laboratory systems, meteorology, infectious diseases, biochemistry, genetics, law enforcement, emergency management, detection technology, systems engineering, decision and information science, and operations research. Among the members should be state and local officials responsible for responding to a BAR.

To further coordination and communication between DHS and HHS, the advisory panel should report jointly to both secretaries. The committee anticipates that the input from the advisory panel would lead to improved collabo-

ration among the state- or local-level constituents of the lead federal agencies, which would improve working relationships between those responsible for implementing environmental sampling technologies and those responsible for acting on the information that such sampling produces. It could also lead to better communication at all stages in the BioWatch implementation process, including selection of participating cities, identification of monitoring sites, ongoing program operation, and response planning and evaluation.

The BioWatch program is already benefiting from the work of some advisory groups. The BioWatch Technical Advisory Committee (BTAC), for example, consists of federal experts with backgrounds in microbiology and engineering from DHS (OHA and S&T) and CDC who provide DHS with technology input. But this group lacks sufficient breadth of expertise and the external perspectives that the BioWatch program needs. DHS also benefits from the work of the Stakeholder Panel on Agents for Detection Assays (SPADA), which is developing voluntary consensus standards for the collection and analysis of environmental samples of biothreat agents.⁹ DHS and HHS need such external expert input for all aspects of the BioWatch system.

AOAC (Association of Analytical Communities). 2008. Much accomplished, much to do: SPADA update. http://www.aoac.org/ILM/jan_feb_08/spada.htm (accessed April 8, 2009).

Argonne National Laboratory. 2008. Software package coordinates response to biological threats. http://www.dis.anl.gov/publications/fact_sheets/BWIC_FactSheet.pdf (accessed June 17, 2009).

Bergamin, H., E.A. Zagatto, F.J. Krug, and B.F. Reis. 1978. Merging zones in flow injection analysis: Part 1. Double proportional injector and reagent consumption. Analytica Chimica Acta 101(1):17–23.

Brown, M.J., M.D. Williams, G.E. Streit, M. Nelson, and S. Linger. 2006. An operational event reconstruction tool: Source inversion for biological agent detectors. 86th American Meteorological Society Annual Meeting. Atlanta, GA. http://ams.confex.com/ams/pdfpapers/126686.pdf (accessed July 9, 2009).

CDC (Centers for Disease Control and Prevention). 2002. Comprehensive procedures for collecting environmental samples for culturing Bacillus anthracis. http://emergency.cdc.gov/agent/anthrax/environmental-sampling-apr2002.asp (accessed July 9, 2009).

CDC. 2004. Responding to detection of aerosolized Bacillus anthracis by autonomous detection systems in the workplace. Morbidity and Mortality Weekly Report 53(RR-7):1–11.

Chadwick, S.H. 2007. Defense acquisition: Overview, issues, and options for Congress. RL34026. Washington, DC: Congressional Research Service. https://www.policyarchive.org/bitstream/handle/10207/18699/RL34026_20070604.pdf?sequence=2 (accessed August 5, 2009).

Chow, F.K., B. Kosovič, and S. Chan. 2008. Source inversion for contaminant plume dispersion in urban environments using building-resolving simulations. Journal of Applied Meteorology and Climatology 47:1553–1572.

Delle Monache, L., J.K. Lundquist, B. Kosovič, G. Johannesson, K.M. Dyer, R.D. Aines, F.K. Chow, R.D. Belles, W.G. Hanley, S.C. Larsen, G.A. Loosmore, J.J. Nitao, G.A. Sugiyama, and P.J. Vogt. 2008. Bayesian inference and Markov Chain Monte Carlo sampling to reconstruct a contaminant source on a continental scale. Journal of Applied Meteorology and Climatology 47:2600–2613.

DHS (Department of Homeland Security). 2007a. Homeland security exercise and evaluation program. Vol. III: Exercise evaluation and improvement planning. Washington, DC: DHS. https://hseep.dhs.gov/support/VolumeIII.pdf (accessed July 1, 2009).

DHS. 2007b. Target capabilities list: A companion to the National Preparedness Guidelines. Washington, DC: DHS. https://www.llis.dhs.gov/docdetails/details.do?contentID=26724 (accessed July 1, 2009).

DHS. 2008a. Annual performance report, fiscal years 2007–2009. Washington, DC: DHS. http://www.dhs.gov/xlibrary/assets/cfo_apr_fy2007.pdf (accessed August 5, 2009).

DHS. 2008b. BioWatch Gen-3 autonomous detection system: Operational requirements document. December 11. Washington, DC.

DHS. 2008c. BioWatch outdoor program: Guidance documents for BioWatch jurisdictions. Washington, DC: DHS.

DHS. 2008d. NAS information request #2, DHS OHA written responses. Document submitted to the Committee on Effectiveness of National Biosurveillance Systems: BioWatch and the Public Health System, October 31, Washington, DC.

DHS. 2009a. BioWatch Gen-3 industry day. https://www.fbo.gov/download/18d/18d10ae7b9c2106327715c7a5dc53c43/BiowatchGen3_Industry-Day_Slides.pdf (accessed March 24, 2009).

DHS. 2009b. BioWatch Gen-3 request for proposal: HSHQDC-09-R-00045. https://www.fbo.gov/download/54e/54e9ae25a4a9d2d39a9e9e836955001f/HSHQDC-09-R-00045_(5-27-2009).pdf (accessed August 5, 2009).

DHS. 2009c. BioWatch Gen-3—Request for proposals: HSHQDC-09-R-00045. Attachments J.1–J.5 and J.7–J.9. Washington, DC: DHS. https://www.fbo.gov/download/c38/c38d6b04219a25945582398067211499/HSHQDC-09-R-00045_(Attachments_J_1__J_2__J_3__J_4__J_5__J_7__J_8__J_9).pdf (accessed August 5, 2009).

DHS. 2009d. Department of Homeland Security annual performance report, fiscal years 2008–2010. Washington, DC: DHS. http://www.dhs.gov/xlibrary/assets/cfo_apr_fy2008.pdf (accessed June 17, 2009).

DoD (Department of Defense). 2006. Defense acquisition guidebook. http://www.acq.osd.mil/sse/pg/guidance.html (accessed July 9, 2009).

Downes, F.P. 2008. Testimony before the Committee on Homeland Security, U.S. House of Representatives, July 16. http://homeland.house.gov/SiteDocuments/20080716143630-34598.pdf (accessed June 18, 2009).

Dvorak, P. 2005. Health officials vigilant for illness after sensors detect bacteria on Mall. The Washington Post, Sunday, October 2, C13. http://www.washingtonpost.com/wp-dyn/content/article/2005/10/01/AR2005100101209_pf.html (accessed April 21, 2009).

EPA (Environmental Protection Agency). 2008. Standardized analytical methods for environmental restoration following homeland security events. Revision 4.0. EPA/600/R-04/126D. Cincinnati, OH: EPA. http://www.epa.gov/nhsrc/pubs/600r04126d.pdf (accessed July 10, 2009).

GAO (Government Accountability Office). 2007. Anthrax detection: DHS cannot ensure that sampling activities will be validated. GAO-07-687T. Washington, DC: GAO. http://www.gao.gov/new.items/d07687t.pdf (accessed August 5, 2009).

GAO. 2008a. Biosurveillance: Preliminary observations on Department of Homeland Security’s biosurveillance initiatives. GAO-08-960T. Washington, DC: GAO. http://www.gao.gov/new.items/d08960t.pdf (accessed August 5, 2009).

GAO. 2008b. Homeland security: First responders’ ability to detect and model hazardous releases in urban areas is significantly limited. GAO-08-180. Washington, DC: GAO. http://www.gao.gov/products/GAO-08-180 (accessed July 9, 2009).

GAO. 2009. Defense acquisitions: Assessments of selected weapon programs. GAO-09-326SP. Washington, DC: GAO. http://www.gao.gov/new.items/d09326sp.pdf (accessed August 5, 2009).

Harrison, D.J., K. Fluri, K. Seiler, Z. Fan, C.S. Effenhauser, and A. Manz. 1993. Micro-machining a miniaturized capillary electrophoresis-based chemical analysis system on a chip. Science 261(5123):895–897.

Hooks, R. 2008a. APDS block-1 deployment strategy. Document submitted to the Committee on Effectiveness of National Biosurveillance Systems: BioWatch and the Public Health System, September 19, Washington, DC.

Hooks, R. 2008b. BioWatch program. Presentation to the Committee on Effectiveness of National Biosurveillance Systems: BioWatch and the Public Health System, Meeting 1, July 30–31. Washington, DC.

HSARPA (Homeland Security Advanced Research Projects Agency). 2003. Detection systems for biological and chemical countermeasures. RA 03-01. Washington, DC: Department of Homeland Security. https://baa.st.dhs.gov/Solicitations/HSARPA_RA-03-01_Body.pdf (accessed July 16, 2009).

Hunkapiller, T., R.J. Kaiser, B.F. Koop, and L. Hood. 1991. Large-scale and automated DNA sequence determination. Science 254(5028):59–67.

IEc (Industrial Economics, Incorporated). 2009. Revised estimated cost of BioWatch generations 2 and 3. Commissioned paper. Committee on Effectiveness of National Biosurveillance Systems: BioWatch and the Public Health System, Washington, DC.

Jacobson, S.C., R. Hergenrider, L.B. Koutny, R.J. Warmack, and J.M. Ramsey. 1994a. Effects of injection schemes and column geometry on the performance of microchip electrophoresis devices. Analytical Chemistry 66(7):1107–1113.

Jacobson, S.C., L.B. Koutny, R. Hergenroeder, A.W. Moore, and J.M. Ramsey. 1994b. Microchip capillary electrophoresis with an integrated postcolumn reactor. Analytical Chemistry 66(20):3472–3476.

Keats, A., E. Yee, and F.-S. Lien. 2007. Bayesian inference for source determination with applications to a complex urban environment. Atmospheric Environment 41(3):465–479.

Kuske, C. 2005. Analysis of pathogen backgrounds in soils and EPA aerosol samples. Report No. LA-UR-04-3191. Los Alamos, NM: Los Alamos National Laboratory.

Kuske, C.R., S.M. Barnes, C.C. Grow, L. Merrill, and J. Dunbar. 2006. Environmental survey for four pathogenic bacteria and closely related species using phylogenetic and functional genes. Journal of Forensic Science 51(3):548–558.

Lindley, C. 2009. BioWatch actionable result response: 2008 Democratic National Convention, August 28, 2008. Document submitted to the Committee on Effectiveness of National Biosurveillance Systems: BioWatch and the Public Health System, February 19, Washington, DC.

Manz, A., D.J. Harrison, E.M. Verpoorte, J.C. Fettinger, A. Paulus, H. Ludi, and H.M. Widmer. 1992. Capillary electrophoresis and sample injection systems integrated on a planar glass chip. Journal of Chromatography 593(1–2):253–258.

NRC (National Research Council). 2008a. A framework for assessing the health hazard posed by bioaerosols. Washington, DC: The National Academies Press.

NRC. 2008b. Department of Homeland Security Bioterrorism Risk Assessment: A call for change. Washington, DC: The National Academies Press.

NRC. 2008c. Test and evaluation of biological standoff detection systems. Washington, DC: The National Academies Press.

Rao, K.S. 2007. Source estimation methods for atmospheric dispersion. Atmospheric Environment 41:6964–6973.

Ruzicka, J., and T. Guebeli. 1991. Principles of stopped-flow sequential injection analysis and its application to the kinetic determination of traces of a proteolytic enzyme. Analytical Chemistry 63(17):1680–1685.

Ruzicka, J., and E.H. Hansen. 1978. Flow injection analysis: Part X. Theory, techniques and trends. Analytica Chimica Acta 99(1):37–76.

Ruzicka, J., and E.H. Hansen. 1988. Flow injection analysis, 2nd ed. New York: Wiley & Sons.

Saiki, R.K, S. Scharf, F. Faloona, K.B. Mullis, G.T. Horn, H.A. Erlich, and N. Arnheim. 1985. Enzymatic amplification of beta-globin genomic sequences and restriction site analysis for diagnosis of sickle cell anemia. Science 230(4732):1350–1354.

Senocak, I., N.W. Hengartner, M.B. Short, and W.B. Daniel. 2008. Stochastic event reconstruction of atmospheric contaminant dispersion using Bayesian inference. Atmospheric Environment 42:7718–7727.

Yee, E. 2008. Theory for reconstruction of an unknown number of contaminant sources using probabilistic inference. Boundary-Layer Meteorology 127:359–394.