Bulk Explosives Detection
In this chapter the FAA's progress in deploying bulk explosives-detection equipment and the operational performance of this equipment are evaluated. Bulk explosives-detection equipment includes any device or system that remotely senses a physical or chemical property (or combination thereof) of an object in an attempt to detect (semiquantitatively) the presence of an explosive concealed in a container (e.g., passenger baggage). Arguably, bulk explosives-detection equipment is the foundation of the TAAS in the current environment. The critical performance metrics for explosives-detection equipment include probability of detection (Pd), the probability of false alarm (Pfa), and throughput rate. At the same time, this equipment must function without unreasonably interrupting passenger flow.
An FAA-certified EDS (explosives-detection system) is a self-contained unit (composed of one or more integrated devices) that has passed the FAA's certification test. To obtain certification, equipment must meet the following standards:
• The detection rates against various types of explosives contained in baggage must have an overall Pd of no less than X1
• The Pfa (as determined for airline-type baggage) must not exceed Y.
• The baggage throughput rate of the equipment must meet or exceed 450 bags per hour.
A number of nuclear and x-ray-based techniques have been investigated, but only three systems (all based on x-ray computed tomography [CT]) have been certified.
Application of Bulk Explosives-Detection Equipment to Possible Threat Vectors
Current passenger-screening requirements were developed in 1972 in response to an increase in hijackings (NRC, 1996b). Current passenger-screening procedures involve metaldetector portals that can detect only metallic weapons. Emerging imaging technologies, however, can detect the presence of both metallic and nonmetallic weapons, as well as explosives concealed under multiple layers of clothing (NRC, 1996b). These technologies, which include passive and active millimeter-wave imaging and active x-ray imaging, all require human operators to view and interpret the images. Because the images are somewhat revealing of the human anatomy, passengers are likely to object to the images being displayed to an operator. Furthermore, active imaging techniques require radiation, which raises some health concerns. Because of these concerns about privacy and health, imaging technologies will probably not be deployed for passenger screening at the current threat level (NRC, 1996b).
At present, all carry-on baggage is screened by conventional x-ray radiography. Manufacturers of explosives-detection equipment are working on new technologies that are just being evaluated by the FAA. Operator-assisted x-ray, for example, highlights areas in a radiographic image that could be a threat object (e.g., a weapon or bomb); the highlighted image is then evaluated by the operator (Polillo, 1998). Because this technology is in its infancy and has not been widely deployed, not enough data were available for the panel to evaluate its performance. Nuclear quadrupole resonance is another possible technology for this application.
1 The actual values of X and Y are classified and can be found in classified FAA reports (FAA, 1992).
However, to date, the range of explosives this technology can detect is limited.
Because checked baggage is the threat vector that has received the most attention from the FAA, bulk explosives-detection equipment has become the most critical component of the TAAS by default. To date, three explosives-detection systems have been certified by the FAA, all x-ray CT-based systems manufactured by In Vision Technologies, Inc. (CTX-5000, CTX-5000 SP, and CTX5500 DS). Other vendors have developed bulk explosives-detection equipment, but none of them has passed certification testing. In addition to the FAA-certified In Vision CTX series EDSs, some noncertified explosives-detection devices have been selected for deployment, including Vivid, EG&G Z-Scan, and Heimann devices. The discussion in this chapter focuses predominantly on findings pertaining to the In Vision CTX series EDSs and the available data on the performance of deployed units.
Deployed Bulk Explosives-Detection Equipment2
Based on the recommendations of the White House Commission on Aviation Safety and Security (1996, 1997), Congress mandated the deployment of bulk explosives-detection equipment in U.S. airports. In late 1996, the FAA associate administrator for civil aviation security instructed the SEIPT to assess the availability of explosives-detection equipment for deployment in commercial airports, develop a deployment strategy and plan, and execute the plan. The plan called for the deployment of the FAA-certified In Vision CTX-50003 series systems, as well as deployment of socalled advanced technology (AT) hardware, namely eight Vivid devices, 10 EG&G Z-SCAN devices, two In Vision/Quantum Magnetics Q-Scan devices, and two Heimann devices by December 1997. The first In Vision CTX-5000 was deployed in January 1997, and two to 10 deployments were planned for every month during 1997. The FAA/SEIPT's progress as of January 1, 1999, is summarized in Table 6-1; the locations of the equipment in airports are summarized in Table 6-2.
The installations are classified into two types, stand-alone and integrated. Stand-alone installations are divided into four types based on their location: ticket counter, lobby, baggage area, and security area. Integrated installations are divided into two types, output and fully integrated installations. Output installations are configured to run directly into the baggage-handling system and can be located in the lobby, at the ticket counter, or curbside. Fully integrated EDSs are completely integrated (i.e., both input and output) into the baggage-handling system.
Installation costs are dependent on the type of installation and whether the installations require major modifications. For example, the lobby installation of a stand-alone CTX-5000 SP that does not require major airport modifications costs on the order of $10,000 to $30,000. However, a stand-alone lobby installation that requires major modifications, such as reinforcing the floor, cement work, or moving staircases or other structures, can raise the cost to $100,000 or $200,000. For more difficult integrated installations at existing airport terminals, the costs can be even higher. In the United Kingdom, fully integrated CTX-5000 installations account for 25 to 50 percent of the cost of the whole baggage-handling infrastructure. The British Airport Authority has determined that, in general, every $2 million spent installing CTX-5000 EDSs will require approximately $6 million to complete the modifications to the baggage-handling infrastructure. Note that this installation cost is not just relevant to CTX-5000 EDSs. The cost of installation (due to modifications to the baggage-handling infrastructure) of explosives-detection equipment of similar size and weight (e.g., EG&G, Vivid, and Heimann) would probably be comparable.
Most of the bulk explosives-detection equipment that has been deployed, or is being considered for deployment, is x-ray basedthe exception being the In Vision/Quantum Magnetics Q-Scan, which is based on a nuclear quadrupole resonance measurement technique. The x-ray based technologies that have been deployedor are scheduled to be deployedinclude transmission x-ray, dual energy x-ray, and CT. Of these, only the CT-based technology has passed the FAA bulk explosives-detection certification test. The others were selected because they have large enough apertures to handle oversized bags, and operational data would be useful. The performance baseline for the noncertified explosives-detection devices has been determined at the FAA Technical Center. The sole electromagnetic instrument scheduled for deployment, developed by Quantum Magnetics, Inc. (recently Quantum Magnetics was bought out by In Vision, Inc.), is a nonimaging technique based on nuclear quadrupole resonance (NQR). The conditions specified by the FAA for use of the deployed equipment are shown in Box 6-1.
2 Note that bulk explosives-detection equipment predominantly addresses the threat of an explosive device being brought aboard an airplane via checked baggage. The current deployment of this equipment does not address other threat vectors, such as a passenger carrying explosives on his or her person, carry-on baggage, cargo, mail, or catered food.
3 This number includes one CTX-5000 already deployed at San Francisco International Airport and two CTX-5000s deployed at Atlanta's Hartsfield International Airport for previous operational testing.
To date, most of the performance data on deployed explosives-detection equipment have been generated from tests conducted at the FAA Technical Center. However, operational test data on Pfa (false-alarm rates) are also reviewed in this section.
Test Data from the FAA Technical Center
Most performance data for x-ray CT-based EDSs are from certification testing and operational testing of the In Vision CTX-5000 SP. Data on the performance of the other FAA-certified EDSs and from the recent certification testing of In Vision CTX-5500 DS and L3 Communications 3DX-6000 were not available at the time this report was written. The In Vision CTX-5500 DS has been certified for two different inspection modes: SURE98 mode and CERT98 mode. In SURE98 mode, it has a lower Pfa but also a lower throughput rate than the CTX-5000 SP. In CERT98 mode, it has a similar Pfa to the CTX-5000 SP but a much higher throughput rate. The panel's analysis of performance data focuses on the certified CTX-5000 SP and CTX-5500 DS, although some data on other deployed explosives-detection equipment are also presented. Table 6-1 shows the performance factors for deployed explosives-detection equipment, including the Pd' Pfa' and the bag throughput rate. Because the actual Pd numbers are classified,4 Pd is given as a percentage of the overall Pd required for certification (X), and Pfa is given as a percentage of the Pfa required for certification (Y). The In Vision/Quantum Magnetics Q-Scan has not been tested at the FAA Technical Center.
Operational Test Data
In 1995, the FAA initiated the Airport Operational Demonstration Project to determine the operational performance of the In Vision CTX-5000 SP in the field as compared to its performance in certification testing (FAA, 1995). Three sites were selected for the project: San Francisco International Airport (United Airlines); Atlanta's Hartsfield International Airport (Delta Airlines); and Manila International Airport (Northwest Airlines). This operational demonstration project was not initially related to the congressionally mandated deployment of explosives-detection equipment. Recently, however, the FAA decided to include the CTX-5000s installed during the operational demonstration project in the overall deployment.
Two In Vision CTX-5000s were installed in Atlanta and one each in San Francisco and Manila. The demonstration project included four open tests and one blind (so-called ''red team") test using improvised explosives devices (IEDs) to determine Pd. The Pfa was measured routinely throughout the project on real passenger bags. Only the data from the San Francisco and Atlanta deployments have been documented in final reports (FAA, 1997a, 1997b, 1997c). Some of the performance data from San Francisco International Airport are given in Table 6-3.
The automated explosives-detection capability of the CTX-5000 SP in the field (% X = 102) was about the same as the capability measured during laboratory testing (% X = 106) at the FAA Technical Center. However, operator intervention to resolve alarms measurably reduced the overall Pd. This tendency was also observed during blind testing at San Francisco and Atlanta. During the operational demonstration project at San Francisco, the automated Pfa was 113 to 150 percent higher than the certification standard. For the present study, supplementary data were provided to the panel by SEIPT with Pfa from January 5, 1998, to April 20, 1998. These data show that Pfa varies between 125 and 250 percent higher than the maximum rate allowed during certification. Operational data reviewed by the inspector general of the U.S. Department of Transportation suggested that the Pfa was as high as 169 percent higher than the certification standard (DOT, 1998).
Data from the first of the four open tests show an average of 50 seconds for alarm-resolution time using the CTX-5000 SP. Although the resolution time was lower during subsequent tests, the combination of a high Pfa and a long alarm resolution time can have a significant impact on the throughput rate, and, in fact, was determined to be the limiting factor for throughput rate.
4 The actual values required for certification are recorded in classified FAA documents (FAA, 1992).
Conclusions and Recommendations
The deployment of bulk explosives-detection equipment has not progressed as quickly as planned. Initially, 54 certified bulk explosives-detection systems were scheduled to be deployed by December 1997. The deployment plan was then modified, and the EDSs, as well as 22 noncertified bulk explosives-detection devices, were to be deployed by March 1999. In the interim, the FAA developed a program to purchase the equipment along with developed and implemented factory and site-acceptance protocols and testing procedures. As of January 1999, more than 70 certified x-ray CT-based EDSs had been deployed and seven other bulk explosives-detection devices. Therefore, the deployment of more explosives-detection equipment based on other technologies would yield useful operational data.
The FAA should first deploy and obtain operational data on advanced technology (AT), including EG&G Z-Scan, Vivid devices, Heimann devices, and In Vision/Quantum Mechanics QR devices. The FAA should then deploy and obtain operational data on any warehoused In Vision CTX-5000 SPs or CTX-5500 DSs.
The location of a CTX-5000 in an airport is largely dictated by physical constraints, which in turn can affect its utility. In newly designed airports or airport terminals, the placement of bulk explosives-detection equipment can be incorporated into the design of the terminal (e.g., Terminal One at John F. Kennedy International Airport). After site visits to three airports, the panel concluded that the data are not sufficient to assess the operational installation configuration of bulk explosives-detection equipment in airports.
The FAA should encourage airlines and airports to implement different explosives-detection equipment installation configurations so that their effectiveness can be assessed.
Because foreign airports often use different installation configurations for explosives-detection equipment than U.S. airports, the FAA should collaborate with foreign governments to collect data on the effectiveness of various configurations to assist in establishing the best practices.
The FAA has not developed a plan for collecting data on the Pfa and operator alarm resolutions (e.g., actions taken, time to resolve, etc). During three separate site visits, the panel found no evidence of measures being used to assess the performance of deployed equipment or of an FAA-specified data-collection protocol. In Vision Technologies has taken the initiative of collecting data on its deployed systems. The panel concluded, however, that the available data were insufficient to evaluate the operational effectiveness of deployed equipment. Furthermore, the panel believes that little data on the operational effectiveness of the deployed equipment will be forthcoming unless the FAA develops a plan with the airlines and explosives-detection equipment manufacturers to obtain such data.
In cooperation with the airlines and explosives-detection equipment manufacturers, the FAA should develop and implement a plan to collect specific data on false-alarm rates and operator alarm resolutions. The FAA should also develop a plan to collect operational data on detection rates, with and without operator involvement. In addition, the FAA should ensure that the data-collection plan is carried out and systematically documented.
The main conclusion in the final report of the FAA Operational Demonstration Project at San Francisco International Airport was that the time required to resolve alarms must be reduced to increase throughput. However, the panel concluded that more data are necessary to evaluate the combined performance capability of deployed equipment and
operators. Data on the reliability, maintainability, and availability of the deployed equipment should be collected and made available to the airlines.
Controlled testing, such as the testing done during the Airport Demonstration Project, should be conducted for a variety of equipment configurations. Data should be collected and maintained on performance (e.g., probability of detection) and the test conditions. The reports on the Manila airport demonstration projects should be completed and reviewed for confirmation of or challenges to the results of the San Francisco tests.
Certification tests only reflect the ability of the equipment to identify a bag that contains an explosive. The detection rate is based on the alarm being set off for a bag containing the explosive, even if the alarm was triggered by a nonexplosive object in the bag. Certification testing does not measure alarm resolution and does not include testing in the operational environment of an airport. In the panel's opinion, some of the problems encountered with the CTX-5000 SP in the field can be reasonably attributed to the limitations of certification testing. Furthermore, under current certification guidelines, equipment certified in the future may encounter similar problems.
During certification testing, the FAA should, whenever possible, measure both true detection rates (i.e., identification of the correct location of an explosive when an alarm occurs) and false-detection rates (i.e., an alarm set off by something in a bag other than an explosive).
The FAA should assess the feasibility of including airport testing of an explosives-detection system as part of the certification process.
The FAA should include the ability of explosives-detection equipment to aid the operator in resolving alarms as part of certification testing. For example, alarm resolution should be a factor in the determination of throughput rate, detection rate, and false alarm rate.
The bag set used to determine the false-alarm rate during certification testing does not contain many of the items normally found in passenger bags, such as foods and liquids. This difference could account for the 50 percent increase in false-alarm rates in the field over the certification standard. Furthermore, the throughput rate measured during certification testing is based on the continuous flow of bags and does not include time for alarm resolution. Consequently, the throughput rates measured during certification testing are not representative of throughput rates in the field.
The bag sets used for estimating false-alarm rates during certification testing should include all of the items usually found in checked passenger bags (e.g., sand, books, food items, liquids, jewelry, toiletries, and clothes).