Ballistic Imaging (2008) / Chapter Skim
Currently Skimming:
PART II: Current Ballistic Imaging and Databases, 4 Current Ballistic Imaging Technology
PART II: Current Ballistic Imaging and Databases, 4 Current Ballistic Imaging Technology
Pages 89-132
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 89... ...
PART II Current Ballistic Imaging and Databases
|
From page 91... ...
." In this chapter, we briefly review the background of imaging technology in firearms identification (Section 4–A) , the basic structure of Integrated Ballistics Identification System (IBIS)
|
From page 92... ...
together, a summary and our conclusions on the evidence in this chapter are in Chapter 6, together with those from Chapter 5. 4–A Background Contemporary ballistic imaging technology is the latest step in a gradual move over several decades to use technology to make it easier to maintain and search open case files of ballistics evidence, including cases distant in time.
|
From page 93... ...
use IBIS. IBIS is also in use by law enforcement agencies in several foreign countries; through IBIS, FTI is essentially the only provider of ballistic imaging technology.
|
From page 94... ...
. Most of the IBIS installations under the NIBIN program take the form of Remote Data Acquisition Stations (RDASs)
|
From page 95... ...
To the greatest extent possible, ector (2002) considers the effect of one cleaning process on IBIS performance for match R ing bullet evidence.
|
From page 96... ...
Although the case identification number is displayed in a column when comparison results are returned for analysis, the system does not make it readily apparent where individual exhibits from a particular case fall in the list of rankings reported by IBIS. IBIS training materials emphasize the importance of correct entry of auxiliary, context data about evidence and exhibits, the "demographic data." For cartridge case markings, the training guide indicates that "automatic correlation requests use all of the following demographic information" -- occurrence date, caliber, firing pin shape, and event type -- "to select the test candidates from the database," and all these pieces of information are described as "crucial for the correlation process" (Forensic Technology WAI, Inc., 2002a:2-10, 3-2)
|
From page 97... ...
of the bullet may also be recorded. Although accurate demographic data entry is essential to the IBIS comparison process, the physical positioning of bullet or cartridge evidence under the microscope (and camera)
|
From page 98... ...
Once the regions of interest are set for acquiring a breech face, the image is taken using the IBIS standard ring lighting, intended to provide uniform illumination, and the system automatically suggests a lighting intensity "to provide optimum lighting for acquisition." However, the IBIS training materials note (Forensic Technology WAI, Inc., 2002a:2-18) : In numerous cases the suggested lighting may not appear optimal (for e xample, with smooth surfaces or uncommon metal primer compositions)
|
From page 99... ...
firing pin image using the standard ring, center light, acquired from the same cartridge casing. Although they are difficult to see in this reproduction, circular region-of-interest delimiters are indicated on images (a)
|
From page 100... ...
The IBIS operators' manual describes "optimal lighting" for the firing pin impression as "eliminat[ing] as many clusters of washed out pixels in the firing pin region as possible, without the region being too dark" (Forensic Technology WAI, Inc., 2002a:2-27)
|
From page 101... ...
Smaller signatures are less detailed but more efficient to use" (Forensic Technology WAI, Inc., 2001:129)
|
From page 102... ...
, and an ejector mark image and signature. Bullet images and signatures are substantially larger due to the acquisition of multiple images (for each LEA)
|
From page 103... ...
For example, .38 auto ammunition can be fired with a 9mm Makarov pistol. This reflects the interchangeability of bullets and cartridge cases in firearms" (Forensic Technology WAI, Inc., 2002a:3-2)
|
From page 104... ...
. Adjusting the threshold level is not impossible but requires intervention from FTI; the study of IBIS performance by George To accommodate clerical delays," Forensic Technology WAI, Inc.
|
From page 105... ...
CURRENT BALLISTIC IMAGING TECHNOLOGY 105 FIGURE 4-2 Sample "cover sheet": Top 10 ranking report from an IBIS comparison. 4-2.eps NOTE: Page is from the committee's experimental work at the New York State bitmap image Police Forensic Investigation Center.
|
From page 106... ...
. The process for comparing signatures from bullet evidence does not involve a coarse comparison or threshold step, but is more complex than the cartridge comparison routine due to the nature of the exhibits.
|
From page 107... ...
The basic questions inherent in working with IBIS comparison scores is what meaning to put on a particular score and how deep in a list of sorted results an analyst should look for possible matches. Aside from the basic guidance that "the higher each score is, the more similar the test and reference exhibits are" (Forensic Technology WAI, Inc., 2001:131)
|
From page 108... ...
Most of the relevant studies are intended to address specific performance issues suggested by the creation of a large-scale RBID, containing many exhibits with common class and possibly subclass characteristics; others have scrutinized specific parts of the IBIS comparison process, such as the 20 percent threshold step.
|
From page 109... ...
that "it is apparent that existing research is too limited and that further study of current and emerging technologies is needed before creating an RBID in California"; this further research should include alternatives such as microstamping and "would be most comprehensive if conducted at the federal level." The report expressed optimism on the "potential to develop ballistic imaging into a powerful crime-solving tool," and suggested that "a national RBID could be an extremely valuable tool for law enforcement in generating leads and solving crimes" (Lockyer, 2003:9)
|
From page 110... ...
Spreadsheets of the test results indicate that any case where a breech face rank is listed as "Not in Selection" also has a firing pin rank "Not in Selection," and vice versa, suggesting that these are cases where an expected match did not survive the coarse comparison pass and 20 percent threshold. (The tests performed in the California evaluation, and the formal responses to the results of those tests, are summarized in the appendix to this chapter, Section 4–G.)
|
From page 111... ...
However, it does not provide insight into the number of possible matches that may be missed because the comparison fails to pass the coarse comparison and IBIS-default 20 percent threshold (under which the effective sample size would be somewhat more than 120)
|
From page 112... ...
was that the default 20 percent threshold did hamper IBIS's ability to generate matches. In total, 183 comparisons were made between an "evidence" or "blind" exhibit and a database containing a sister image, from the same gun and using ammunition of a particular type.
|
From page 113... ...
In his analysis, George notes high percentages of cases with rank below 170, which is 20 percent of the database size; as shown in Table 4-1, the effective thresholded sample size is almost certainly higher than 170 because high-ranking exhibits from any of the three marks (breech face, firing pin, and ejector mark) are retained from the coarse comparison pass, and we use 190 as a tabulation comparison.
|
From page 114... ...
, so the IBIS 20 percent threshold would include any exhibits in the top 20 percent by any of the three marks. Hence, the effective 20 percent threshold almost certainly involves more than 170 rankings; we use 190 as a rough approximation to the effective thresholded sample size.
|
From page 115... ...
authored two studies on the performance of IBIS comparison routines. The first study summarizes and suggests a common notation for direct comparison of the major previous studies of IBIS performance (the same studies we describe in this section)
|
From page 116... ...
4–E.5 FTI Benchmark Evaluation As the developer and maintainer of IBIS, Forensic Technology WAI, Inc., enjoys unique advantages in testing the system's performance, including the ability to vary the level of thresholding used in the coarse comparison stage and to directly study the signatures derived from images. More directly, FTI's position offers great latitude with respect to one key performance variable: Because it can tap image data from IBIS installations worldwide, it can assemble larger image datasets than is possible for any particular agency, including large numbers of exhibits within particular caliber groupings.
|
From page 117... ...
Their work forecasts that -- when searching a database of 1,000,000 exhibits -- IBIS performance in detecting sister pairs within the top 10 ranks looking at both breech face and firing pin marks is on the order of 30–35 percent. Based on the smaller set of 9mm exhibits for which ejector marks were also considered, the estimated success at finding a known match in a 1,000,000exhibit set is about 50 percent when all three marks are considered.
|
From page 118... ...
118 Breech Face Images Firing Pin Images Firing 1 Firing 2 Firing 1 Firing 2 A: Browning firearm; same ammunition in both firings B: Smith & Wesson firearm; same ammunition type 4-3 left.eps C: Walther firearm; same ammunition type
|
From page 119... ...
D: Walther firearm; same ammunition brand E: Unknown Luger-type firearm; same ammunition brand 4-3 right.eps F: Smith & Wesson; different ammunition brands FIGURE 4-3 Sample matched pairs of breech face and firing pin images. SOURCE: Images from Forensic Technology WAI, Inc., benchmark evaluation dataset of matched pairs of exhibits, shared with 119 the committee.
|
From page 120... ...
(1994) requested a benchmark comparison between the DRUGFIRE system and the early-IBIS BULLETPROOF systems, in the early days of ballistic imaging and as the potential for overlap became apparent.
|
From page 121... ...
Schecter and Giverts (2005) suggest a workaround to improve IBIS performance when comparing Glock-type cartridge cases, for which the ejector mark impression lies within the casing's headstamp on the edge of the primer surface, and not on the outer rim of the casing.
|
From page 122... ...
These casings were processed using both IBIS and three-dimensional metrology techniques, and were also run through IBIS waiving the coarse comparison and 20 percent threshold steps. We also performed limited IBIS experimentation using the New York CoBIS RBID and the independent IBIS database of the New York Police Department.
|
From page 123... ...
Yet it is also, fundamentally, a limitation of the technology. Since its origins in the early 1990s, the progress in developing the existing IBIS platform for ballistic imaging has been evolutionary rather than revolutionary, in that it has remained anchored to the premise of emulating the functions of a comparison microscope.
|
From page 124... ...
. seeking an order requiring that his handgun be tested under the Integrated Ballistics Identification System (IBIS)
|
From page 125... ...
"Firing pin impressions and breech face marks -- a closer look," muses the investigator; instantaneously, the system makes a loud shuf fling sound and several beeps. The camera now shows the "IBIS" screen, which prominently shows a single image of the entire base of a cartridge, headstamp and all; some text indicating "Halo On" and "Magnification 150X," among other things, is superimposed over the corner of the image.
|
From page 126... ...
4–G Appendix: Summary of Performance Tests in the California Evaluation of a Reference Ballistic Image Database This appendix describes the tests performed by Tulleners (2001) in response to the California legislature's directive that the state's Department of Justice study the feasibility of a reference ballistic image database.
|
From page 127... ...
of these known-match comparisons failed to clear IBIS' coarse comparison and 20 percent threshold. It does not appear that one mark was superior to the other in terms of generating possible matches: the 31 instances where the known sister was found in the top 10 by either mark are fairly evenly divided between cases where both marks were in the top 10 (10) , only the breech face was in the top 10 (9)
|
From page 128... ...
failed to clear the coarse comparison and 20 percent threshold. 11 A third part of the test timed the comparison times for three selected exhibits for database subsets of different sizes (100, 250, 500, 792)
|
From page 129... ...
. However, no match was possible from the pre-alteration to the post-alteration exhibits; in both cases, the technical evaluation's data appendix lists the matches as "not in selection list," suggesting that the deliberate alteration prevented the exhibits from clearing the IBIS coarse comparison pass.
|
From page 130... ...
Test 5 was intended to assess IBIS performance using cartridges fired from SIG Sauer firearms, which are known among examiners for having minimal breech face characteristics. An extensive set of SIG Sauer test fires was subsequently used in Tulleners' joint study, De Kinder et al.
|
From page 131... ...
In his review, De Kinder (2002b:9–11) rejected this argument, citing research by the Forensic Institute in The Netherlands on IBIS score results using 18 common primers suggesting that Federal showed medium performance in registering marks.
|
From page 132... ...
He countered that the passage quoted above "is the same type of expression as saying at the beginning of the 1990[s] that automated comparison of bullets and cartridge casings is impossible." He preferred instead the revised statement that "the current scientific knowledge and state-of-the-art technology does not allow one to be more accurate than a microscope comparison by a firearms examiner." De Kinder held that dropping the believed-"unmatchable" exhibits from analysis is "unacceptable," particularly given that the study was oriented to studying the feasibility of an RBID.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.