Forensic Analysis: Weighing Bullet Lead Evidence (2004)

The primary objective of compositional analysis of bullet lead (CABL) is to produce evidence for use in court. Although the evidence is analyzed with scientific instrumentation and statistical methods, its presentation and use in court are subject to human interpretation and error. Attorneys, judges, juries, and even expert witnesses can easily and inadvertently misunderstand and misrepresent the analysis of the evidence and its importance. It is therefore essential to discuss whether and how the evidence can be used. It is first necessary to introduce the lead and bullet manufacturing processes so that the implications of bullet production for the legal system are fully understood. This chapter is split into two sections: “Significance of the Bullet Manufacturing Process” and “Compositional Analysis of Bullet Lead as Evidence in the Legal System.”

The following description of the processes leading to the production of loaded ammunition represents the bullet manufacturing practices currently in place at large-scale producers in the United States. (Processes used overseas are less well documented.) As shown in this chapter, the processes vary at numerous points, depending on such factors as the manufacturer, the caliber and style of bullet, the magnitude of a production run (which is often dictated by the demand for a particular caliber), and the size of the manufacturing facility. This

section details procedures that are believed to account for the manufacturing processes used for .22 caliber rimfire and other bullets by major producers in the United States. (This process is described because .22 caliber rimfire ammunition is one of the most popular ammunition rounds produced.) It has been estimated that 50–75 percent of all ammunition sold in the United States originates with U.S. manufacturers and that about 50 percent of ammunition used by the U.S. military (for example, 9-mm, 7.62-NATO, and 5.56-NATO ammunition) and more than 50 percent of non-U.S. issue military calibers (such as 7.62 × 39 <AK-47> and British .303 <Enfield>) are imported.^1,2,3

On the order of 85–118 million pounds of lead is used each year in the production of bullets⁴ in the United States.^5,6 The exact number of each caliber and type of bullet (such as jacketed or hollow point) is not known, but some estimates of production volumes have been provided by the Sporting Arms and Ammunition Manufacturers’ Institute⁷ and are shown in Table 4.1. It is generally acknowledged that .22 caliber bullets are the dominant type sold. Table 4.2 provides some examples of typical bullet masses for various calibers. Using 70 grains (0.16 oz, 4.54 g) as an arbitrarily assumed average bullet mass allows the estimation that the 85–118 million pounds of bullet lead produces about 8.5–11.8 billion bullets per year in the United States.

Figure 4.1 is a simplified flow chart for bullet production and approximate mass of material involved in each of the processed materials. Table 4.3 has been prepared from the general information given in Figure 4.1 to illustrate the approximate number of bullets associated with each of the manufacturing steps or

products. Calculations assumed a mass of 40 grains (0.0914 oz, 2.59 g) for a .22 rimfire projectile. The number of projectiles is based on 100 percent yield. Since some material is not converted directly to the final bullets (for example, initial piece of extruded wire, weep from bullet presses), the actual number of projectiles produced will be lower.

In the United States, secondary smelters melt recycled lead (primarily from recycled lead-acid storage batteries) for bullet lead processing in large pots.⁸ The designation of primary smelter is reserved for manufacturing facilities that produce lead from ores. Such facilities are rarely associated directly with bullet production in the United States, but this is not the case in some foreign countries. Secondary smelting is reported to account for half the lead produced in the

United States. There are 50 plants, with capacities ranging from 1,000 to 120,000 tons/year.⁹

Refining of the melt to remove various elements present either as impurities or as previously added alloy elements can occur at the secondary smelter.^10,11 After refinement, Sb, less frequently Sn, and sometimes both elements may be added to harden the bullet. Finally, the melt is poured into various smaller products, including billets, which are sent to the bullet manufacturer.

The bullet manufacturer may use the purchased billets directly for production, but it is not uncommon for bullet manufacturers to remelt the purchased lead and cast their own billets for production.¹² The bullet manufacturer extrudes bullet wire from a solid billet; this results in one or more wires per billet, depending on whether the extruder die has one or more extrusion ports. Generally, a continuous wire is not produced from multiple billets due to the likelihood of discontinuity and the production of a flawed slug at the junction due to lead lamination. The size of the extruded wire is dictated by the caliber (diameter) of the bullet to be produced from that wire.

The bullet wire is then fed into a machine that cuts it to predetermined lengths to generate slugs of the approximate weight and dimensions of the final bullet. The slugs are collected in bins, whose size varies from plant to plant. In larger manufacturing facilities, several extruders may be operated in parallel in the production of slugs of a given caliber, and the slugs from the various extruders may be collected in the same bin. A given wire is converted to slugs of a given length and diameter.

The slugs are then pressed into the final shape of the bullet, a jacket is applied (if appropriate), and the bullets are again collected in bins.¹³ The bullets are seated into appropriately prepared cartridge cases (loaded with primer and powder) to form the loaded ammunition, which is either collected in bins or sent directly to machinery for packing in boxes. The boxes generally contain 20–50 rounds each, depending on the caliber and the products being offered by the company. A more specific example of the wire-to-ammunition production steps for .22 caliber rimfire bullet production is as follows:¹⁴

• Pinch cut to partially perforate the wire.

• Tumble the partially perforated wire to break it into slugs.

• Swage press to final shape (three steps are needed).

• Wash and rinse.

• Flash plate with copper alloy (if high-velocity product is being made).

• Lubricate.

• Assemble into loaded ammunition.

• Pack ammunition in boxes.

The boxes are then generally bundled into appropriately sized shipping quantities—such as cartons, crates, or pallets—and sent to jobbers, distributors, wholesalers, or large retailers. They then go to the retailer’s shelf for purchase by the consumer.

Reloaders, both commercial and private, are another source of loaded ammunition and are less directly connected to large-volume manufacturers.¹⁵ Using refurbished cases for reloading, reloaders make less-expensive ammunition. In some instances, reloaded bullets are made from melted scrap lead, such as discarded wheel-balancing weights that are remelted and poured into bullet molds.

This section details the various stages leading to the production and distribution of boxes of loaded ammunition. Comments on the variations that are known to exist at various stages are given here, but their implications for the homogeneity of melts, billets, wires, and so on, are discussed in the section titled “Compositional Information.”

With over 3.5 billion pounds of lead smelted each year in the Unites States, the 85–118 million pounds used in bullet manufacturing comprises about 2.5–3 percent of total lead use; lead-acid storage batteries probably represent the largest product.^16,17 Secondary smelters that produce bullet lead are also gen-

erally involved in the production of “battery lead.” Chemical compositional requirements for bullet lead are much less stringent (that is, they have less-restrictive tolerances) than are needed for battery lead. However, a hardened lead is generally needed for bullets.^18,19 Hardening is typically accomplished by the addition of Sb to the melt. Sn can also be used, but it is more expensive. Other components of bullet lead are generally carried over from the lead source, and maximal tolerances in their concentrations are normally specified by the bullet manufacturer.

Bullets are reportedly produced mainly from recycled lead in the United States. Therefore, it is impossible to trace bullet lead back to the original source of the ore,²⁰ and no detailed discussion will be presented here on the primary smelters and ore processing except to note that the ores are sulfides and contain small amounts of Cu, Fe, Zn, precious metals, and other trace and minor elements, such as As, Sb, and Bi. The primary smelting process involves removal of those elements by reduction and refining.

As noted previously, the dominant source of bullet lead is the electrode materials from recycled batteries. The melting process takes place in pots that may contain, for example, 50–350 tons of melt. The descriptions given below are typical; they might not be applicable to all smelters.

The first step in secondary lead refining is treatment of scrap to remove metallic and nonmetallic contaminants. That is done by mechanical breaking and crushing to separate extraneous contaminants and then “sweating” the separated lead scrap in a reverberatory furnace to isolate the lead from metals that have higher melting points. The next step is smelting in a blast furnace to make “hard” (high-Sb) lead or in a reverberatory furnace to make “semisoft” (3–4 percent Sb) lead. Refining is normally done in a batch process that takes a few hours to a few days in kettle-type furnaces that have production capacities of 25–150 tons/day.²¹ In the refining process, Cu, Sb, As, and Ni are the main elements removed. It is generally assumed that Sb is the element whose content is most critical because it determines the bullet hardness.^22,23

In the production of bullet lead, the manufacturer generally has requirements for the concentrations of the final lead alloy.²⁴ The elemental compositional requirements can vary with the bullet manufacturer. Depending on the element, either maximal allowable or ranges of concentrations may be specified. Table 4.4 shows an example of one manufacturer’s compositional requirements for lead to be used in .22 long rifle bullets. Some bullet producers use as-received billets from secondary smelters, and others conduct tertiary melting to make additional adjustments to the lead composition or to recycle scraps of lead produced during bullet production.

A secondary smelter may produce solid lead of various shapes, including ingots, pigs, and billets. An analysis certificate accompanies the product shipped to the bullet manufacturer; it uses a smelter-dependent format that contains various degrees of analytical detail. Spark-emission optical spectroscopy is the technique generally used for analysis of the alloy at the smelters.²⁵ The technique

generally produces precision on the order of ±10–20%; however, when the most stringent standardization procedures are implemented, precision may approach ±5 percent.²⁶

There is no requirement by the bullet manufacturers that all lead ingots received from a smelter come from a single pour or melt. It is generally assumed that the composition of a given melt is constant and homogeneous from the beginning to the end of the pour if nothing is added to the pot during the pour.²⁷ The assumption of homogeneity is based on the convective mixing in the vat and the relatively short pouring time. It should be noted that during a pour material may be added to the original melt, thus producing time-varying compositional changes. The additions may include bulk material (ingots, pigs, and so on), manufacturing scrap (pieces of bullet wire, scrap from bullet-forming operations, and the like), or molten lead introduced from a secondary vat. Examples of the time-dependent variation in composition can be seen in some of the data of Koons and Grant.²⁸ In the case of at least one manufacturer, billets are not poured from a vat that has a constant composition; instead, while the vat is being poured, molten lead from another pot is continuously added to maintain the level of molten lead in the vat being poured. Thus, compositional changes can occur during casting. The data of Koons and Grant²⁹ indicate that compositional change occurs over several 60 lb ingots that were being poured. For example, the concentration of Sn decreased by 60 percent (from 0.030 to 0.012 percent Sn) over a 30 minute period, the largest change of the data presented. Combining this information with the standard deviations for the analytical measurement (that is, < 0.001 percent Sn) it can be estimated that approximately 15 ingots (approximately 850 lbs of Pb) were poured before the average concentrations changed by one standard deviation. Thus, it can be reasonably assumed that the rate of compositional change—even when molten lead batches are mixed during a pour—from one poured ingot to the next poured ingot is much smaller than the measurement precision available. It also follows that any compositional change in the lead initially poured into an ingot (or billet) would be indistinguishable from the molten lead added to the mold to complete the pour of that ingot, as long as the casting of the ingot was completed in a single pour.

Randich et al.³⁰ also showed occasional distinct concentration changes in some elements as samples were extracted from the beginning, middle, and end of the pour. Statistical analysis of the changes showed that there was no distinct time-dependent one-directional change (that is, always increasing or decreasing

as the pour proceeded), which would suggest for these data that lead of a different composition was being added during the pour, rather than that some chemical process occurred that depleted or enriched a given element as a function of time. The former possibility (the addition of lead during the pour) is supported by the data of Koons and Grant,³¹ who presented a more detailed analysis of billets resulting from pours. Koons and Grant used several of the same data sets as Randich et al.³²

Billets weigh 70–350 lbs (32–159 kg), depending on the manufacturer and the size and type of extruder that is used in the production of bullet wire.^33,34 In some instances, the secondary smelter is also a bullet manufacturer, and the billets produced are used on site in the production of wire, slugs, and so forth. In other instances, the lead ingots, pigs, or billets are shipped to bullet manufacturers, and the bullet manufacturers may use the billets directly in their extruders to produce wire. There are also instances in which the ingots or pigs obtained from the secondary smelters are remelted to pour new billets at the bullet manufacturing plant.

Various activities can occur during this tertiary melting that affect the final billet composition. For example, melted lead prior to casting in billets is typically “fluxed” to remove oxidized lead metal elements and other impurities. The fluxing agent can contain a number of different materials, and is often borate-based in commercial bullet manufacturing operations. Nitrogen gas is also a common fluxing agent. The flux entrains the impurities and floats them to the surface of the lead melt for removal.

Billets are used without alteration (in their original, solid state) in the extruders to produce bullet wire. The mass of the wire is somewhat less than the mass of the billet, because the tail end of the billet cannot be forced through the extrusion die by the ram.^35,36 The length of the wire is governed by the billet

size and the wire diameter (bullet caliber). For example, a 70-lb billet should produce about 114 ft of wire intended for .22 caliber ammunition, but the same billet should produce about 27 ft of wire if .45 caliber bullets are the intended product. The extruder die may have a single exit port that produces a single wire strand from the billet, or it may have multiple extrusion ports that produce several wires from a single billet. Several feet of the wire formed at the beginning of the extrusion process may be discarded and recycled into a future billet.³⁷

In brief, the wire is used as feed for a cutter, which consists of a machine that automatically introduces the wire into a cutting device to produce slugs, small cylinders of lead whose length and mass are close to those of the final bullet. The slugs are stored in large bins that may hold substantial quantities of slugs from different wires.

The binned slugs are fed into hoppers that feed the presses that form the bullets. Although it is not a true swaging process, this term is commonly encountered in the literature describing the process. Thus formed, the bullets are then tumbled, sometimes lubricated, and stored in bins.^38,39 For some bullet types, a metal jacket is added.

The loaded ammunition, which is sometimes referred to as rounds or cartridges, consists of a brass case that is charged with primer and powder and into which the bullet is pressed. Bullets and cases from bins are fed into hoppers, and the process of ammunition production proceeds in an automated fabrication machine. The product is sent directly to the packaging operation or is placed in large bins for later packaging.^40,41

The bullet manufacturer packages the ammunition in boxes for shipment. The box typically is labeled with a stamp that refers to the “boxing lot,” which may be recorded as a date or simply a number. In some manufacturing plants,

the boxing lot number refers to the date the ammunition was loaded; in others, the date or number is not necessarily related to a particular stage in the production process. A typical box contains 20–50 cartridges, but some units or boxes are larger, depending on product line and caliber. For example, .22 long rifle “value packs” are commonly sold in 550-round boxes, and 100-round boxes of 9 × 19 mm ammunition have recently become common at larger retailers.⁴² The boxes are arranged in larger shipping units (such as cartons, crates, and pallets) and shipped to jobbers, distributors, wholesalers, or large retailers.

Attempts to obtain details on the shipping and distribution processes for loaded ammunition were unsuccessful and therefore are not clearly understood by the committee. For example, the committee has no evidence that distribution from a given manufacturer is regional as has been suggested in one report.⁴³ Similarly, the frequency and size of shipments are unknown, but they are expected to vary widely, depending on the customer and the type of ammunition. However, it is reasonable to assume that high-turnover ammunition (for example, .22 caliber) is shipped more frequently than others and in larger quantities.

The committee has a similar lack of knowledge about retail dispersion of boxes. For example, it is not known whether first-in-first-out sales occur—that is, whether older shipments are arranged on shelves to be sold first.

Multiple steps are required to move from bullet production to boxes of ammunition, and manufacturers vary in their processing of materials leading to bullet formation. In addition, storage times before actual packaging and shipping depend heavily on caliber; for example, high-production munitions, such as .22 caliber, probably move more rapidly from slug production to shipping than less-common munitions.

There is much debate of the homogeneity of the lead “source.” It is unclear whether macro- and microscale inhomogeneities are present at some or all of the stages of lead and bullet production and if such inhomogeneities would affect CABL. The poor definition and understanding of the term “source” causes additional confusion. These topics are clarified below.

• Melt. It is reasonable to assume that a given batch of molten lead exhibits sufficient mixing (such as convective stirring because of the heating process)

for compositional homogeneity to develop quickly in the melt, assuming that there are no additions to the molten vat during pouring. Some constituents—such as Sb, As, and Sn—oxidize in air, and their loss or flotation to the surface is expected to take place slowly. However, the rate of compositional change is unlikely to be significant relative either to the rate of casting of billets or to the uncertainty of the concentrations of these materials. The assumption that the rate of compositional change is insignificant is supported by the small surface area exposed to air relative to the total mass of the melt.

• Pigs, Ingots, and Billets. The homogeneity of ingots, pigs, and other large blocks of smelted lead is not an issue, because they are always remelted before billets are cast. Inhomogeneity of billets can arise from two factors. First, a billet may be cast in two stages, with the second stage long enough after the first for a measurable compositional difference to exist, depending on the constancy of the melt between the two pours that finalize billet production. Second, solutes inevitably segregate to the center of the billet during solidification.

• Wires, Slugs, and Bullets. The extrusion process used to produce the wire from a billet is thought to negate the inhomogeneity due to segregation during solidification because the flow of the solid is turbulent as the billet enters the mouth of the die. Uniformity along the length of wire has not been substantiated. However, Koons and Grant have sampled wires produced from billets from a pour and found that concentrations remained constant (that is, within analytical precision) over several billets.⁴⁴ Small compositional differences may exist along the length of the wire as a result of several factors. Segregation of material at the end of the billet mold may enrich the less refractory constituents in the lead, and detectable segregation will diminish as the impurity level decreases. If this segregation occurs, it still might not contribute to compositional differences along the length, because several feet of the first length of wire extruded are discarded and returned to a scrap bin. If multiple billets are loaded into an extruder, a continuous, single wire is extruded, but is cut into separate wires where the change of billets takes place.⁴⁵ It is not clear from the data available whether the concentration of Sb is segregated in the billet or wire. While a paucity of data also exists for the spatial dependence of concentration of the other impurities along the length of wire (or in the billet), their significantly lower concentration should make spatial inhomogeneities less likely. It is reasonable to assume that cutting the wire to produce the slugs and pressing the slugs to form the final bullets produce no substantial segregation of elements in the lead.

• Mixing of Slugs, Bullets, and Loaded Ammunition. Some manufacturers

use multiple cutting machines with distinct wire feeds to simultaneously produce slugs that are collected in a common slug bin. Similarly, a given production run may require sequential cutting of several wires and collection in a common bin. Thus, if wires are not of the same composition, a bin can contain slugs with a finite number of distinct compositions and if slugs from previous runs went unused at the start of the cutting of new wires, they contribute to the mixing of slugs of different compositions in a bin.

The slug bins are emptied into hoppers that feed the bullet-shaping presses, and the bullets formed may be collected in bullet bins before they are fitted into cases to form loaded ammunition. “Tail-in-tail-out” mixing can occur in the bins if their full contents are not used in a single production run of ammunition. The mixing with previously formed bullets will not occur if the pressed bullets are used immediately (without storage in bullet bins) in ammunition production.

The loaded ammunition can be routed directly to a packaging area, in which case no additional mixing occurs. However, loaded ammunition is sometimes stored temporarily in ammunition bins, where batch mixing and tail-in-tail-out procedures that contribute to mixing can occur.

The likelihood of mixing in the various bins described above is supported by the compositional analyses conducted on the bullets in a given box of ammunition.⁴⁶ It is routinely found that a single box contains multiple distinct compositional groupings—as many as 14.⁴⁷

• Boxes, Crates, and Distribution. The boxes of ammunition are generally stamped with a box lot number. Depending on the manufacturer, this lot number may only reflect the packaging date, may be a direct indication of the date and shift during which the ammunition was loaded, or may be a code indicating packing date and shift, which can be traced through the manufacturer’s internal records to one or more shifts of loading operations. A stamped date does not reflect the date of pouring of billets, extrusion of wire, or formation of bullets. If filled boxes are stored on shelves because of overruns, boxes of different runs (with different dates) may be mixed in larger shipping units. Thus, a large-volume shipping unit for more commonly used ammunition might or might not contain only boxes with the same lot number and date.

As noted previously, distribution of boxes, crates, pallets, and other quantities of ammunition is poorly understood; there is minimal documentation to assist in establishing general trends. It is clear that distribution can lead to varied scenarios regarding retail dispersion of bullets from a distinct compositional group.

When the metal compositions of two bullets are analytically indistinguishable, it is commonly suggested that they may have originated in the same “source.” It might be good to replace that vague term with “compositionally indistinguishable volume of lead” (CIVL). The CIVL, produced during one production run at one point in time, is at least as large as the sample taken for analysis. From the current understanding of the bullet production process, CIVL can refer to different tangible products associated with the manufacturing cycle. At its largest, the CIVL may be a vat of molten lead whose composition is not altered during the pouring of billets. Similarly, the CIVL may consist of a series of billets that were poured before the vat composition was altered by, for example, the addition of more molten lead to replenish the vat. At the very least, a CIVL may consist of several wires. The ramifications of identifying bullets whose compositions are analytically indistinguishable and their possible association with a single CIVL are discussed later in this chapter.

This section discusses the legal aspects of CABL evidence. Knowledge of the lead and bullet manufacturing processes underlies the proper interpretation of CABL evidence. The topics covered here include admissibility standards (including evaluation of match data) and pretrial discovery.

The admissibility of CABL raises issues concerning expert testimony and relevance.

Experts are called by the prosecution to testify to the fact of matching and, in most cases, the evidentiary implication of a match. Federal Rule of Evidence 702 governs the admissibility of expert testimony in federal trials:

In Daubert v. Merrell Dow Pharmaceuticals, Inc.,⁴⁸ the Supreme Court interpreted an earlier version of Rule 702 to require that scientific evidence meet a reliability test. The Court wrote that “in order to qualify as ‘scientific knowledge,’ an inference or assertion must be derived by the scientific method. Proposed testimony must be supported by appropriate validation—i.e., ‘good grounds,’ based on what is known. In short, the requirement that an expert’s testimony pertain to ‘scientific knowledge’ establishes a standard of evidentiary reliability.”⁴⁹ The Court held that the Frye test,⁵⁰ which required that a novel scientific technique be generally accepted in the relevant scientific community as the sole condition for admissibility,⁵¹ had been superseded by Rule 702 of the Federal Rules of Evidence.

Under the Daubert analysis, the trial court must make “a preliminary assessment of whether the reasoning or methodology underlying the testimony is scientifically valid and of whether that reasoning or methodology properly can be applied to the facts in issue.”⁵² In performing this “gatekeeping function,” the trial court may consider a number of factors: whether the theory or technique can be and has been tested,⁵³ whether it has been subjected to peer review and

publication,⁵⁴ a technique’s known or potential error rate, the existence and maintenance of standards controlling the technique’s operation,⁵⁵ and a technique’s general acceptance in the relevant scientific community.⁵⁶ Those factors, however, are neither dispositive nor exhaustive. The Court emphasized that the Rule 702 standard is “a flexible one.”

The Court followed with General Electric Co. v. Joiner⁵⁷ and Kumho Tire Co. v. Carmichael⁵⁸ to make up what is now known as the Daubert trilogy. Daubert and its progeny have come to be viewed as establishing a stringent standard of admissibility.⁵⁹ In Weisgram v. Marley Co.,⁶⁰ the Supreme Court remarked: “Since Daubert, … parties relying on expert evidence have had notice of the exacting standards of reliability such evidence must meet.”⁶¹ More-

over, some federal courts have read the Daubert trilogy as inviting a “reexamination even of ‘generally accepted’ venerable, technical fields.”⁶²

In 2000, Rule 702 was amended⁶³ to codify Daubert and Kumho.⁶⁴ The Advisory (drafting) Committee’s note to that rule supplements the Daubert factors with other considerations: whether the underlying research was conducted independently of litigation, whether the expert unjustifiably extrapolated from an accepted premise to an unfounded conclusion, whether the expert has adequately accounted for obvious alternative explanations, whether the expert was as careful as he or she would be in professional work outside of paid litigation, and whether the field of expertise claimed by the expert is known to reach reliable results.⁶⁵

The Daubert decision is restricted to federal trials; it does not apply to other jurisdictions.⁶⁶ Thus, states are free to determine their own standards for admissibility of expert testimony, even in the 40 or so jurisdictions that have adopted evidence rules based on the Federal Rules of Evidence. Many jurisdictions have adopted the Daubert framework.⁶⁷ Moreover, other jurisdictions had rejected the Frye test before the Daubert decision,⁶⁸ and many of these now look to Daubert for guidance.⁶⁹

Nevertheless, some jurisdictions have retained the Frye rule.⁷⁰ Because Federal Bureau of Investigation (FBI) examiners testify in state trials, the Frye general-acceptance standard may apply to CABL in some cases.⁷¹

Relevance is the threshold issue for all evidence. Federal Rule 401 defines relevant evidence as “evidence having any tendency to make the existence of [a material or consequential fact] more probable or less probable than it would be without the evidence.” Rule 401’s standard does not require that the evidence make a consequential (material) fact “more probable than not” (“preponderance of evidence”) but only that the material fact (for example, the identity of a perpetrator) be more probable or less probable with the evidence than without the evidence.⁷²

Rule 402 makes relevant evidence admissible in the absence of a rule of exclusion, and Rule 403 specifies circumstances under which a trial court is permitted to exclude relevant evidence. Rule 403 reads: “Although relevant, evidence may be excluded if its probative value is substantially outweighed by the danger of unfair prejudice, confusion of the issues, or misleading the jury, or by considerations of undue delay, waste of time, or needless presentation of cumulative evidence.” In Daubert, the Supreme Court noted that “expert evi-

dence can be both powerful and quite misleading because of the difficulty in evaluating it. Because of this risk, the judge in weighing possible prejudice against probative force under Rule 403 of the present rules exercises more control over experts than over lay witnesses.”⁷³ As suggested by that passage, scientific evidence is often cited for its potential to mislead the jury because it may “assume a posture of mystic infallibility in the eyes of a jury of laymen.”⁷⁴ Furthermore, expert testimony using such terms as “match” can be misleading unless explained.

Although CABL evidence has been admitted in evidence for 30 years, there are relatively few published cases on the technique. The overwhelming majority of them are homicide prosecutions,⁷⁵ some of which are capital cases. Because there are few federal homicide statutes, CABL evidence is most commonly used in state prosecutions. The courts that have addressed the admissibility of CABL evidence have admitted it—at least in the published cases.⁷⁶ CABL evidence is often used in cases in which numerous other items of evidence are introduced, but courts have sometimes indicated that it played an important role in securing a conviction.⁷⁷

The published cases reveal a wide variety of interpretive conclusions with respect to CABL evidence. In many cases, the experts apparently have not, in their testimony, recognized the limitations of such evidence. We first describe some of the testimony and then turn to a description of permissible conclusions.

In some cases, experts have testified only that two exhibits are “analytically indistinguishable,”⁷⁸ but it is often unclear whether that was the only conclusion rendered at trial. In other cases, experts concluded that samples could have come from the same “source” or “batch”;⁷⁹ in still others, experts stated that the samples came from the same source.⁸⁰

The testimony in a number of cases goes further and refers to a “box” of ammunition (usually 50 loaded cartridges, sometimes 20). For example, two specimens

• Could have come from the same box,⁸¹

• Could have come from the same box or a box manufactured on the same day,⁸²

• Were consistent with their having come from the same box of ammunition,⁸³

• Probably came from the same box,⁸⁴

• Must have come from the same box or from another box that would have been made by the same company on the same day.⁸⁵

The transcript in State v. Earhart contains the following testimony: “We can—from my 21 years experience of doing bullet lead analysis and doing research on boxes of ammunition down though the years I can determine if bullets came from the same box of ammunition….”⁸⁶ In People v. Kennedy, the examiner testified: “If you are comparing two and they have exactly the same composition that’s what you do, expect they came out of the same box.”⁸⁷

Several other (and different) statements appear in the published cases. An early case reported that the specimens “had come from the same batch of ammu-

nition: they had been made by the same manufacturer on the same day and at the same hour.”⁸⁸ One case reports the expert’s conclusion with a statistic.⁸⁹ In another case, the expert used the expressions “rare finding”⁹⁰ and “a very rare finding”.⁹¹ In still another case, the expert “opined that the same company produced the bullets at the same time, using the same lead source. Based upon Department of Justice records, she opined that an overseas company called PMC produced the bullets around 1982.”⁹²

In recent years, testimony appears to have become more limited. A 2002 FBI publication states the conclusion as follows: “Therefore, they likely originated from the same manufacturer’s source (melt) of lead.”⁹³ Testimony to the same effect has also been proffered.⁹⁴

Recent laboratory reports reviewed by the committee contain the following conclusion: “The specimens within a composition group are analytically indistinguishable. Therefore, they originated from the same manufacturer’s source (melt) of lead.”⁹⁵ Another laboratory report used more cautious language: “This is consistent with the specimens within those groups originating from the same manufacturer’s source (melt) of bullet lead.”⁹⁶

The most recent edition of the FBI Handbook of Forensic Sciences contains the following comment: “Differences in the concentrations of manufacturer-controlled elements and uncontrolled trace elements provide a means of differentiating among the lead of manufacturers, among the leads in individual manu-

facturer’s production lines, and among specific batches of lead in the same production line of a manufacturer.”⁹⁷

The opinions in some cases indicate that prosecutors and courts have overstated the probative impact of matching evidence. For example, in its appellate division brief in State v. Noel,⁹⁸ “the State asserted that this testimony is reliable scientific proof not only that the bullets ‘came from the same source of lead at the manufacturer’ but were ‘sold in the same box.’” Part of the problem in this case was the prosecutor’s summation, which made this argument. The intermediate appellate court believed that the argument was prejudicially misleading,⁹⁹ but the New Jersey Supreme Court, although conceding that the argument may have been “excessive,” held that it might pass as “fair comment.”¹⁰⁰ Similarly, in United States v. Davis,¹⁰¹ the court wrote that “the evidence made it more probable than not that the expended bullets originated from the cartridge box found in the Nissan.”¹⁰² The committee has made several recommendations (see infra) concerning how trial testimony should be presented.

CABL involves three steps: chemical analysis, statistical analysis, and the interpretation of data derived from them. As one commentator noted when evidence based on neutron activation analysis (NAA) was first introduced, “most of the legal problems surrounding NAA [now inductively coupled plasma-optical emission spectroscopy (ICP-OES)] do not involve its validity as a technique of chemical analysis. Rather, interpretation of the results of the chemical analysis—the relevance of the results to a particular legal issue—causes most of the difficulties.”¹⁰³ Because the analytical technique (ICP-OES) has not been an issue, we deal here with the third step—relevance and interpretation.¹⁰⁴

Evidence that crime scene bullets and loaded cartridges associated with a suspect came from the same melt is relevant under the definition of Rule 401, which is a low standard.¹⁰⁵ It has a “tendency to make the existence of any fact that is of consequence to the determination of the action [that is, the identity of the perpetrator] more probable … than it would be without the evidence.”¹⁰⁶

The critical issues, however, are how probative such a finding is¹⁰⁷ and how that probative value is conveyed to the jury.

There are two aspects of relevance in this context: the likelihood that crime scene bullets came from the same CIVL as the defendant’s bullets and the likelihood that the crime scene bullets came from the defendant.¹⁰⁸

A description of the probative force of evidence is given by the likelihood ratio for such evidence. The likelihood ratio for bullet lead match data is the probability that two bullets would match if they came from the same CIVL divided by the probability that they would match (coincidentally or through error) if they came from different CIVLs. If the likelihood ratio is much larger than 1, the fact of a match is strong evidence that the bullets came from the same CIVL; if not, the evidence is weak.¹⁰⁹

To illustrate how this concept could be used quantitatively, assume for the sake of discussion that the probability that two bullets would match if they came from the same CIVL (the sensitivity of the test) is 0.90, and the probability of a match by coincidence or error of two bullets from different CIVLs (the false positive probability) is 1 in 500 or 0.002 The likelihood ratio¹¹⁰ would then be 0.90/0.002 = 450. That can be interpreted in two ways: the probability of such a match is 450 times greater if the bullets came from the same melt than if they came from different melts, and the odds that the bullets came from the same melt are 450 times greater with the match evidence than without it (that is, there is no

evidence either way on matching).¹¹¹ With either interpretation, the evidence in this example would strongly support the conclusion that the bullets came from the same CIVL.

However, in reality the sensitivity and the false positive rate of CABL as applied by the FBI are not available. Therefore, the interpretation can be given only in qualitative terms: the probability of a match is greater if the bullets came from the same CIVL than if they came from different CIVLs, and the odds that the bullets came from the same CIVL are greater with the matching evidence than without it. Note that the witness may not testify as to the probability or odds that the bullets came from the same CIVL but only, in the first interpretation, as to the relative increase in probability of a match if the bullets came from the same vs different CIVLs or, in the second interpretation, as to the relative increase in the odds that the bullets came from the same CIVL if they matched vs no evidence of match status.

The admissibility of the above-described evidence depends on whether the assumption made above, namely, that bullets from the same CIVL have a greater probability of having the same composition as bullets from different CIVLs, has sufficient scientific support to be reliable. That requires us to look at two assumptions currently made in the use of CABL: homogeneity within CIVLs (which affects the likelihood that two bullets from the same CIVL have the same composition), and homogeneity between CIVLs (which affects the likelihood that two bullets from different CIVLs have the same composition.)

• Homogeneity within CIVLs. FBI expert witnesses frequently imply or state in their testimony that if bullets came from the same melt,¹¹² they will always match, that is, the test has perfect sensitivity. A single study by FBI personnel tested the assumption of homogeneity of melts and found it to be reasonable (sensitivity more than 90 percent).¹¹³ A study by critics of the assumption (Randich et al.) concludes that lead from a single melt can be inhomogeneous.¹¹⁴ Possible reasons for this conclusion were discussed. However, no measure of sensitivity is given in the study, and the authors did not publish the standard deviations of their measurements, so it cannot be determined to what extent the differences found were analytically indistinguishable. Despite the debate, the existence of inhomogeneity in a melt should not seriously affect the probative value of the evidence and may, in some respects, enhance it. We discuss the reason for this below.

Even if there is considerable inhomogeneity in a melt, two bullets that come from one melt and that have the same composition must have come from a subpart of the melt that was homogeneous. Fewer bullets can be made from a subpart than from the whole melt, so the fact of inhomogeneity within a melt, if it exists, does not weaken the inferences that can be legitimately made about matching bullets. However, because the degree of inhomogeneity will in general not be known, it must be assumed, conservatively, that the number of bullets of the same composition is such as would be produced from an entire melt. The principal risk of inhomogeneity is a false negative—two bullets declared not to match when they come from the same melt. Under our system of justice, such errors are less objectionable than false positives because they would usually favor a suspect.

The committee has addressed the issue of homogeneity by defining a source not as a melt, but rather as a CIVL (compositionally indistinguishable volume of lead), which may be limited to a subpart of a melt.

• False Positives. False positives occur when a laboratory error or a coincidence (two CIVLs with analytically indistinguishable composition) causes two bullets to match. The rate of laboratory error is unknown because the FBI Laboratory does not have a program of testing by an external agency that has been designed to assess the proficiency of its examiners. The FBI’s internal testing program does not appear to be designed to determine an error rate. If we

assume the laboratory’s error rate is in fact low (an assumption not currently grounded in evidence and made here only for the sake of the argument at hand), then the overwhelming contribution to the denominator of the likelihood ratio is CIVLs that are coincidentally identical in their composition.

The frequency of coincidentally identical CIVLs is unknown. Based on available data, the frequency of coincidental matches has been studied by the FBI. The data used in the FBI study have been further analyzed by the committee as described in Chapter 3. Those analyses have found some evidence supporting the assumption that the frequency of coincidental false positives is quite low. However, the FBI’s study is weakened because (1) the data used by the FBI were culled by the Bureau from a larger data set consisting of a collection of bullets analyzed by the FBI over a period of 14 years, and the method of culling may have introduced statistical bias; (2) the 2-SD overlap and range overlap method used by the FBI for declaring a match do not have quantifiable error rates (although approximate error rates can be calculated as in Chapter 3); and (3) the FBI study has been neither peer-reviewed nor published.¹¹⁵

The Daubert/Kumho factors previously referred to provide an indication of whether proposed expert testimony is sufficiently reliable to be admissible at trial. They expressly apply to the federal courts, to the state courts in those states that have adopted Daubert, and are likely to be influential to some degree in those states retaining the Frye standard. We briefly examine below the assumptions of homogeneity and low false positive error rates from this perspective.¹¹⁶

• Whether the theory can be and has been tested. Both homogeneity and a low false positive rate are assumptions that can be and have been tested, as described above and in Chapter 3. As noted in those discussions, the tests of both assumptions have weaknesses. For the reasons stated above, the assumption of homogeneity within a melt is not crucial to the value of the evidence. The

assumption of a low false positive rate is important. As the analysis in Chapter 3 indicates, the statistical method used by the FBI may be leading to a false positive rate much higher than that assumed by examiners. A statistical method can be chosen to minimize the false positive rate, but this is always done at the expense of a higher false negative rate. Additional testing would be needed to fully satisfy the Daubert/Kumho testing requirement.

• Whether the theory has been subjected to peer review and publication. There are very few peer-reviewed articles on homogeneity and the rate of false positive matches in bullet lead composition.¹¹⁷ Early articles focused on NAA¹¹⁸ and other techniques,¹¹⁹ used fewer elements in the analysis, and did not address the question of statistical interpretation. Moreover, some of the published articles appeared in FBI publications.¹²⁰ Outside reviews have only recently been published.¹²¹ Because this evidence is less than conclusive and the case volume that utilizes this technique is low, the subject has not received the broad review that DNA testing and some other techniques have. Again, more such work would be needed to provide a strong basis for this admissibility factor.

• Whether the theory has a known error rate. The false positive probability due to coincidence has been estimated by the FBI, as noted above, but has not been published. Furthermore, as discussed in Chapter 3, this estimate is not

based upon an appropriately random sample of the bullet population. Laboratory error is another important factor in the false positive probability; the FBI has not estimated this factor and assumes it is essentially zero. In sum, the Daubert/ Kumho factor requiring a theory to have a known error rate is only partially satisfied.

• The existence and maintenance of standards controlling the technique’s operation. The FBI has standards controlling the training of examiners, the laboratory protocol, and the statistical method for declaring a match. However, the laboratory protocol needs to be revised to reflect current practice.¹²² Moreover, the FBI does not have detailed standards governing the content of laboratory reports and the testimony that may be given by examiners. As a result, this Daubert/Kumho factor in significant part is not satisfied.

• General acceptance in the relevant scientific or technical community. The analytical technique used (that is, previously NAA and now ICP-OES) has general acceptance of the scientific community for this sample type. However, to the committee’s knowledge the FBI is the only laboratory performing this type of lead analysis for forensic use, so any inquiry into “general acceptance” will not provide the broad consensus that this factor assumes. The fact that courts have generally admitted this testimony is not the equivalent of scientific acceptance, owing to the paucity of published data, the lack of independent research, and the fact that defense lawyers have generally not challenged the technique.¹²³

The fact that the specifically mentioned Daubert factors are not fully satisfied does not mean that CABL evidence should not be admitted under the reliability standards of Rule 702. In Kumho Tire, the Court concluded “that a trial court may consider one or more of the more specific factors that Daubert mentioned when doing so will help determine that testimony’s reliability. But as the Court stated in Daubert, the test of reliability is “flexible,” and Daubert’s list of specific factors neither necessarily nor exclusively applies to all experts or in every case. Rather the law grants a district court the same broad latitude when it decides how to determine reliability as it enjoys in respect to its ultimate reliability determination.”¹²⁴ However, the reliability and acceptance of the evidence would be strengthened if the FBI took the steps that the committee recommends.

As noted earlier, relevance in this context depends not only on an association between the crime scene bullet and the same melt as the suspect’s bullet but also on the further inference that this association suggests that the crime scene bullet came from the defendant. A conclusion that two bullets came from the same melt does not justify an expert in further testifying that this fact increases the odds that the crime bullet came from the defendant. The large number of bullets made from a single melt and the absence of information on the geographic distribution of such bullets¹²⁵ precludes such testimony as a matter of expertise.¹²⁶ Such an inference is a matter for the jury. An expert with distributional information might be able to provide such testimony to aid the jury.

The available data do not permit any definitive statement concerning the date of manufacture or the identity of the manufacturer based on elemental composition alone. However, in some cases, boxes with lot numbers are recovered, which may provide some information on this issue.¹²⁷ In other cases, physical (as opposed to chemical) characteristics of crime bullets are observed, which

may augment the probative value of the evidence.¹²⁸ Also, “matches” of multiple crime scene bullets to multiple suspect’s bullets from different CIVLs may add to the probative value of the evidence in a particular case.¹²⁹ Similarly, a case with a “closed set” of suspects presents a different situation.¹³⁰

The need for pretrial disclosure of the nature and content of expert testimony is critical if the adversary system of trial is going to work. The American Bar Association (ABA) Standards note that the “need for full and fair disclosure is especially apparent with respect to scientific proof and the testimony of experts. This sort of evidence is practically impossible for the adversary to test or rebut at trial without an advance opportunity to examine it closely.”¹³¹ Never-

theless, pretrial discovery is often less extensive in criminal litigation than in civil cases.¹³²

Federal Criminal Rule 16 governs discovery in federal trials. Four distinct provisions are relevant to expert testimony: scientific reports, summaries of experts’ expected testimony, other documents,¹³³ and independent testing.¹³⁴

• Reports. Rule 16(a)(1)(F) makes the “results or reports of physical or mental examinations, and of scientific tests or experiments” discoverable. Under this provision, reports are discoverable if they are either material to the preparation of the defense or are intended for use by the prosecution as evidence in its case-in-chief at trial.¹³⁵ Unfortunately, the rule does not specify the content of a laboratory report. While the measurement data (means and standard deviations) on CABL evidence are discoverable, it is more logical and of greater use to include these data in the laboratory report.

The conclusions in laboratory reports should be expanded to include the limitations of CABL evidence.¹³⁶ In particular, a further explanatory comment should accompany the laboratory conclusions to portray the limitations of the evidence. Moreover, a section of the laboratory report translating the technical conclusions into language that a jury could understand would greatly facilitate the proper use of this evidence in the criminal justice system.¹³⁷ Finally, measurement data (means and standard deviations) for all of the crime scene bullets and those deemed to match should be included.

• Summaries. Rule 16(a)(1)(G) requires the government, on defense request, to disclose a written summary of the testimony of the experts that it intends to use during its case-in-chief. The summary must describe the witnesses’ opinions, the bases of and reasons for the opinions, and the witnesses’ qualifications. This provision was intended to “expand federal criminal discovery” in order to “minimize surprise that often results from unexpected expert testimony, reduce the need for continuances, and to provide the opponent with a fair opportunity to test the merit of the expert’s testimony through focused cross-examination.”¹³⁸ Although the ABA Standards recommend this type of discovery,¹³⁹ most states do not have comparable provisions.

• Conclusions. Like the NRC’s Committee on DNA Technology in Forensic Science, the present committee concludes that broad discovery is needed to the extent feasible: “The prosecutor has a strong responsibility to reveal fully to defense counsel and experts retained by the defendant all material that might be necessary in evaluating the evidence.”¹⁴⁰ As one court put it,

“there are no scientific grounds for withholding information in the discovery process.”¹⁴¹

A statement of the limitations of CABL evidence should be included in the laboratory report. Providing an express statement of the limitations of the technique in the laboratory report not only provides notice to the parties, it affords substantial protection for experts from overreaching by attorneys. Experts are sometimes pressured by the prosecutor to “push the envelope”—not a surprising occurrence in the adversary system.¹⁴² ABA Criminal Justice Standard 3-3.3(a) states: “A prosecutor who engages an expert for an opinion should respect the independence of the expert and should not seek to dictate the formation of the expert’s opinion on the subject. To the extent necessary, the prosecutor should explain to the expert his or her role in the trial as an impartial expert called to aid the fact finders….” The commentary to this standard states: “Statements made by physicians, psychiatrists, and other experts about their experiences as witnesses in criminal cases indicate the need for circumspection on the part of prosecutors who engage experts. Nothing should be done by the prosecutor to cast suspicion on the process of justice by suggesting that the expert color an opinion to favor the interests of the prosecutor.”¹⁴³

Finding: Variations among and within lead bullet manufacturers makes any modeling of the general manufacturing process unreliable and potentially misleading in CABL comparisons.

Recommendation: Expert witnesses should define the range of “compositionally indistinguishable volumes of lead” (CIVL) that could make up the source of analytically indistinguishable bullets, because of variability in the bullet manufacturing process.

Finding: The committee’s review of the literature and discussions with manufacturers indicates that the size of a CIVL ranges from 70 lbs in a billet to 200,000 lbs in a melt. That is equivalent to 12,000 to 35 million 40-grain, .22 caliber longrifle bullets from a CIVL compared with a total of 9 billion bullets produced each year.

Finding: CABL is sufficiently reliable to support testimony that bullets from the same compositionally indistinguishable volume of lead (CIVL) are more likely to be analytically indistinguishable than bullets from different CIVLs. An examiner may also testify that having CABL evidence that two bullets are analytically indistinguishable increases the probability that two bullets came from the same CIVL, versus no evidence of match status.

Recommendation: Interpretation and testimony of examiners should be limited as described above and assessed regularly.

Finding: Although it has been demonstrated that there are a large number of different compositionally indistinguishable volumes of lead (CIVLs), there is evidence that bullets from different CIVLs can sometimes coincidentally be analytically indistinguishable.

Recommendation: The possible existence of coincidentally indistinguishable CIVLs should be acknowledged in the laboratory report and by the expert witness on direct examination.

Finding: The available data do not support any statement that a crime bullet came from, or is likely to have come from, a particular box of ammunition, and references to “boxes” of ammunition in any form is seriously misleading under Federal Rule of Evidence 403.¹⁴⁴ Testimony that the crime bullet came from the defendant’s box or from a box manufactured at the same time is also objectionable because it may be understood as implying a substantial probability that the bullet came from defendant’s box.

Finding: Compositional analysis of bullet lead data alone do not permit any definitive statement concerning the date of bullet manufacture.

Finding: Detailed patterns of distribution of ammunition are unknown, and as a result an expert should not testify as to the probability that a crime scene bullet came from the defendant.¹⁴⁵ Geographic distribution data on bullets and ammunition are needed before such testimony can be given.

Recommendation: The conclusions in laboratory reports should be expanded to include the limitations of compositional analysis of bullet lead evidence.¹⁴⁶ In particular, a further explanatory comment should accompany the laboratory conclusions to portray the limitations of the evidence. Moreover, a section of the laboratory report translating the technical conclusions into language that a jury could understand would greatly facilitate the proper use of this evidence in the criminal justice system.¹⁴⁷ Finally, measurement data (means and standard deviations) for all of the crime scene bullets and those deemed to match should be included.