Forensic Analysis Weighing Bullet Lead Evidence (2004) / Chapter Skim
Currently Skimming:
Appendix K: Statistical Analysis of Bullet Lead Data by Karen Kafadar and Clifford Spiegelman
Appendix K: Statistical Analysis of Bullet Lead Data by Karen Kafadar and Clifford Spiegelman
Pages 169-214
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 169... ...
Are the data consistent with the hypothesis that the mean chemical concentrations of the two bullets are the same or different? If the data suggest that the mean chemical concentrations are the same, the bullets or fragments are assessed as "analytically indistinguishable." Intuitively, it makes sense that if the seven average concentrations (over the three measurements)
|
From page 170... ...
The converse is called the alternative hypothesis, for example, "drug is effective" or in the CABL context, "bullets match" or "mean concentrations are the same." 2. Determine an acceptable level of risk posed by rejecting the null hypothesis when it is actually true.
|
From page 171... ...
An alternative approach, based on the theory of equivalence t tests, is presented in Section 4. A level of risk is set for each equivalence t test to compare two bullets on each of the seven elemental concentrations; if the mean concentrations of all seven elements are sufficiently close, the overall false-positive probability (FPP)
|
From page 172... ...
The concentration distribution is based on the mean element concentrations and twice the standard deviation of the results for the known population composition group. If all mean element concentrations of a questioned specimen overlap within the element concentration distribution of one of the known material population groups, that questioned specimen is described as being "analytically indistinguishable" from that particular known group population.
|
From page 173... ...
is 14.74. Thus, the 2-SD overlap procedure would conclude that the two bullets are analytically indistinguishable (Ref.
|
From page 174... ...
3. The distributions of the concentrations of a given element across many different bullets from various sources are lognormally distributed with much more variability than seen from within-bullet measurement error or within-lot error.
|
From page 175... ...
This 800-bullet data set provided individual measurements on the three bullet lead samples which permitted calculation of means and SDs on the log scale and within-bullet correlations among six of the seven elements measured with ICP-OES (As, Sb, Sn, Bi, Cu.
|
From page 176... ...
The analysis of this data set confirms the homogeneity of the material in a lot within measurement error. TABLE K.3 Characteristics of 1,837-Bullet Data Set Element As Sb Sn Bi Cu An Cd No.
|
From page 177... ...
, one can estimate the measurement SD in each set of three measurements. As mentioned above, when the RSD is small, the lognormally distributed measurement error will have a distribution that is close to normal.
|
From page 178... ...
Because Sjj iS based on within-bullet SDs from 200 bullets, the square root of Sjj (called a pooled standard deviation) provides a more accurate and precise estimate of the measurement uncertainty than an SD based on only one bullet with three measurements (see Appendix F)
|
From page 179... ...
A plot of three points does not show very much, but one would not expect to see all 20 plots showing consistent directions if there were no association in the measurement errors of Sb and Cu. In fact, for all four manufacturers,
|
From page 180... ...
180 o I It 8 b 0 9 1l n~ r z z g 0L 0 0L n~ n~ O D lD g O O o O _ 1 1 1 1 1 Z O ; n~ n~ 0 -u~ 1111llt _° l~ _O _ 9 Z Z 9 SL S 0L b 0 b b tn Q o CM o Q b O n~ n~ b - U
|
From page 181... ...
The nonindependence will affect the overall false positive probability of a match based on all seven intervals. 1,837-Bullet Data Set Estimates of correlations among all seven elements measured with ICP-OES is not possible with the 1,837-bullet data set because the three replicates have been summarized with sample means and SDs.
|
From page 182... ...
(Robust estimates of the correlations can be obtained by trimming any terms in the summation that appear highly discrepant from the others.) A nonparametric estimate of the linear association, Spearman's rank correlation coefficient, can be computed by replacing actual measured values in the formula above with their ranks (for example, replacing the smallest Sb value with 1 and the largest with 1,837~.
|
From page 183... ...
The Spearman correlations on the ranks on the 1,837-bullet data set, the number of data pairs of which both elements were nonmissing, and the Spearman rank correlation coefficient on the 1,373-bullet subset (with no missing values) are given in Appendix F; the values of the Spearman rank correlation coefficients are very consistent with those shown in Table K.6.
|
From page 184... ...
184 Q U' o Q o °c~ oR oS a C ~C~ .~ ca~ o~, 1 °° O Oo° `^O Oo % ~CO=g ° ° ~o ~S oOO 8 ~,8 ° °~°C~° °~° `oOoWO ° oOo 8 ~% 9 8 o ~ t~Oo°ooOOO ,e8 O8 0~0 - ~0 Ro o o°c=O o o Oo o cbo ° Oo o - o ~ o o ~ OoC~ ° 3o oo C,O o ZD oo b o oOoO4D ,o o )
|
From page 187... ...
are discernible for measurements within lots on any of the elements, and, except for five lots with highly dispersed Cu measurements, the within-lot variability is about the same as or smaller than the measurement uncertainty (Appendix G)
|
From page 188... ...
188 ~ mm 41 u)
|
From page 189... ...
So, although the mean concentrations of elements in most of these 854 bullets differ by a factor that is many times greater than the measurement uncertainty, some pairs of bullets (selected by the FBI to be different) show mean differences that can be as small as 1 or 2 times the relative measurement uncertainty.
|
From page 190... ...
190 APPENDIX K TABLE K.8 Comparisons of 47 Pairs of Bullets from Among 854 of 1,837 Bullets Having Seven Measured Elements, Identified as Match by 2-SD-Overlap Method Bullet 1 Bullet 2 (Difference in Mean Concentration) /SD Elements No.
|
From page 191... ...
. Values less than 1 indicate that the measured mean difference in concentration is less than or equal to the measurement uncertainty (a 2~% in most cases)
|
From page 192... ...
We do not expect the sample means x; and yj to differ from the true mean concentrations ,UX and ,uy by much more than the measurement uncertainty (2 ~j/~ ~ 1.15oj) , but it is certainly possible (probability, about 0.10)
|
From page 193... ...
The CS average, x, is an estimate of the true mean concentration, fix; similarly, the PS average, y, is an estimate of its true mean concentration, ,uy. We simulate three measurements, normally distributed with mean fix= 1 and measurement uncertainty c,, to represent the measurements of the CS bullet, and three measurements, normally distributed with mean ,uy = ,UX + ~ and measurement uncertainty c, to represent the measurements of the PS bullet, and determine whether the respective 2-SD intervals and range intervals overlap.
|
From page 194... ...
< min~x~,x2,x3~. The minimum and maximum of three measurements in a normal distribution with measurement uncertainty c, can be expected to lie within 0.8463c, of the true mean, so, very roughly, range overlap occurs on the average when the difference in the sample means lies within 0.8463 + 0.8463 = 1.6926c, of each other.
|
From page 195... ...
mean concentrations for single element. Each curve corresponds to different level of measurement uncertainty (MU)
|
From page 196... ...
, for different levels of 6 Levels of measurement uncertainty (a) = 0.5%, 1.0%, 1.5%, 2.0%, 2.5%, 3.0% FIGURE K.6 Plot of estimated FPP for FBI 2-SD-overlap procedure as function of ~ = true difference between (log~mean concentrations for seven elements, assuming independence among measurement errors.
|
From page 197... ...
. Figures K.7 and K.8, and Tables K.ll and K.12 give the corresponding FPPs, assuming independence among the measurement errors on all seven elements and assuming that the true mean difference in concentration is 100 ~ percent.
|
From page 198... ...
, 1 element 0 1 ~ (% difference between means) analytical error c: 0.5%, 1.0%, 1.5%, 2.0%, 2.5%, 3.0% `2.5°~ 2.0°~ 6 FIGURE K.7 Plot of estimated FPP for FBI range-overlap procedure as function of ~ = true difference between Log mean concentrations for single element.
|
From page 199... ...
analytical error c,: 0.5%, 1.0%, 1.5%, 2.0%, 2.5%, 3.0% FIGURE K.8 Plot of estimated FPP for FBI range-overlap procedure as function of ~ = true difference between (log~mean concentrations for seven elements, assuming independence among measurement errors. Each curve corresponds to different level of measurement uncertainty (MU)
|
From page 200... ...
The simulation suggests that measurement uncertainty may exceed 2-2.5%, and/or the measurement errors may be correlated. Note that the FPP computation would be different if the mean concentrations differed by various amounts.
|
From page 201... ...
that was used for the calculation from Table K.8 (0.9027 = 0.486, but 0.90264 = 0.517~. Because homogeneous batches of lead, manufactured at different times, could by chance have the same chemical concentrations (within measurement error)
|
From page 202... ...
The FBI 2-SD-overlap procedure declares a match on an element if the mean difference in concentrations lies within twice the sum of the standard deviations that is.
|
From page 203... ...
203 - ~ ~ A o (a - m 5' _ CO SL00 0~00 SOOO 0000 uo! ~ueauo~ SL00 0~00 SOOO 0000 uo!
|
From page 204... ...
· The procedure is designed to claim a match only if the true mean element concentrations differ by roughly the measurement uncertainty (6 ~ c, ~ 2 - 4%) or, at most, ~ ~ 1.5c, ~ 3-6%.
|
From page 205... ...
That is, assuming that the uncertainty measuring a single element is 2.5 percent and the true mean difference between two bullet concentrations on this element is at least 2.5 percent [3.8 percent] , then, with a probability of 0.30, caused by the uncertainty in the measurement process and hence in the sample means x; and yj, the two sample means will, by chance, lie within 0.63sjpoo`7 [or 1.071 of each other, and the bullets will be judged as analytically indistinguishable on this one element (even though the mean concentrations of this element differ by 2.5%~.
|
From page 206... ...
, and that mean concentrations with ~ = c, (that is, within the measurement uncertainty) are considered analytically indistinguishable.
|
From page 207... ...
on F002 ICP-Sb ICP-Cu ICP-Ag ICP-Bi ICP-As ICP-Sn a 10.27491 5.62762 4.33073 2.77259 7.29506 7.52994 b 10.26928 5.63121 4.20469 2.77259 7.27170 7.49387 c 10.27135 5.64191 4.34381 2.70805 7.28001 7.47760 mean 10.27185 5.63358 4.29308 2.75108 7.28226 7.50047 SD 0.00285 0.00743 0.07682 0.03726 0.01184 0.02679 Sj pool 0.0192 0.0200 0.0825 0.0300 0.0432 0.0326 RMD Sj pool 0.631 0.233 -0.916 0.717 -0.347 0.241 distribution) , and that a "CIVL" has been defined to be as small a volume as is needed to ensure that the variability of the elemental concentrations within this volume is much smaller than the measurement uncertainty (i.e., within-lot variability is much smaller than c,)
|
From page 208... ...
See also Box K.1 4.3 Hotelling's 1~ A statistical test procedure that is designed for comparing two sets of 7 sample means simultaneously rather than 7 individual tests, one at a time, as in the previous section, uses the estimated covariance matrix for the measurement errors. The test statistic can be written t2 =nd'S-~d =nCd/S4'R-~(d/ where: · n = number of measurements in each sample mean (here, n = 3~.
|
From page 211... ...
(When applied to the log~concentrations) on Federal bullets F001 and F002 in Table K.14, the value of Hotelling's T2 statistic, using only six elements, is 2.354, which is small enough to claim "analytically indistinguishable" when 6/c, = 1.0 and the overall FPP is 0.002, or 1 in 500.)
|
From page 212... ...
Many more studies would be needed to assess the reliability of Hotelling's T2 (for example, types of differences typically seen between bullet concentrations, precision of estimates of the variances and covariances between measurement errors, and departures from Log normality. 4.4 Use of T Tests in Court One reason for the authors' recommendation of seven individual equivalence t tests versus its multivariate analog based on Hotelling's T2, is the familiarity of the form.
|
From page 213... ...
In each trial, 3 measurements on seven elements were simulated from a normal distribution with mean vector fix, standard deviation vector ox, and within-measurement correlation matrix R where ,UX is the vector of 7 mean concentrations from one of the bullets in the 854-bullet data set, ox is the vector of 7 standard deviations on this same bullet, and R is the within-measurement correlation matrix based on data from 200 Federal bullets (see Appendix F)
|
From page 214... ...
D "Bullet Lead Elemental Composition Comparison: Analytical Technique and Statistics." Presentation to committee.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.