Skip to main content

The Polygraph and Lie Detection (2003) / Chapter Skim
Currently Skimming:

2 Validity and Its Measurement
Pages 29-64

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 29...
... RELIABILITY, ACCURACY, AND VALIDITY Psychophysiological testing, like all diagnostic activities, involves using specific observations to ascertain underlying, less readily observable, characteristics. Polygraph testing, for example, is used as a direct measure of physiological responses and as an indirect indicator of whether an examinee is telling the truth.
From page 30...
... Internal consistency is another aspect of reliability. For example, a polygraph test may be judged to indicate deception mainly because of a strong physiological response to a single relevant question.
From page 31...
... Criterion Validity (Accuracy) Criterion validity refers to how well a measure, such as the classification of polygraph test results as indicating deception or nondeception, matches a phenomenon that the test is intended to capture, such as the actual deceptiveness or truthfulness of examiners on the relevant questions in the test.
From page 32...
... Important threats to construct validity for this theory come from the fact that the physiological correlates of psychological arousal vary considerably across individuals, from the lack of scientific evidence to support the claim that deception has a consistent psychological significance for all individuals, and from the fact that psychological arousal is associated with states other than deception. We discuss these issues further in Chapter 3.
From page 33...
... This is one reason that evidence of accuracy, though necessary, is not sufficient to demonstrate test validity. X-ray screening is not presumed to have perfect validity: this is why objects deemed suspicious by X-rays are checked by direct inspection, thus reducing the number of false positive results on the X-ray examination.
From page 34...
... Accuracy of this screening polygraph might be defined as the extent to which the polygraph scoring corresponds to actual truthfulness of responses to these target questions. It might also be defined for a multi-issue polygraph screening test as the extent to which the test results correctly identify which of the target behaviors an examinee may have engaged in.
From page 35...
... Some representatives of the DOE polygraph screening program believe that the program is highly accurate because all 85 employees whose polygraphs indicated deception eventually admitted to a minor security infraction. If detecting minor security violations is the target of a security polygraph screening test, then these 85 are all true positives and there are no false positives.
From page 36...
... Another difficulty in measuring the accuracy of preemployment polygraph tests is that adverse personnel decisions made on the basis of preemployment polygraph examinations are not necessarily due to readings on the polygraph chart.4 For instance, we were told at the FBI that applicants might be rejected for employment for any of the following reasons: (1) they make admissions during the polygraph examination that specifically exclude them from eligibility for employment (e.g., admitting a felony)
From page 37...
... For example, cholesterol tests give a range of values that are typically collapsed into two or three categories for purposes of medical decision: high risk, justifying medical intervention; low risk, leading to no intervention; and an intermediate category, justifying watchful waiting or low-risk changes in diet and life-style, but not medical intervention. Polygraph tests similarly
From page 38...
... are two closely related measures of test performance that are critical to polygraph screening decisions.6 The FPI is the ratio of false positives to true positives and thus indicates how many innocent examiners will be falsely implicated for each spy, terrorist, or other major security threat correctly identified. The PPV gives the probability that an individual with a deceptive polygraph result is in fact being deceptive.
From page 40...
... . The choice of this particular cutoff point represents the judgment, common in medical diagnosis, that it is more important to avoid false negatives than to avoid false positives.
From page 41...
... 41 to Last in : o a IS ~ \.
From page 42...
... The effect would be to undermine claims that the quality of polygraph examinations is sufficiently controlled that a polygraph test result has the same meaning across test formats, settings, and agencies. As shown in the second panel of Figure 2-1, any given decision threshold will produce a certain proportion of true-positive decisions (equal to the shaded proportion of total area under the curve in the upper part of the panel, which represents examiners with the target condition present)
From page 43...
... Measure of Accuracy The position of the ROC on the graph reflects the accuracy of the diagnostic test, independent of any decision thresholds that may be used. It covers all possible thresholds, with one point on the curve reflecting the performance of the diagnostic test for each possible threshold, expressed in terms of the proportions of true and false positive and negative results for each threshold.
From page 44...
... / // 1.0 0.8 0.6 0.4 0.2 0.0 1 .o 4' 1 1 1 1 _ - 0.0 0.8- _ S': A' 06 / / F f 0.2 0.4 .2 7 -0.6 ~ IL -0.8 0.O ~ V 1 1 1 1 -1.0 0.0 0.2 0.4 0.6 0.8 1.0 False Positive Rate FIGURE 2-2 A representative plot of the receiver operating characteristic (ROC) curve for a diagnostic test with accuracy index (A)
From page 45...
... Similarly, for any false positive rate, shown on the axis at the bottom of the figure, the more accurate the test, the greater the proportion of positive cases that are accurately identified.8 Decision Thresholds Figure 2-2 shows three points corresponding to different thresholds on a curve with A = 0.8. The point B is the balanced threshold, meaning
From page 46...
... , many more people are called deceptive: the test catches 88 percent of the examiners who are being deceptive, but at the cost of falsely implicating 50 percent of those who are not.9 Selection of Decision Thresholds Decision theory specifies that a rational diagnostician faced with a set of judgment calls will adopt a threshold or cutoff point for making the diagnostic decisions that minimizes the net costs of false positive and false negative decisions. If all benefits and costs could be measured and expressed in the same units, then this optimal threshold could be calculated for any ROC curve and base rate of target subjects (e.g., cases of deception)
From page 47...
... The eight-category scale allows for seven possible thresholds for dividing the charts into groups judged truthful or deceptive. An indication of the results of using different decision thresholds among polygraph interpreters is the false positive proportions that would result if each interpreter had set the threshold at the fifth of the seven possible thresholds and had made yes/no, binary judgments at that cutoff.
From page 48...
... 48 5U ~ o ~ to o o ~ MU .~ ~ Do o 11 it .
From page 49...
... Treating this three-alternative scoring system as a rating procedure gives a two-point ROC curve.~° Because of the way polygraph data are most commonly reported, our analyses in Chapter 5 draw heavily on two-point ROC curves obtained when "no-opinion" or "inconclusive" judgments are reported. Using the Percent Correct to Measure Accuracy Treating no-opinion or inconclusive judgments as an intermediate category and estimating two ROC points handles neatly a problem that is not dealt with when percent correct is used to estimate accuracy.
From page 50...
... When the base rate of guilt is 0.1 percent, it makes 84, 97, and 99.5 percent correct classifications with the same three thresholds. Finally, as a single-number index, the percent correct does not distinguish between the two types of error false positives and false negatives which are likely to have very different consequences.
From page 51...
... VALIDITY AND UTILITY The practical value of polygraph testing depends on at least five conceptually different factors that are often not distinguished: · The ability to detect deception from polygraph charts by analyzing the data collected by polygraph instruments (i.e., psychophysiological detection of deception)
From page 52...
... That is, the polygraph is said to be valid only if deception is strongly and uniquely associated with a discernable pattern in the record of physiological responses made on or from the polygraph. Chapter 3 discusses the scientific basis for believing that deception produces specific psychological and physiological processes that influence polygraph readings, which indicates the construct validity of the polygraph test.
From page 53...
... if they are facing a polygraph examination. In addition, people in sensitive positions may take greater care to avoid even minor security infractions in order to avoid the possibility of a future deceptive reading on a polygraph test.
From page 54...
... It is worth noting that deterrence has costs as well as benefits for an organization that uses polygraph testing. The threat of polygraph testing may lead desirable job candidates to forgo applying or good employees to resign for fear of suffering the consequences of a false positive polygraph result.
From page 55...
... Admissions during polygraph testing of acts that had not previously been disclosed are often presented as evidence of the utility and validity of polygraph testing. However, the bogus pipeline research demonstrates that whatever they contribute to utility, they are not necessarily evidence of the validity of the polygraph.
From page 56...
... This practice makes it impossible to assess the validity of federal polygraph screening programs from the data those programs provide. Polygraph examinations that yield admissions may well have utility, but they cannot provide evidence of validity unless the circumstances of the admission are taken into account and unless the veracity of the admission itself is independently confirmed.
From page 57...
... Rather, those effects depend on people's beliefs about validity. Admissions and confessions, as noted above, provide evidence supportive of the validity of polygraph tests only under very restricted conditions, and the federal agencies that use the polygraph for screening do not collect data on admissions and confessions in a form that allows these field tests to be used to assess polygraph validity.
From page 58...
... CRITERION VALIDITY AS VALUE ADDED For the polygraph test to be considered a valid indicator of deception, it must perform better against an appropriate criterion of truth than do
From page 59...
... Bogus pipeline research illustrates what might be involved in assessing validity of the polygraph using an experimental condition analogous to a placebo. An actual polygraph test might be compared with a bogus pipeline test in which the examinee is connected to polygraph equipment that, unbeknownst both to examiners and examiners, produced charts that were not the examinee's (perhaps the chart of a second examinee whose actual polygraph is being read as the comparison to the bogus
From page 60...
... CONCLUSIONS Validity and Utility · The appropriate criteria for judging the validity of a polygraph test are different for event-specific and for employee or preemployment screening applications. The practical value of a polygraph testing and scoring system with any given level of accuracy also depends on the application because in these different applications, false positive and false negative errors differ both in frequency and in cost.
From page 61...
... · There is little awareness in the polygraph literature and less in U.S. polygraph practice of the concept that false positives can be traded off against false negatives by adjusting the threshold for declaring that a chart indicates deception.
From page 62...
... 6. The false positive index is not commonly used in research on medical diagnosis but seems useful for considering polygraph test accuracy.
From page 63...
... A different distinction between validity and utility is made in some writings on diagnostic testing (Cronbach and Gleser, 1965; Schmidt et al., 1979~. That distinction concerns the practical value of a test with a given degree of accuracy in particular decision-making contexts, such as screening populations with low base rates of the target condition.
From page 64...
... 13. We found many polygraph validation studies in which assessment was done only by tests of statistical significance without any attempt to estimate effect size or strength of association.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.