Measurement Problems in Criminal Justice Research Workshop Summary (2003) / Chapter Skim
Currently Skimming:
2. Measuring Crime and Crime Victimization: Methodological Issues
2. Measuring Crime and Crime Victimization: Methodological Issues
Pages 10-42
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 10... ...
Skogan details the large margins of sampling error for many key estimates from the National Crime Survey (NCS) , a survey that used a very large sample (and which later evolved into the National Crime Victimization Survey)
|
From page 11... ...
, underwent lengthy development periods featuring record check studies and split-ballot experiments to determine the best way to measure crime victimization. In the records check studies, the samples included known crime victims selected from police records.
|
From page 12... ...
Two recent controversies illustrate the problems of the NCVS and of crime surveys more generally. One controversy centers on the number of incidents of defensive gun use in the United States; the other concerns the number of incidents of rape.
|
From page 13... ...
, interviewed respondents by different methods (in-person versus telephone) , covered different recall periods (six months in NCVS versus five years in the Kleck study)
|
From page 14... ...
Once again, the studies from which the estimates are drawn differ in many crucial particulars they sample different populations, ask different questions that are based on different definitions of rape, conduct data collection via different methods, and cover different recall periods. As with the estimates of defensive gun use, what is surprising is not that the estimates differ from each other but that they differ so widely.
|
From page 15... ...
Once the respondent completes these initial screening items, further questions gather more detailed information about each incident; the final classification of an incident in the NCVS reflects these detailed reports rather than the answers to the initial screening questions. Most of the other surveys on rape differ from this procedure in two key ways first, they ask multiple screening questions specifically crafted to elicit reports about rape and, second, they omit the detailed follow-up questions.
|
From page 16... ...
There were twice as many positive answers to the rape screening questions as there were incidents ultimately classified as rapes based on the detailed reports. (The rape screening items also captured many incidents involving some other type of sexual victimization.)
|
From page 17... ...
First, we address the potentially sensitive nature of the questions on many crime surveys and the impact of the mode of data collection on the answers to such questions. This is followed by an examination of the effects of the context in which survey items are presented, including the physical setting of the interview, the perceived purpose and sponsorship of the study, and .
|
From page 18... ...
First, various technological changes such as the introduction of lighter, more powerful laptop computers, development of the World Wide Web, widespread adoption of e-mail, and improvements in sound card technology have made the new methods possible. Second, the need for survey data on sensitive topics, such as illicit drug use and sexual behaviors related to the spread of AIDS, has made the new methods highly desirable, since they combine the privacy of self-administration with the power and flexibility of computer administration.
|
From page 19... ...
Year Time Period Lifetime administration, Figure 2-1 plots the ratio between the level of illicit drug use reported when survey questions are self-administered to the level reported when interviewers administer the questions. For example, if 6 percent of respondents report using cocaine during the previous year under self-administration but only 4 percent report using cocaine under interviewer administration, the ratio would be 1:5.
|
From page 20... ...
A number of other national surveys have also recorded whether other people are present during the interviews, but researchers who have examined these data have found little evidence that the presence of others affects reports about such potentially sensitive topics as sexual behavior (Laumann et al., 1994) or illicit drug use (Schober et al., 1992; Turner et al., 19921.
|
From page 21... ...
Still, in crime surveys one would expect the presence of family members to have an impact, particularly on reports involving domestic violence; similarly, in surveys on rape the presence of family members is likely to inhibit reports of spousal rape, if not rape in general. In fact, there is some recent evidence that the presence of a spouse during an interview is associated with reduced reporting of rape and domestic violence (Coker and Stasny, 19951.
|
From page 22... ...
Receptive Oral Sex What all of these findings suggest is that crime reports especially reports about victimizations involving crimes that carry stigma could be dramatically affected by the mode of data collection. Self-administration of the questions is likely to increase the number of rape victimizations reported; on the other hand, it may sharply reduce reports of defensive gun use, since defensive gun use is likely to be seen as a positive or socially desirable response to crime (as Hemenway, 1997, argues)
|
From page 23... ...
The interview touched on a number of sensitive topics, including sexual partners, sexually transmitted diseases, illicit drug use, and abortion. No consistent impact on reporting by interview site was found.
|
From page 24... ...
Interviews as Conversations Unfortunately, in the process of interpreting survey questions, respondents often draw unintended inferences about their meaning. They draw these incorrect inferences because they carry over into the survey interview interpretive habits developed over a lifetime in other settings, such as conversations (Schaeffer, 19911.
|
From page 25... ...
Grice (1989) distinguished four specific manifestations of the cooperative principle four conversational maxims that shape everyday conversations and serve as a kind of etiquette for them: · the maxim of quantity, which requires us to make our contribution as informative as necessary, but not more informative; · the maxim of quality, which says to tell the truth and to avoid statements we cannot support; rlty.
|
From page 26... ...
If "0" conveys essentially no success in life, "-5" suggests catastrophic failure. Respondents may also draw unintended inferences based on other features of the questions or their order.
|
From page 27... ...
Over the past 15 years or so, survey researchers have begun to apply findings from the cognitive sciences, particularly cognitive psychology, in a systematic program to understand reporting errors in surveys (see Sirken et al., 1999; Su~man et al., 1996; Tourangeau et al., 2000, for recent reviews of this work)
|
From page 28... ...
It covers both the respondents themselves and other members of their households; it lists several possible relevant scenarios ("for self-protection or for the protection of property at home, work, or elsewhere"; and it notes several exclusions (incidents involving "military service and police security worked. The demands of interpreting this item, mentally implementing its complicated logical requirements, and searching memory for relevant incidents over such a long period are likely to exceed the working memory capacity of many respondents, even well-motivated ones Just and Carpenter, 19921.
|
From page 29... ...
compared detailed reports about incidents with responses to the rape screening items in the National Violence Against College Women Study, they classified only about a quarter of the incidents mentioned in response to the rape screening items as actually involving rapes. (Additional incidents that qualified as
|
From page 30... ...
Accorcling to Grice's maxim of relation, the participants in a conversation are supposed to stick to the topic; they are not supposed to shift gears without giving proper warning. For the most part, survey questions follow these rules, signaling shifts in topics with introductions or transitional phrases ("The next few items are about .
|
From page 31... ...
Another way that prior items can affect answers to later questions is by reminding respondents of things they would not have otherwise recallecl. The process of retrieving information from memory is partly deliberate and partly automatic.
|
From page 32... ...
" may help prompt fuller recall of more serious incidents but may also suggest that almost any unwanted sexual experience is of interest, encouraging overreporting. Bounding Another procedure used in some crime surveys may also help frame later questions for respondents and trigger the recall of relevant events; this is the review of incidents reported in the previous round as part of the bounding procedure.
|
From page 33... ...
Bounding procedures can serve still another function both previously reported incidents and landmark events can serve as powerful retrieval cues. When people are asked to remember events from a given period, they tend to recall incidents that fall near temporal boundaries, such as the beginning of the school year or major holidays (Kurbat et al., 1998; Robinson, 19861.
|
From page 34... ...
(McDowell et al., 2000, and Fisher and Cullen, 2000, are exceptions they present evidence testing specific hypotheses about why different procedures gave different results.) Throughout, we have offered conjectures about the variables that affect reporting in crime surveys.
|
From page 35... ...
Our first hypothesis, then, is that self-administration will dramatically increase reports of some types of crime, particularly those that carry stigma and those perpetrated by other household members; self-administration will reduce reports of incidents that put the respondent in a favorable light, including perhaps defensive gun use. A related hypothesis involves the presence of other household members; for the topics raised in crime surveys, we believe that the presence of other household members must make a difference (at least for crimes involving domestic violence)
|
From page 36... ...
By itself a prior interview may not be all that effective as a bounding event; our final hypothesis is that a full bounding procedure that includes a review of the incidents reported in the previous interview will reduce reporting relative to the more truncated procedure used in many surveys that simply instructs respondents to report incidents that occurred since the last interview. Compared to temporal or personal landmarks, the prior interview may not mark off the relevant time period very clearly.
|
From page 37... ...
Cantor 1984 A longitudinal analysis of bounding, respondent conditioning, and mobility as sources of panel bias in the National Crime Survey. In Proceedings ofthe American Statistical Association, Survey Research Methods Section.
|
From page 38... ...
Stasny 1995 Adjusting the National Crime Victimization Survey's Estimates of Rape and Domestic Violencefor "Gag"Factors. Washington, DC: U.S.
|
From page 39... ...
Public Opinion Quarterly 62:610-622. Mosher, W.D., and A.P.
|
From page 40... ...
Colic Opinion Quarterly 55 :618-630.
|
From page 41... ...
Anderson 1986 The presence of others and overreporting of voting in American national elections. Public Opinion Quarterly 50:228-239.
|
From page 42... ...
Miller 1998 Automated self-interviewing and the survey measurement of sensitive behaviors. In Computer-Assisted Survey Information Collection, M.P.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.