Skip to main content

Currently Skimming:

Chapter 2: After the Seminar
Pages 25-70

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 25...
... The first section describes a multiyear collaborative research program involving cognitive scientists and survey researchers . The plan for this program, which is already under way, was developed by CASM participants Sirken and Fuch~berg for the National Center for Health Statistics (NCHS ~ .
From page 26...
... Although survey researchers and cognitive scientists are both concerned with the manner in which individuals handle information, their approaches to the problem and the methods used to study the problem are quite different, and there has been very little communication between them. Surrey researchers are concerned about the survey measurement process and use field experiments to test response effects in terms of the wording, response categories, and orderings of.
From page 27...
... Benefits The laboratory is the ideal setting for conducting interdisciplinary research in which the combined technologies of the cogniti Ire, social, biological, and computer sciences can be applied in researching the cognitive aspects of survey methodology. Participation of NCHS staff in the interdisciplinary laboratory, as described later in this plan when discussing collaborative arrangements, will help to bridge the gap that currently exists between government agency survey methodologists and university survey researchers and social scientists.
From page 28...
... Work Plan Experiments will be conducted to tent the application of laboratory-based methods for two broad types of questionnaire design problems: development of survey instruments (2) investigation of specific cognitive issues The NHIS questionnaire will be used by the laboratory as the survey instrument for both types of experiments.
From page 29...
... These activities exemplify the traditional method of constructing survey instruments. The sine qua non of this method is that the instruments are field tented under conditions that simulate the actual survey conditions as closely as possible.
From page 30...
... Testing the Field Pretest Draft of the NHIS Supplement During the four-month period, December 1985 until the subcontract ends in March 1986, the field pretest versions of the NHIS supplements will be pretested in the laboratory and then independently field pretested during April 1986. Pretesting identical drafts of the NHIS supplement by both laboratory and field methods will make it possible to compare and evaluate how well each method assessed whether the NHIS supplement was doing what it is supposed to be doing and, if not, what revisions were needed.
From page 31...
... And feeding the laboratory research findings on these cognitive issues back to survey scientists will, in turn, -- simulate the development of improved statistical models of survey measurement errors and improved methods of constructing survey instruments. For cognitive science, the ultimate benefit will be a better understanding of the way people process information, and for survey science, it will be improved control over the cognitive component of survey measurement.
From page 32...
... Conditioning in survey research usually refers to the distorting effect of the total survey measurement process on survey responses, but in the narrower sense used here it refers more specifically to the response effects of collecting an item or ret of items of information on another item or set of information items. For example, it refers to the response effects of adding one set of questions to another set of questions, such as the effects of NHIS core questions on the questions in the NHIS supplement or vice versa.
From page 33...
... Perceptions of Confidentiality The response effects of asking for information about sensitive topics is a major survey concern because (1) policy makers and other users of health survey data often require this type of information, and (2)
From page 34...
... The problem of respondent compliance in surveys on sensitive topics is representative of a wider class of cognitive phenomena in which people are faced with the task and the rink of making decisions on the basis of information that they may neither fully comprehend nor believe. Numerous applications of the anonymity techniques in surveys on sensitive topics have produced mixed results that traditional survey research methods have been unable to explain satisfactorily.
From page 35...
... The University of Chicago program, developed by William Salter, Steven Shevell, Lance Rips, and Norman Bradburn, in entitled Recognitive Processes in Survey Responding: Time and Frequency Estimation.8 It focuses on the remaining class of survey questions, those that concern behavior. It includes studies on the retrieval and Judgment processes and on how these processes interact when respondents must Judge the timing or frequency of events.
From page 36...
... NORC Research Program Roger Tourangeau, Roy D'Andrade, and Norman Bradburn We propose a series of laboratory studies that explore cognitive processes in survey responding. The studies focus on questions that elicit attitudes or opinions and on those that ask for the reasons or causes of behavior.
From page 37...
... The two types of attitudes may have different relations to demographic and other variables and may be important to distinguish in surveys. University of Chicago Research Program Lance Rips, Norman Bradburn, Steven She~rell, and William Salter This research will examine the cognitive processes and knowledge representations that are implicit in responses to many survey items.
From page 38...
... many of the responses, such ~s the relative salience of , diverse familiar historical events, the levels of personal and national importance ascribed to each, and the nature of the impacts felt to have resulted. Nonetheless, given the t~me-dependent focus of the study, it will be important to do much analytic work having partitioned the sample by age cohorts.
From page 39...
... The study would be carried out using experienced professional Survey Research Center telephone interviewers. This will considerably reduce the cost over face-to-face interviewing, without appreciably reducing coverage or response rates.
From page 40...
... 40 Thirst, I'd like you to think back and tell me what you personally feel have been the one or two most important national or world events or changes in the past 50 years. ~ ~ If only one given: "Is there any other national or world event or change over the past 50 years that you personally feel was important?
From page 41...
... Nonetheless, for some events -- especially con~croversial ones like the Vietnam War -- it would be of interest to learn something about the drift in collective memory an the event has receded in time. Toward this end, we would like to resurrect one or more opinion poll items that were frequently used at the time to measure the division of sentiment, and pose it again to our "moderns respondents, first by way of asking what their current reactions to the event are; and second, by asking them to respond to it as they feel they would have at the time.
From page 42...
... lathe research can also be viewed an a study of social memory that parallels laboratory studies of memory. Those studies document both primacy and recency effects in single-~ession tests of memory (Martindale, 1 981)
From page 43...
... 43 neither. At the same time, attention is frequently addressed to the perceived long-run significance of such events and trends on those who have lived through them, typically based on documents and testimony from individuals who are obviously unrepresentative survivors.
From page 44...
... Memory capabilities of the population, together with other cognitive abilities, represent an important part of the nation's intellectual resources. No general and systematic information regarding the quality of these resources is available at the present time.
From page 45...
... These different kinds of memory operate according to somewhat different principles and, at the level of psychometric analyses, show different correlational patterns of individual differences. Thus, students of memory have talked about visual versus auditory memory, verbal memory versus pictorial memory, rote memory versus meaningful memory, voluntary versus involuntary memory, as well as about facial memory, spatial memory, recall memory, recognition memory, and many other sorts of specialized memories.
From page 46...
... (Note that in the realm of semantic memory, there is usually no need to distinguish between the memory task and the memory test: a semantic memory tack can consist of nothing else but a test.) No generally accepted standard battery of memory tasks exists.
From page 47...
... Thus, there exists a potentially very large set of memory tasks, that is, combinations of materials and conditions of their study and test. The exact consititution of the battery that eventually would be used represents one of the many sub-problems that would have to be solved in the course of the project.
From page 48...
... The maximum score is 24. ~ 5 ~ ~ v ~ -- yes/no recognition test .
From page 49...
... For thin reason, no delayed free-recall tests are included in the battery. The test for the order of the six tasks of Part A (A7)
From page 50...
... Respondents' performance with the ~new" fragments of this test provides a measure of one aspect of their semantic memory. Procedures of Data Collection and Analysis The idea in to administer such a battery of memory tasks to a large probability sample of the U.S.
From page 51...
... Thi ~ should all be done on a small scale before anything major in attempted. The survey instrument would include a questionnaire designed to provide background information on each subject appropriate to relating reduced memory scores (x)
From page 52...
... It will be necessary to define a sample of individuals who will be used to establish the basic populations that will then be used for future classification purposes. The Memory Battery: Instructions, Materials, and Tests A somewhat more complete description of a possible battery of memory tasks is given next .
From page 53...
... Materials: 24 line drawings of common objects (sample objects shown below) : BASKET GLOVE LION LEAF MOUSE BELT COATHANGER FROG TIE BED BATH CHAIR CANDLE SNOWMAN BOTTLE BUTTERFLY BELL ZEBRA HAMMER AIRPLANE RIFLE SAILBOAT FLAG KEY in, Transition: nThat's all for the pictures.
From page 54...
... I will show you a number of faces and later on ask you to recognize them. Pay close attention to each [ace [or you'll see it only once, very briefly.
From page 55...
... 55 Presentation: Present 6 categories of 3 words at the rate of 12 neconds/category. Record for each category whether the subject named it or whether you had to provide it.
From page 56...
... Looking at line drawings of common objects c. Recalling short 3-letter words d.
From page 57...
... Here we gold Four-alternative forced-choice recognition test of pictures of objects: Present to the subject 24 sets of test drawings and allow the subject up to 10 seconds/set to choose one of the alternatives. Record
From page 58...
... 1 58 subject up to 10 seconds/~et to choose one of the alternatives. Record the choice and whether the subject reports remembering or guessing.
From page 59...
... We are now ready to test your memory for them. It is a recognition memory test.
From page 60...
... A 1982 Priming effects in word-fragment completion are independent of recognition memory.
From page 61...
... To explore the feasibility of a protocol analysis approach to the problem of how people retrieve personal experiences of the type required on, say, the National Health Interview Survey, we asked five pilot subjects to think aloud while answering specific questions. We first gave subjects some practice questions so they could gain experience in Verbalizing their thought processes.
From page 62...
... . six months ago ~ went to the dentist.
From page 63...
... . Yes, ~ was -- T was thinking about my car, and I had come tapes stolen from my car, in Montlake, about six months ago." We could speculate that if subjects had been responding using a more formal checklist technique in which they simply had to Ray eyed or "no" that these two instances might never have been reported.
From page 64...
... , the results were as follows: Health Care: A La Card: all + all both ~ and no information 4 2 16 23 all + all both + and - 3 no information ~ 20 7 2 These results were taken as evidence that, at least in response to these questions, subjects tend to retrieve autobiographical memories in a predominantly past-to-present, or forward, direction. References Ericsson, E.A., and Simon, H.A.
From page 65...
... While such precautions may be entirely appropriate in the context of political surveying or other attempts to ascertain attitudes, beliefs, or other subjective states of the respondent, they may be less necessary when the responses in question deal with specific concrete past actions and outcomes by the respondent. Furthermore, in attempting to determine the respondent's past actions and outcomes through measurement of his or her recollections or estimates, a rather different set of potential sources of error or bias come into play -- i.e., the types of factors with which cognitive psychologists have long been concerned in their study of human memory and Judgment.
From page 66...
... e., to compare subjects' certainty (or, to be more precise, the range of their uncertainty) about the frequency and magnitude of their own past behaviors or outcomes with their certainty about frequencies for parallel future responses.
From page 67...
... can be summarized succinctly. Confidence limits for predictions of the future were only TABLE ~ Relative Width of 50 Percent Confidence Intervals for Estimates of Past Behavior and Predictions of Future Behavior Estimate of Prediction of Past Behavior Future Behavior Combined Judgment about self ~ 00 ~ 3 ~ ~ ~ 5 Judgment about roommate 1 1 1 124 117 Combined ~ 05 ~ 28 Note: All intervals were transformed to reflect magnitude relative to interval for past behavior of self.
From page 68...
... e., that the accuracy of any inferences we might want to make about that behavior would be facilitated by procedures that facilitated recall or encouraged more accurate estimation strategies on the part of the respondent. Finally, the data comparing self-est imates and roommate estimates are interesting in their own right, beyond any relevance to concerns of survey methodology.
From page 69...
... At the annual meeting of the American Statistical Association held on August ~ 3- ~ 6, ~ 984, in Philadelphia, Pennsylvania, a session on cognitive aspects of survey methodology was sponsored by the Section on Survey Research Methods and cosponsored by the Social Statistics and Statistical Education Sect ions . Organized and chaired by Judith Tanur, the session included a paper by Roger Tourangeau entitled Interchanges between cognitive science and survey methodology and a paper authored by David C
From page 70...
... In response to a preliminary proposal presented at its board meeting in June 1984, the Social Science Research Council (SSRC) is organizing a working group on cognition and survey research in order to prepare a detailed plan for the activities of a possible SSRC committee that would bear thesame name.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.