Applications of an Analytic Framework on Using Public Opinion Data for Solving Intelligence Problems Proceedings of a Workshop (2022) / Chapter Skim
Currently Skimming:
1 Public Opinion Data and the Analytic Framework
1 Public Opinion Data and the Analytic Framework
Pages 5-20
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 5... ...
interests. To fulfill this mandate, she continued, it is imperative that intelligence analysts keep pace with current trends in public opinion research methods.
|
From page 6... ...
However, as public opinion research becomes exponentially more sophisticated in the United States, the quality of o pinion research abroad varies widely and often lags behind that of the United States, creating challenges for drawing on the findings from international survey research to inform U.S. foreign policy.
|
From page 7... ...
, which was generated with the contributions of an a dvisory panel of 12 experts in survey and social science research methods; 5 subjectmatter experts who authored four commissioned papers; a technical writer who wrote the synthesis; National Academies of Sciences, Engineering, and Medicine staff; and liaisons from the IC. Because public opinion data are a critical input into intelligence analysts' assessments of attitudes in foreign populations, she continued, the following question guided the work of the expert contributors to and authors of the Analytic Framework: how can the IC analyze, make inferences from, and present public opinion data in ways that align with best practices, while acknowledging that these data are not always perfect or complete?
|
From page 8... ...
outlined the structure of the Analytic Framework, which consists of a foundational layer, a synthesis layer, and a graphic layer. Foundational Layer Zechmeister noted that the foundational layer is targeted toward intelligence analysts with prior training and experience in survey methodology.
|
From page 9... ...
designed data, which emerge from a systematic attempt to address a particular question, versus organic data, which emerge for another purpose but might still provide insight into public opinion; and (2) primary data (i.e., collected by the researcher)
|
From page 10... ...
. The paper emphasizes that before integrating data, intelligence analysts should understand the nature of the data and harmonize those data to confirm that integration and comparison are possible, using processes such as data cleaning, linking sample data with the population, addressing data missingness, generating a composite measure, and validating.
|
From page 11... ...
Zechmeister described this layer as an "orientation to the fundamental ideas and best practices of public opinion research." Graphic Layer Zechmeister explained that the top layer of the Analytic Framework is a graphic that presents a decision tree with four iterative phases, which could be useful to guide the work on a time-sensitive task of an intelligence analyst who has either attended this workshop or has read the synthesis layer. During Phase 0, the analyst generates or is assigned a research question.
|
From page 12... ...
Convey credibility PHASE 3 CONSIDER: CONFIRM: • Sensitive topics • Data granularity • Sample • Target population • Time • Consistent measurement • Questionnaire, question wording • Translation FIGURE 1-1 The Analytic Framework's graphic representation of a decision tree for public opinion data collection and analysis.
|
From page 13... ...
ETHICAL AND CULTURAL CONSIDERATIONS IN THE COLLECTION AND USE OF PUBLIC OPINION DATA Ethical Considerations Zachariah Mampilly (expert contributor for the Analytic Framework and Marxe Endowed Chair of International Affairs at the Marxe School
|
From page 14... ...
Various professional associations (e.g., the American Association for Public Opinion Research [AAPOR] and the American Political Science Association)
|
From page 15... ...
She suggested supporting individuals in their thoughts about and challenges in doing ethical work as well as raising awareness of context and the integration of that context into the work. Cultural Considerations Michele Gelfand (expert contributor for the Analytic Framework; John H
|
From page 16... ...
, or the interview could lack standardization. To avoid the influence of cultural ki's, she suggested developing a structured interview protocol, reviewing the content and structure with local collaborators and being prepared to make changes as needed, using local interviewers with characteristics similar to those of the participants, conducting a pre-interview/focus group to detect problems, and standardizing interviewers.
|
From page 17... ...
Because cultural assumptions can affect choices made in the models, however, she reiterated the importance of working with local collaborators. Overarching concerns about analyzing data relate to cultural response sets, structural equivalence, and aggregation issues.
|
From page 18... ...
He also urged the IC to foster a "culture of ethics." He described the immense pressure on junior analysts who want to make both actionable and ethical recommendations -- ethics should not be compromised to make actionable recommendations, but a culture shift is required to embrace this mindset. An intelligence analyst explained that experienced intelligence analysts who have completed graduate studies have been introduced to these ethical standards, but new analysts have not had the opportunity to think deeply about ethical considerations and would benefit from additional guidance.
|
From page 19... ...
She reiterated that every analyst who engages with data from humans about humans has an individual responsibility, equivalent to that of anyone else in the hierarchy; in other words, ethical responsibility extends beyond ethical practice in data collection to ethical use of the data. James Druckman (expert contributor for the Analytic Framework and Payson S
|
From page 20... ...
She emphasized that this shift requires both training and thought experiments with feedback as well as local collaboration to help intelligence analysts make the most ethical decisions.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.