5

Issues and Innovations in Population Data Collection and Measurement: Survey Research

SURVEYS OF FORCED MIGRANTS

Jon Pedersen (Fafo) explained that large-scale surveys that try to cover the whole population can be divided into two categories: (1) surveys in the areas of emigration that use indirect data from information provided by nonmigrants, and (2) surveys in the area of immigration that use data gathered directly from migrants. Regardless of the category, such surveys are typically interested in the numbers of migrants and the characteristics of migrants. However, these are different kinds of results and can call for different kinds of surveys, Pedersen noted.

Surveys in areas from which people move typically take two different approaches. One is to use information gathered directly from households. For example, a survey might ask about “relatives abroad,” thereby producing indirect migration estimates. However, such an approach can suffer from selection bias and limit the amount of information that can be gathered. For example, they can double-count immigrants when two different households report on the same relative who has left. Various methods can correct for these problems, Pedersen said, but they have not had great success.

Another approach is to analyze aggregate patterns to derive numbers and characteristics. Very large migration numbers are typically needed to separate the signal from the noise, Pedersen said, but such surveys can reveal how populations are changing. As an example, Pedersen cited surveys to monitor the development of the Eritrean population from 1995 to 2015. They show that people, especially males, started to move away between the ages of 15 to 19. The number of older men exceeded the number of older women, most likely because older women have an easier time of migrating for family reunification, for health reasons, and to help migrant families with children. The data indicate that the population in Eritrea is getting less educated and child mortality is going up, probably because the overall population is getting poorer as a consequence of outmigration. This type of analysis worked in the case of Eritrea, according to Pedersen, because of the size of the migration. In other cases, such as the migration of Iraqis, it does not work, because the migration is not large enough to separate signals from the noise in the data.

Pedersen noted one disadvantage of surveys in areas of immigration is that they typically are done only in a specific country. A survey of Syrian refugees in Jordan cannot reach Syrian refugees in Turkey, Lebanon, or other countries, yet generalizations often are made about migrants surveyed beyond the area studied. Other problems he cited include that the sampling usually entails covering a population that is a small proportion of the host population, irregularly distributed, spatially clustered, often living in nonstandard dwellings, often marked by large inequalities, living in ways that may make them inaccessible for interviews, and are unwilling or afraid to be interviewed. Finally, the sampling may take place in one area of a host country while sampling in another area would produce different results. Under such circumstances, said Pedersen, traditional sampling does not work well.

To counter these difficulties, researchers may turn to probability samples. Different types of samples have both benefits and drawbacks (see Table 5-1). For example, large samples quickly become very expensive, and oversampling of target population strata can produce variance inflation. Respondent-driven sampling has been popular, especially for health purposes, but Pedersen said it is not really known whether this type of

TABLE 5-1 Benefits and Drawbacks of Probability Samples of Forced Migrants

SOURCE: Workshop presentation by Jon Pedersen, May 21, 2019.

sampling works; it is also difficult to apply for a large population distributed geographically.

Pedersen used the case of Jordanian immigration to illuminate many of these issues. Since 1948, Palestinians, Iraqis, and Syrians have migrated in large numbers into Jordan. The country also has a large number of workers from other countries. According to the 2015 census, Jordan has 2.9 million foreigners out of a total population of 9.5 million, representing 31 percent of the population. Jordan is not a party to the 1951 Refugee Convention (see Chapter 2) and prefers terms such as “Syrians in Jordan” or “Iraqis in Jordan.” Pedersen also noted that Iraqis and Syrians are perceived by many Jordanians as a burden.

In 2007 and 2017, Pedersen and his colleagues carried out surveys of Iraqis and Syrians in Jordan, respectively. The sampling assumed that a 2004 census conducted in Jordan was informative: that is, that the enumeration areas with Iraqis in 2004 probably also had Iraqis in 2007. Sampling then was allocated optimally with respect to the number of Iraqis. In Amman, an adaptive stratified sample was adopted to take account of the 2004 census data without totally committing to it. Outside Amman, the adaptive cluster sample assumed that, because of the lower density of Iraqis, it was more likely that they would have moved, even though maps suggested clustering. In Amman, 600 enumeration areas were used, while 400 enumeration areas were used outside Amman.

When a cluster was selected, Iraqi households within it were selected, and all or 12 Iraqi households, whichever was smaller, were interviewed. Two other households were also interviewed to check for incorrect listings. This approach to sampling turned out to work quite well, said Pedersen, though it led to a public relations dispute with the Jordanians because of the lower-than-expected number of refugees it revealed—133,000, with a 95 percent confidence level between 110,000 and 156,000. He also said that interviewing the two extra households did not work well because the variance produced was too large to provide useful information. Pedersen said that it would have been better to spend more time on verifying the rest of the sample in order to understand what was going on.

With the 2015 census, Pedersen and his colleagues assumed that the enumeration area count was related to the present numbers of Syrians living in the country. The sample consisted of three parts. The first included the camps, which were unproblematic and relatively simple to sample. The second consisted of enumeration areas with Syrians in 2015, which were sampled for the number of Syrian households. The third was a small sample of enumeration areas without Syrians in 2015 to pick up Syrians who had moved out of previously identified clusters. This approach found considerably fewer Syrians than in the census—only 31 percent as many. Instead of 1.2 million, said Pedersen, they estimated approximately 350,000 Syrians living in Jordan.

Pedersen drew several conclusions from this work. Neither Iraqis nor Syrians in Jordan are rare or difficult to find. However, samples tended to yield less of the target group than expected. In the Iraqi case, this was because there were considerably fewer Iraqis than expected. In the Syrian case, the census may have overstated the numbers or many may have moved out of the country.

Despite shortfalls, the census data were useful in designing the sample, Pedersen said. But sample designs must be sufficiently robust to allow for surprises. In particular, the sample must be large enough to handle a situation in which the population is differently distributed or smaller than expected. This is difficult to do in practice, according to Pedersen, because a sample size often reflects the budget for the survey rather than what is necessary for statistical purposes.

In response to a question about the quality of Jordanian censuses compared with survey results, Pedersen noted that the results are not always consistent. The use of tablets for data entry may have caused problems with the 2015 census, he conjectured, but an evaluation would need to be carried out. He also briefly discussed some of the problems with network-based methods, which rely on social-psychological premises that are not known to be accurate. For example, when migrants are spread throughout a country, a sampling frame is difficult to design to represent people between sampling places. Similarly, if migrants have to come to a place to be interviewed, the necessary centralization requires techniques and assumptions that may not hold. As an example, adaptive sampling techniques work fine in simulations, although whether they work in practice is difficult to say

THE MEXICAN AND LATIN AMERICAN MIGRATION PROJECTS

Karen Pren (Princeton University), current director of the Mexican Migration Project (MMP), explained that the MMP began in 1982 with the interviewing of people in four communities in western Mexico, which resulted in the book Return to Aztlan (Massey et al., 1990). Douglas Massey and Jorge Durand then decided to survey more places systematically using what they termed an “ethnosurvey.” Since 1987, the MMP has conducted interviews in 170 communities throughout Mexico, interviewing more than 27,000 households in Mexico and more than 1,000 households in the United States, representing more than 176,000 people and 8,000 migrants. It also has constructed a longitudinal life file with more than 1 million observations and more than 800,000 spouse observations.

The Latin American Migration Project (LAMP) started in 1998 to extend the MMP’s research to migration flows originating in other Latin American countries. LAMP has carried out research in Guatemala, El Salvador, Nicaragua, and Costa Rica in Central America; in Haiti, the

Dominican Republic, and Puerto Rico in the Caribbean; and in Colombia, Ecuador, Peru, Paraguay, and Uruguay in South America. In Uruguay, surveys of Venezuelan, Cuban, Dominican Republic, and Peruvian immigrants are taking place with the same methodology as in other countries.

Though the methodology needs to be adjusted to the cultural reality of each new country, it is largely the same as in the 1980s (Massey, 1987). It is a multimethod data collection that mixes qualitative and quantitative survey information (see Figure 5-1). The qualitative surveys are relatively unstructured and rely heavily on conversations with participants. The quantitative surveys are more like a census with formal and standardized choices.

The precise timing of the questions and the way they are asked depend on the circumstances, with much depending on interviewer judgment. For example, when talking with undocumented migrants in the United States, the interviewer has to understand the situation. The goal is to make the surveys informal and nonthreatening while still producing a standard set of data.

Communities are selected on the basis of observation and personal reconnaissance of the geographic area. Migration indices and sex ratios are also taken into account, as is information from other researchers who know the area. As a final check, said Pren, the project codirector visits the location and ensures the safety of the interviewers.

The project usually targets places with middle indices of migration (i.e., neither high nor low migration). It includes communities from a variety of regions with a wide range of population sizes, ethnic compositions, and economic bases. Communities are selected from metropolitan areas, midsized cities, towns, and rural villages, and sites are chosen to enhance the strength of inference in both the qualitative and quantitative analyses.

Applying the ethnosurvey consists of three steps, followed by the compilation of data from multiple levels (see Figure 5-2). Although individuals are the ultimate unit of behavior, decisions are always made within larger social and economic contexts. The ethnosurvey is explicitly designed to provide quantitative data for multilevel analysis by gathering data at both the individual and household levels. Detailed community-level data are compiled and applied to interpret the socioeconomic context within which individuals and households interact.

The ethnosurvey is designed to compile event histories on specific aspects of social and economic life, including employment, migration, marriage, fertility, and property ownership. The project also conducts parallel sampling in the home country and in receiving countries. For example, while migrants from Central America go mainly to the United States, migrants from South America tend to go to many locations.

Pren said the application of this methodology has yielded a number of important contributions (Massey et al., 2002), making it possible to do the following:

• Compare the characteristics and behavior of documented and undocumented migrants.
• Measure trends in the characteristics of both groups of migrants over time.
• Support longitudinal analysis of the migration process.
• Discern the background and characteristics of migrants before and after they enter the United States.
• Undertake detailed cross-tabulations of Mexicans based on large samples.
• Study transitions between different legal statuses and model selective movements back and forth across the border.
• Provide an ongoing source of longitudinal data capable of monitoring the effects of shifting U.S. and Mexican policies.

Though it is not focused specifically on forced migration, Pren noted, the methodology has worked for many years, and the same methodology is now being applied in other countries, including Germany, Poland, China, and Bangladesh.¹

During the general discussion, Pren indicated that the MMP and LAMP work through personal contacts, including friends, acquaintances, and family, to trace where people go. They make sure the people contacted know that their names will not be used and that they are “just going to be another number to the survey,” she said. Even the place of birth of respondents is concealed. In addition, some respondents are not concerned about providing information if they have the proper documentation. Tracking people down has become more complex as migrants have gone to more destinations and not just to large cities such as Los Angeles or Chicago, Pren said.

DISCUSSION ON SURVEY RESEARCH

Sarah Staveteig (U.S. Department of State)² began the general discussion with some comments on the state of the field of survey research on forced migration. The interdisciplinary nature of immigration research is both a strength and a source of complexity, she said. Survey research on forced migration clearly benefits from rigorous methods from demography, sociology,

___________________

¹ For more information about the results of the MMP, see https://mmp.opr.princeton.edu.

² Staveteig clarified that her opinions were her own and did not necessarily represent those of the U.S. Department of State.

anthropology, as well as the field of survey research itself. Civil and vital registration systems discussed in the previous session are crucial, Staveteig said, in terms of providing numbers and characteristics of migrants; emerging “big data” sources on migration can also complement these approaches. However, even if civil and vital registration systems are strengthened, it remains highly likely that surveys will be needed in order to uncover the economic and social drivers of migrants, along with uncovering individual narratives and in-depth characteristics. She said that it is therefore imperative to conduct both vital registrations and survey research, as they complement each other.

Both types of data also highlight the importance of conceptualizations and the operationalization of those conceptualizations, she added; it is important as a scholarly community to clearly operationalize concepts and think closely about how best to measure them. Adapting traditional probabilistic survey methods to uncover forced migrants in a larger population expands the bounds of what surveys can do. Similarly, the complexities of achieving coverage, comparing sites, providing information about both the sending and receiving contexts, and combining levels of information attest to both the difficulties and the potential of this survey research on forced migration.

Part of the rationale for the workshop was to explore research that is needed in the 21st century, Staveteig said. Current challenges for survey research on forced migration include the dynamicity of the flows, multisite sources of data, the need for rapid assessments, and the simultaneous need for careful, well-designed studies. New technologies such as geolocations can help meet these challenges while also raising issues of their own, such as privacy concerns. Survey fieldwork in sending and even in receiving countries also faces enormous challenges to interviewer safety. Staveteig added that data collection is heavily intertwined with analysis, and what social scientists can feasibly study and how they choose to study it is closely tied to what they can produce.

In answer to a question from a participant, Jon Pedersen addressed how to secure cooperation without being coercive. He said surveys in middle- and low-income countries have not produced major problems with cooperation, although it is because, unfortunately, the power relationship between the interviewer and the respondent is to the benefit of the interviewer. When a high-status interviewer is interviewing a low-status person, the low-status person can have trouble resisting, he said. As a result, his team has had high response rates in places such as Jordan. In contrast, when interviewing Norwegians in Norway, response rates are much lower because a student or other perceived low-status person is typically interviewing a high-status person who may not cooperate. For vulnerable people in Norway, such as illegal migrants and street beggars, response rates are much higher than the 50 percent rate typical of Norwegians and people in other European countries, for the same reasons. Pedersen indicated that in the specific case of Palestinian refugees, Fafo has a relatively good track record of interview response rates because the refugees

respect Fafo’s participation in the Oslo peace process (even though many did not like it) and know that Fafo is not trying to cause them harm.

Holly E. Reed noted that in recent surveys of undocumented migrants at City University of New York (CUNY), migrants were eager rather than reluctant to tell their stories. The students who came forward did so because they wanted information about programs available to them, and they felt isolated and were not part of the Dreamers or organized groups on campus. Reed suggested that this represented an opportunity for forced migration research in general, as informational incentives could also encourage people to participate in research.

Several participants discussed the possibility of adding a small number of questions to demographic and health surveys that could capture information on migration—for example, “Where did you live 5 years ago?” Katherine Donato (Georgetown University) said the group that conducts the Latin American Public Opinion Survey has discussed such a possibility, but in the end, the group had the resources to include only one question, which was less informative than might be hoped. However, at least some work is being done along these lines, she said.

Marwan Khawaja noted that a survey³ supported by the European Commission and conducted in several countries includes questions on forced migration, but he added that relatively little work has been done on the methodology of such questions, raising questions about the quality of the resulting data compared with, for example, estimates of fertility and mortality.

Pren noted that MMP and LAMP questionnaires have begun to ask relatively simple questions about health, such as whether respondents have diabetes or hypertension. Initially these questions did not indicate whether a health condition originated before, during, or after migration, so a question was added about the year of diagnosis. The interviewer has to assess the quality of the information—for example, whether the memory of an older head of a household can be trusted. But the questions and corresponding answers also connect to each other, which allows adjustments to be made after the interview. The same day that interviewers gather data, they need to review the data and make corrections before giving the data to a supervisor for approval. Subsequently, the person who is doing the data entry, who has extensive experience with the project, makes a judgment about whether the data have problems and can direct questions back to the supervisor. The data are reviewed again when sent to Princeton University and compared with other data sources to detect any problems.

During the wrap-up session at the end of the first day of the workshop, planning committee cochair Ellen Percy Kraly provided a recap of some of the key emergent issues in forced migration research (see Box 5-1).

___________________

³ MED-HIMS (Households International Migration Surveys in the Mediterranean countries).