Many racial and ethnic groups intheUnited States, including blacks, Hispanics, Asians, American Indians, and others, have historically faced severe discrimination—pervasive and open denial of civil, social, political, educational, and economic opportunities. Today, large differences in outcomes among racial and ethnic groups continue to exist in employment, income and wealth, housing, education, criminal justice, health, and other areas. Although many factors may contribute to such differences, their size and extent suggest that various forms of discriminatory treatment persist in U.S. society and serve to undercut the achievement of equal opportunity.
In these circumstances, it is critically important to identify where racial discrimination occurs and to measure the extent to which discrimination may contribute to racial and ethnic disparities. The Committee on National Statistics convened a panel of scholars to consider the definition of racial discrimination, assess current methodologies for measuring it, identify new approaches, and make recommendations about the best broad methodological approaches. Specifically, this panel was asked to carry out the following tasks:
Give the policy and scholarly communities new tools for assessing the extent to which discrimination continues to undermine the achievement of equal opportunity by suggesting additional means for measuring discrimination that can be applied not only to the racial question but in other important social arenas as well.
Conduct a thorough evaluation of current methodologies for measuring discrimination in a wide range of circumstances where it may occur.
Consider how analyses of data from other sources could contribute to findings from research experimentation, such as the U.S. Department of Housing and Urban Development paired tests.
Recommend further research as well as the development of data to complement research studies.
There is no single concept of race. Rather, race is a complex concept, best viewed for social science purposes as a subjective social construct based on observed or ascribed characteristics that have acquired socially significant meaning. In the United States, ways in which different populations think about their own and others’ racial status have changed over time in response to changing patterns of immigration, changing social and economic situations, and changing societal norms and government policies. In the late nineteenth and early twentieth centuries, for example, some European Americans, such as Italians and Eastern European Jews, were regarded as distinct racial groups. Although these distinctions are no longer sanctioned by the U.S. government, some segments of the population may still act in ways that are consistent with such distinctions. For certain populations and in some situations, race may be difficult to define consistently; for example, many Hispanics consider themselves to be part of a distinct racial group, but many others hold no such perception. Because concepts of race and ethnicity are not clearly defined for many Hispanics and because of the discrimination they have faced, we include Hispanics, along with specific racial groups, in our discussion of racial discrimination.
The ambiguity involved in defining race has implications for how data on race are collected. The official federal government standards for data on race and ethnicity currently identify five major racial groups (black or African American, American Indian or Alaska Native, Asian, Native Hawaiian or other Pacific Islander, and white) and one ethnic group (Hispanic) that may be of any race. These categories are used by federal program and statistical agencies to collect data through self-reports (preferably) or by assigning individuals to one or more categories. The federal racial categories have changed over time, in part reflecting the changing conception of race in the United States. The government standards are not always consistent with scholarly concepts of race or with concepts held by individuals and groups; as a result, it may be difficult to obtain data on race and ethnicity that are comparable over time or across different surveys and administrative records. Comparability may also be affected by differences in the data collection
methods used. Yet given the salience of race in so many aspects of social, political, and economic life, it is important to continue collecting these data.
Conclusion: For the purpose of understanding and measuring racial discrimination, race should be viewed as a social construct that evolves over time. Despite measurement problems, data on race and ethnicity are necessary for monitoring and understanding evolving differences and trends in outcomes among groups in the U.S. population. (from Chapters 2 and 10)
Recommendation: The federal government and, as appropriate, state and local governments should continue to collect data on race and ethnicity. Federal standards for racial categories should be responsive to changing concepts of race among groups in the U.S. population. Any resulting modifications to the standards should be implemented in ways that facilitate comparisons over time to the extent possible. (Recommendation 10.1)1
Recommendation: Data collectors, researchers, and others should be cognizant of the effects of measurement methods on reporting of race and ethnicity, which may affect the comparability of data for analysis:
To facilitate understanding of reporting effects and to develop good measurement practices for data on race, federal agencies should seek ways to test the effects of such factors as data collection mode (e.g., telephone, personal interview), location (e.g., home, workplace), respondent (e.g., self, parent, employer, teacher), and question wording and ordering. Agencies should also collect and analyze longitudinal data to measure how reported perceptions of racial identification change over time for different groups (e.g., Hispanics and those of mixed race).
Because measurement of race can vary with the method used, reports on race should to the extent practical use multiple measurement methods and assess the variation in results across the methods. (Recommendation 10.2)
For ease of reference, the panel’s recommendations are numbered according to the chapter of the report in which they appear. For example, Recommendation 10.1 is the first recommendation presented in Chapter 10.
DEFINING RACIAL DISCRIMINATION
This report adopts a social science definition of racial discrimination that has two components:
differential treatment on the basis of race that disadvantages a racial group and
treatment on the basis of inadequately justified factors other than race that disadvantages a racial group (differential effect).
In this report, we focus on discrimination against disadvantaged racial minorities. The two components of our definition—differential treatment and differential effect discrimination—are related to but broader than the standards embodied in case law in the U.S. legal system, which are disparate treatment and disparate impact discrimination. An example of potentially unlawful disparate treatment discrimination would be when an individual is not hired for a job because of his or her race. An example of potentially unlawful disparate impact discrimination would be when an employer uses a test in selecting job applicants that is not a good predictor of performance on the job and results in proportionately fewer job offers being extended to members of disadvantaged racial groups compared with whites.2
Because our intention in this report is to provide guidance to social science researchers interested in measuring discrimination, both components of our definition include a range of behaviors and processes that are not explicitly unlawful or easily measured. For example, many governmental actions that might fall within the legal definition of disparate impact discrimination would not be unlawful because the Supreme Court has interpreted the constitutional prohibition on denials of equal protection by government agencies to bar only cases of intentional discrimination—that is, disparate treatment discrimination. As a second example, discrimination would occur under our definition when interviewers of job applicants more frequently adopt behaviors (e.g., interrupting, asking fewer questions, using a hectoring tone) that result in poorer communication with and performance by disadvantaged minority applicants compared with other applicants. Even if such behaviors became the subject of a legal challenge, the difficulties in measurement and proof would likely mean that such behav-
iors would not be effectively constrained by law. Measuring them is important, however, to understand ways in which subtle forms of discrimination may affect important social and economic outcomes.
MEASURING RACIAL DISCRIMINATION
That racial disparities exist in a wide range of social and economic outcomes is not in question: They can be seen in higher rates of poverty, unemployment, and residential segregation and in lower levels of education and wealth accumulation for some racial groups compared with others. Large and persistent outcome differences, however, do not themselves provide direct evidence of the presence or magnitude of racial discrimination in any particular domain. Differential outcomes may indicate that discrimination is occurring, that the historical effects of racial exclusion and discrimination (cumulative disadvantage) continue to influence current outcomes, that other factors are at work, or that some combination of current and past discrimination and other factors is operating.
The panel evaluated four major methods used across different social and behavioral science disciplines to measure racial discrimination: laboratory experiments, field experiments, analysis of observational data and natural experiments, and analysis of survey and administrative record reports. Each method has strengths and weaknesses, particularly for drawing a causal inference that an adverse outcome is the result of race-based discriminatory behavior.
Because discriminatory behavior is rarely observed directly, researchers must infer its presence by trying to determine whether an observed adverse outcome for an individual would have been different had the individual been of a different race. In other words, researchers attempt to answer the following counterfactual question: What would have happened to a nonwhite individual if he or she had been white? Understanding the extent to which any study succeeds in answering that question requires rigorously assessing the logic and assumptions underlying the causal inferences drawn by the researchers. As was true in determining that smoking causes lung cancer, using a variety of methods implemented in a variety of settings is likely to be most helpful in measuring discrimination.
Conclusion: No single approach to measuring racial discrimination allows researchers to address all the important measurement issues or to answer all the questions of interest. Consistent patterns of results across studies and different approaches tend to provide the strongest argument. Public and private agencies—including the National Science Foundation, the National Institutes of Health, and private founda-
tions—and the research community should embrace a multidisciplinary, multimethod approach to the measurement of racial discrimination and seek improvements in all major methods employed. (from Chapter 5)
Classically, laboratory experimentation in which a stimulus can be administered to research participants in a controlled environment and in which participants can be randomly assigned to an experimental condition or another (e.g., control) condition provides the best approach for inferring causation between a stimulus and a response. Such experiments come closest to addressing the above counterfactual question.
Laboratory experiments have uncovered many subtle yet powerful psychological mechanisms through which racial bias exists. Yet regardless of how well designed and executed they are, laboratory experiments cannot by themselves directly address how much race-based discrimination against disadvantaged groups contributes to adverse outcomes for those groups in society at large.
The major contributions of laboratory experiments are to identify those situations in which discriminatory attitudes and behaviors are more or less likely to occur, as well as the characteristics of people who are more or less likely to exhibit discriminatory attitudes and behaviors, and to provide models of people’s mental processes that may lead to racial discrimination. Such experiments can usefully suggest hypotheses to be tested with other methodologies and real-world data.
Recommendation: To enhance the contribution of laboratory experiments to measuring racial discrimination, public and private funding agencies and researchers should give priority to the following:
Laboratory experiments that examine not only racially discriminatory attitudes but also discriminatory behavior. The results of such experiments could provide the theoretical basis for more accurate and complete statistical models of racial discrimination fit to observational data.
Studies designed to test whether the results of laboratory experiments can be replicated in real-word settings with real-world data. Such studies can help establish the general applicability of laboratory findings. (Recommendation 6.1)
Large-scale experiments in the field rely on random assignment of subjects to one or more experimental treatments or to no treatment, so that researchers can determine whether an experimental treatment (the stimulus) causes an observed response. Such experiments take longer and are more complex to manage and more costly to conduct than laboratory experiments, and their results are more easily confounded by factors in the environment that the researchers cannot control. However, their results are more readily generalizable to the population at large.
The most significant use of field studies to study discrimination to date has been in the area of housing, specifically seeking new apartments or houses. The results of audit or paired-testing studies—in which otherwise comparable pairs of, say, a black person and a white person are sent separately to realty offices to seek an apartment or house—have been used to measure discrimination in specific housing markets. Audit studies have also been conducted on job seeking. It is likely that audit studies of racial discrimination in other domains (e.g., schooling and health care) could produce useful results as well, even though their use will undoubtedly present methodological challenges specific to each domain.
Recommendation: Nationwide field audit studies of racially based housing discrimination, such as those implemented by the U.S. Department of Housing and Urban Development in 1977, 1989, and 2000, provide valuable data and should be continued. (Recommendation 6.2)
Recommendation: Because properly designed and executed field audit studies can provide an important and useful means of measuring discrimination in various domains, public and private funding agencies should explore appropriately designed experiments for this purpose. (Recommendation 6.3)
Statistical Analysis of Observational Data and Natural Experiments
Observational studies are currently the primary tool through which researchers explore issues of racial disparity and discrimination in the real world. The standard way to explore the difference in an outcome between racial groups is to develop a regression model that includes a variable for race and variables for other relevant observed characteristics. The effect of the former variable on the outcome difference is identified as discrimination.
To support a causal inference from observational data, however, substantial prior knowledge about the mechanisms that generated the data must be available to justify the necessary assumptions. There are two particularly common problems involved in using standard multiple regression models to analyze observational data on outcome differences between race groups: Omitted variables bias occurs whenever a data set contains only a limited number of the characteristics that may reasonably factor into the process under study; sample selection bias occurs when the research systematically excludes subjects from the sample whose characteristics vary from those of the individuals represented in the data. Should either bias be present, it is difficult to draw causal inferences from the coefficient on race (or any other variable) in a regression model, as the race coefficient may overestimate or underestimate the effect labeled as discrimination.
Nationally representative data sets containing rich measures of the variables that are the most important determinants of such outcomes as education, labor market success, and health status can help in estimating and understanding the sources of racial differences in outcomes. Panel data, which include observations over time, are particularly valuable in this regard. There is also an important role for focused studies that target particular settings (e.g., a firm or a school), whereby it is possible to learn a great deal about how decisions are made and to collect most of the information on which decisions are based.
Evaluations of natural experiments are another way to exploit observational data in the measurement of racial discrimination. Such evaluations analyze data before and after enactment of a new law or some other change that forces a reduction in or the complete elimination of discrimination for some groups. Despite limitations, natural experiments provide useful data for measuring the extent of discrimination prior to a policy change and for groups not affected by the change.
Conclusion: The statistical decomposition of racial gaps in social outcomes using multivariate regression and related techniques is a valuable tool for understanding the sources of racial differences. However, such decompositions using data sets with limited numbers of explanatory variables, such as the Current Population Survey or the decennial census, do not accurately measure the portion of those differences that is due to current discrimination. Matching and related techniques provide a useful alternative to race gap decompositions based on multivariate regression in some circumstances. (from Chapter 7)
Conclusion: The use of statistical models, such as multiple regressions, to draw valid inferences about discriminatory behavior requires appropriate data and methods, coupled with a sufficient understanding
of the process being studied to justify the necessary assumptions. (from Chapter 7)
Recommendation: Public and private funding agencies should support focused studies of decision processes, such as the behavior of firms in hiring, training, and promoting employees. The results of such studies can guide the development of improved models and data for statistical analysis of differential outcomes for racial and ethnic groups in employment and other areas. (Recommendation 7.1)
Recommendation: Public agencies should assist in the evaluation of natural experiments by collecting data that can be used to evaluate the effect of antidiscrimination policy changes on groups covered by the changes as well as groups not covered. (Recommendation 7.2)
Indicators of Discrimination from Surveys and Administrative Records
Both self-reports of racial attitudes and perceived experiences of discrimination in surveys and reports of discriminatory events in administrative records can contribute to understanding the extent of racial discrimination. Survey data typically cannot directly measure the prevalence of actual discrimination as opposed to reports of perceived discrimination, but they can provide useful supporting evidence. Perceived discrimination may overreport or underreport discrimination assessed by other methods. As expressions of prejudice and discriminatory behavior change over time and become more subtle, new or revised survey questions on racial attitudes and perceived experiences of discrimination may be necessary. Longitudinal and repeated cross-sectional data, including continuous and new measures, are important to illuminate trends and changes in patterns of racially discriminatory attitudes and behaviors among and toward various groups. Such data are also vital for studies of cumulative disadvantage. Administrative reports of discrimination (e.g., equal employment opportunity complaints) may also be useful for research, although the lack of completeness and reliability of such reports can limit their usefulness.
Recommendation: To understand changes in racial attitudes and reported perceptions of discrimination over time, public and private funding agencies should continue to support the collection of rich survey data:
The General Social Survey, which since 1972 has been the leading source of repeated cross-sectional data on trends in racial attitudes and perceptions of racial discrimination, merits continued support
for measurement of important dimensions of discrimination over time and among population groups.
Major longitudinal surveys, such as the Panel Study of Income Dynamics, the National Longitudinal Survey of Youth, and others, merit support as data sources for studies of cumulative disadvantage across time, domains, generations, and population groups. To further enhance their usefulness, questions on perceived experiences of racial discrimination and racial attitudes should be added to these surveys.
Data collection sponsors should support research on question wording and survey design that can lead to improvements in survey-based measures relating to perceived experiences of racial discrimination. (Recommendation 8.1)
Recommendation: Agencies that collect administrative record reports of racial discrimination should seek ways to allow researchers to use these data for analyzing discrimination where appropriate. They should also identify ways to improve the completeness, reliability, and usefulness of reports of particular types of discriminatory events for both administrative and research purposes. (Recommendation 8.2)
Racial Profiling as an Illustrative Example
To provide a specific example of an area for which research on discriminatory treatment is needed but difficult to carry out, we discuss methodological issues in profiling. Racial or ethnic profiling is a screening process in which some individuals in a population (e.g., automobile drivers or people boarding an airplane) are selected on the basis of their race or ethnicity (and, typically, other observable characteristics) and investigated to determine whether they have committed or intend to commit a criminal act (e.g., smuggle drugs or blow up an airplane) or other act of interest. This definition excludes cases of identified individuals for whom race or ethnicity is part of their individual description. Many recent public statements (e.g., those made by police officials and legislative bodies since 2001) have recognized the unacceptability of racial profiling in police work. Even when such profiling is not explicitly racial, to the extent that it relies on characteristics that are distributed differently for different racial groups, the result may be a racially disparate impact.
Inferring the presence of discriminatory racial profiling from data on disparate outcomes is difficult for the same reasons that it is difficult to infer causation from any statistical model with observational data. We ex-
plore specific methodological concerns for improving the estimation of outcome rates (e.g., traffic stops for whites and minorities) and developing good statistical models for determining the contribution of discriminatory profiling as compared with other factors to differences in rates. Because of renewed interest in the United States in the use of profiling to identify and apprehend potential terrorists before they commit violent acts, we also examine briefly the challenges of identifying screening factors that could potentially select would-be terrorists with a significantly higher probability than purely random selection, as well as issues that must factor into the public debate if race or ethnicity (or factors that correlate highly with race or ethnicity) are considered as potential screening factors.
Much of the discussion about the presence of racial discrimination and the effects of antidiscrimination policies assumes discrimination to be a phenomenon that occurs at one point in time in a particular process or stage of a particular domain (e.g., initial hires by employers). This episodic view of discrimination is likely inadequate. Discrimination may well have cumulative effects, and it is therefore better viewed as a dynamic process that functions throughout the stages within a domain, across domains, across individual lifetimes, and even across generations. For example, discrimination involving teachers’ expectations during schooling may affect students’ later educational experiences or job opportunities; likewise, discrimination against prior generations may diminish opportunities for present generations even in the absence of current discriminatory practices.
Several theories of the processes by which discrimination may have cumulative effects have been developed, including (1) life-course theory of cumulative disadvantage in criminal justice research, which posits that such behavior as juvenile delinquency can affect certain social outcomes, such as failure in school or poor job stability, and thereby facilitate criminal behavior as an adult; (2) ecosocial theory in public health research (similar to the life-course concept), in which health status at a given age for a given birth cohort reflects not only current conditions but also prior living circumstances from conception onward; and (3) feedback models in labor market research. In such a model, for example, people who anticipate lower future returns to skills—possibly as a result of racial discrimination—might invest less in acquiring those skills. In turn, lower investment could perpetuate prejudice, limit opportunities, and sustain racial disparities in the labor market.
Only very limited research has been conducted, however, to test empirically the various theories of cumulative disadvantage and to measure the importance of cumulative effects over time and across domains. Longitudi-
nal data are a necessity for such research, as are methods for credibly identifying initial and subsequent incidents of discrimination.
Conclusion: Measures of discrimination from one point in time and in one domain may be insufficient to identify the overall impact of discrimination on individuals. Further research is needed to model and analyze longitudinal and other data and to study how effects of discrimination may accumulate across domains and over time in ways that perpetuate racial inequality. (from Chapter 11)
Recommendation: Major longitudinal surveys, such as the Panel Study of Income Dynamics, the National Longitudinal Survey of Youth, and others, merit support as data sources for studies of cumulative disadvantage across time, domains, generations, and population groups. Furthermore, consideration should be given to incorporating into these surveys additional variables or special topical modules that might enhance the utility of the data for studying the long-term effects of discrimination. Consideration should also be given to including questions in new longitudinal surveys that would help researchers identify experiences of discrimination and their effects. (Recommendation 11.1)
Our report emphasizes the challenges of measuring racial discrimination in various social and economic domains. Establishing that discriminatory treatment or impact has occurred and measuring its effects on outcomes requires very careful analysis to rule out alternative explanatory factors. In some research to date, the data and analytical methods used are not sufficient to justify the assumptions of the underlying theoretical model. Moreover, many analyses never articulate an explicit model, which makes it difficult to judge the adequacy of the data and analysis to support the study findings.
Just because it is challenging to measure discrimination does not mean that sound, adequate research in this area is not possible. To the contrary, existing methods and data have produced useful results on particular types of discrimination in particular aspects of a domain or process. To make further progress, we believe it will be necessary for funding and program agencies to support research that cuts across disciplinary boundaries, makes use of multiple methods and types of data, and studies racial discrimination as a dynamic process. To be cost-effective, such research should be focused and designed to maximize the analytical value of existing bodies of knowledge and ongoing surveys and administrative records data collections.
Agencies with programmatic responsibilities (e.g., to monitor discrimination, investigate complaints, and operate programs that may be affected by the presence of discrimination and by antidiscrimination laws and regulations) will need to single out priority areas of concern and develop detailed research plans for them. This may require studies of key decision-making processes, combined with theoretical models of the ways in which discrimination might occur. For this purpose, the existing literature of laboratory experiments about the kinds of situations in which discriminatory attitudes are most likely to lead to race-based discriminatory treatment should be reviewed and additional experiments commissioned, if the laboratory results are not sufficiently revealing about the decision processes of interest (e.g., employer decisions about job training and promotion, to take a labor market example). In turn, experimental results can help guide focused case studies of decision processes that may be needed to provide the requisite depth of understanding to permit subsequent statistical analysis with appropriate data and methods. To facilitate data availability and use, program agencies can not only support the addition of relevant questions to ongoing cross-sectional and longitudinal surveys but also work to improve the research potential of agency administrative records data.
Research agencies, both public and private, can best leverage their resources by addressing important areas of research on racial discrimination that are less apt to be considered by program agencies. In particular, they are better positioned to support innovative, cross-disciplinary, multimethod research on cumulative disadvantage. They can also usefully consider ways to augment ongoing and new panel surveys to provide relevant data for basic research on racial discrimination, particularly over long periods of time. The kinds of multifaceted studies that have been conducted in recent years of changes in the well-being of low-income populations following major changes in welfare policies may offer useful guidance for discrimination research, which could similarly make use of multiple data sources and perspectives from economics, psychology, ethnography, survey research, and other relevant disciplines. Such complex research will be difficult to conceptualize and carry out, but it offers the promise to expand knowledge about the role that current and past discrimination may play in shaping American society today.