10

Data and Research Opportunities

This report has outlined a vision for reimagining the criminal justice system in the United States and advancing racial equity. However, in order to sustain reform, significant investments in the collection and transparency of criminal justice data are needed. Without a viable data infrastructure, researchers, policy makers, and community members will be unable to track, monitor, and evaluate the effectiveness of new policies and their effects on racial and ethnic disparities. Data across the criminal justice system—from policing, to sentencing, to the jail and prison populations, and to probation and parole systems—are needed to illuminate the operation of the criminal justice system and its impact on inequality as well as to provide transparency and accountability and build trust.

With that need in mind, in this chapter we outline opportunities for building a data infrastructure that promotes integration across the criminal justice system and linkages with non-criminal justice data sources to better assess and support efforts to reduce racial inequalities. The chapter also highlights key areas where future research is needed to better understand racial and ethnic disparities in the criminal justice system and to evaluate promising policies and programs for reducing inequalities.

BUILDING A ROBUST CRIMINAL JUSTICE DATA INFRASTRUCTURE

A 21st century data infrastructure with timely, accurate collection, validation, curation, and dissemination is needed to reduce racial and ethnic inequalities in the criminal justice system at all levels of government. A

data infrastructure would help improve public understanding of crime and victimization, as well as the operation of police, courts, and correctional agencies. Such an infrastructure requires attention to three main pillars:

1. Integration of data systems and cross-system linkages and collaboration;
2. Consistent reporting of racial and ethnic data; and
3. Incentives for improving data quality and transparency.

Integration of Data Systems and Cross-System Linkages

The most vexing public problems require person-centric, multi-sector solutions, yet most of government today functions in agency siloes, where any given agency is disconnected from adjacent departments and their data systems. For example, addressing the criminal behavior of an individual experiencing homelessness cannot be solved without integrating data about that individual’s situation and needs across employment and education, health and mental health, or substance use sectors.

Linking data outside of the criminal justice area has the potential to deepen insight on racial and ethnic inequalities, yet most government authorities are unable to connect such data across public health, substance use treatment, homelessness, public safety, education, and the workforce, among others. Data and technology capacity, at the state and local levels in particular, is often inconsistent, ranging from high levels of sophistication, to paper-based data storage, to mainframe systems built over 40 years ago. Most state and local jurisdictions do not have staff dedicated to data management and analysis. These problems persist at the federal level as well.

Even more basic attempts to link person-level data within an agency can be difficult, much less linking person-level data across the system to track the individual’s progress. Simple questions like “did the person spend the night in jail?” or “how long did they wait before trial?” can be difficult to answer. Even more difficult are questions that require linking more than one individual record, such as “How many people in this adult prison were once in the juvenile system?” Connecting all stages of an individual’s experience through the criminal justice system in a jurisdiction, from the point of first encounter, through case processing and adjudication, through supervision, and finally to reentry is currently possible but only in a handful of jurisdictions, mainly those with grant-funded outside expertise.

In addition, existing data collections undertaken by the Bureau of Justice Statistics are mainly cross-sectional in nature and focus on relatively narrow, individual parts of the criminal justice system. These cross-sectional series often provide extensive coverage, but data on longitudinal flows and progressions through (and out of) the criminal justice system are rare.

That is, while criminal justice data systems regularly provide snap shots in time of different sections of the criminal justice system (often providing useful benchmarks), such data offer little insight into how individuals flow through the system or get released from it. The ability to follow persons from initial contact with a police stop to arrest and then through exit from the system is important for understanding the fairness and effectiveness of the criminal justice system at all stages of its operations, especially given the amount of discretion exercised throughout the system. Without longitudinal cohort data collections to track such “flow,” it is difficult to analyze how fairly that discretion is exercised. Moreover, linking such longitudinal data among decision points and connecting events to places for spatial analysis would enable greater understanding of the impact of the criminal justice system on racial and ethnic inequalities across communities.

Moreover, data integration can enable the use of data systems as a tool to provide reliable information on the actions of criminal justice agencies and institutional actors, in addition to those who come into contact with the system. For example, early warning systems related to police use of force, tracking of sentencing patterns that may exhibit bias, and complaints generated by people incarcerated in jails and prisons could be useful data for those seeking to reduce racial inequalities. The successful integration of such data systems can enhance transparency and accountability, as well as empower communities to advocate based on their needs.

Data linkage also offers the opportunity of placing criminal justice involvement in the larger social contexts of demography and community. For example, linking criminal justice records to the census and many other federal data collections would greatly enrich the statistical portrait of criminal justice involvement. Such linked data would contribute not only to public understanding of the socioeconomic status of people moving through the criminal courts and the penal system but also to a better understanding of the prevalence of criminal justice involvement in community life. Which communities contend with the greatest levels and disparities in incarceration? What has changed, and where? To be sure, such efforts carry serious technical challenges of accurate record linkage and data security, but overcoming these challenges to develop a highly integrated criminal justice data system would provide an important tool for reducing racial inequality.

The Criminal Justice Administrative Records System is an example of a national-level data system built to follow individuals throughout the criminal justice system. This data system works to track the criminal justice process from an initial arrest to a final sanction outcome though linking U.S. Census Bureau and other survey and administration data.¹ Data are collected through a combination of publicly available data, purchased

___________________

¹ See https://cjars.isr.umich.edu/

public information, and data use agreements with agencies.² This project began in 2016, and the resulting data were made available to the public in 2020 through the Federal Statistical Research Data Center network.

Lastly, it should be acknowledged that the conversation on data integration should be paired with a serious engagement of the potential civil rights and privacy risks that come with this type of data tracking. While data integration is necessary for understanding the scope of and potential solutions for reducing racial inequalities in the criminal justice system, great care and consideration should be taken to avoid unnecessary surveillance that could be used in harmful ways.

Consistent Reporting of Racial and Ethnic Data

Further contributing to the difficulty in understanding racial and ethnic disparities in the criminal justice system is the inconsistent way racial and ethnic data are captured. Such data are not gathered in a standard way across agencies and regions, making it difficult to get an accurate national picture of racial disparity. Many policies that may have a disparate impact on Black, Latino, and Native American people or low-income individuals (e.g., fees, fines, voter disenfranchisement) are not well documented in existing data collection efforts.

For example, according to a survey of state criminal justice data conducted by the Urban Institute, 40 states reported race (e.g., “White,” “Black,” “other”) in their arrest records, but only 15 states reported ethnicity (Eppler-Epstein et al., 2016). A state’s failure to collect and report ethnicity data affects not only Latino people but the entire criminal justice system. States that only count people as “Black” or “White” likely label most of their Latino prison population “White,” artificially inflating the number of “White” people in prison and masking the White/Black disparity in the criminal justice system (Eppler-Epstein et al., 2016).

Moreover, data on American Indians are also often difficult to obtain. A number of state and local data collections rely on an “other” race designation, which lumps together Asian Americans, Pacific Islanders, Native Hawaiians, and American Indians and Alaska Natives. Many also do not distinguish among tribes. This practice obscures differences between these groups and makes it difficult to determine how the criminal justice system plays a role in Indigenous communities (Daniel, 2020).

How official reports of race and ethnicity are actually recorded also remains poorly documented. Social-scientific understanding today views race and ethnicity as subjectively held identities that are best measured through self-reports. Historically, racial data were provided by census enumerators,

___________________

² See https://cjars.isr.umich.edu/overview/agency-partners/

but modern methods of data collection use self-reports. It is often unclear how data on race are obtained for prison, court, and other administrative records. Documenting methods of data collection would assist with interpretation and standardization.

Possible steps for improving the quality and consistency of reporting data on race and ethnicity include these:

• Meet current U.S. Census Bureau standards by collecting race and ethnicity data separately and allowing for a designation of more than one race. This would result in more descriptive and accurate subcategories, such as “non-Hispanic White,” “Hispanic Black,” and multi-racial categories.
• Allow race and ethnicity data to be self-reported. Self-identification within provided categories is consistent with U.S. Census Bureau data collection standards. However, it should be noted that the self-report of race in certain circumstances can potentially exacerbate stereotype threat, and therefore the risks associated with this method should be explored further.
• List Americans Indians and Pacific Islanders as distinct demographic groups rather than as members of an “other” category.

Relatedly, while urban American Indian populations may be growing they remain relatively small, which creates a problem for much quantitative research. Ethnographic research can fill some of the resulting gap, but new quantitative data efforts in areas with large American Indian populations—such as Albuquerque (NM), Phoenix (AZ), and Oklahoma City (OK)—are also needed. The exposure to crime and the criminal justice status of American Indians is generally poorly documented and therefore constitutes an important priority for data collection.

Incentives for Improving Data Quality and Transparency

Data challenges are present at the federal, state, and local levels. Across the 18,000 state and local law enforcement agencies and 50 state corrections departments, and among all the jails, courts, and prosecutors in the nation’s 3,000-plus counties, data entry clerks and professional staff are recording data and agencies are creating systems to track data without standard ways of recording similar information. The use of standards would provide the possibility of far greater ability to share data within a jurisdiction across agencies, and for the comparison of outcomes across jurisdictions.

Adopting shared standards would be an ambitious and long-term undertaking, as it would rely on major investments in technology system

upgrades, which are not only expensive but require procurement processes that may take months or years. Compounding these challenges is the fact that no single entity has responsibility for data quality across the system. Data omissions and errors in one part of the system create problems for accurately linking individual-level data across departments or agencies. At the same time, if data were linked, improving quality in one agency could help not only that agency but also the system.

The Department of Justice could incentivize state and local law enforcement and criminal justice agencies in collecting and sharing high-quality data, including both crime data and the operational and administrative data used internally, and in making the data more available for (1) sharing across agencies within a jurisdiction to aid in data-informed decision making, and (2) analysis by researchers, policy makers, and communities for insight. The federal government could also fund research and technical assistance on data quality metrics and how to implement them. More and better data will mean more state and local ability to manage with data and greater access for researchers, policy makers, and communities to gain insight and to advance the state of knowledge and evidence in the criminal justice system. While the Department of Justice currently requires collection of performance metrics on programs supported by federal grants, those data are reported only to the federal funding agency. The State Justice Statistics program, administered through the Bureau of Justice Statistics, also encourages data sharing and analysis. However, the program is limited, with average awards to state Statistical Analysis Centers amounting to roughly $60,000 per year.

Improving data quality and transparency also requires supporting federal statistical agencies mandated to collect criminal justice data with additional resources. As noted by the National Research Council (2009), the Bureau of Justice Statistics “is one of the smallest of the U.S. principal statistical agencies but shoulders one of the most expansive and detailed legal mandates among those agencies.” Funding for the Bureau of Justice Statistics has declined 25 percent since 2010, and by 37 percent when accounting for inflation, with an additional 11 percent cut in the fiscal year 2022 budget (American Statistical Association, n.d.). This keeps the agency from completing its work in a timely fashion—most of its statistical series (64% of them) have lacked either data collection or publication of results for five years or more. The lack of sufficient funding also hampers the agency’s ability to keep current with rapidly advancing analytics methodologies and modern tools for linking data across sources. It also hinders the Bureau of Justice Statistics from leading on advancing the state of science and from addressing key policy issues such as the difficulty in measuring racial bias in the criminal justice system. As a result, a modern data infrastructure—which is necessary for research, the advancement of evidence-based practices, and community trust—is missing.

From the perspective of the committee’s charge—reducing racial inequality—it is important to see community organizations and residents as a key audience for public data and as an important partner in the design of data systems. The design and dissemination of public data collections are often dominated by agency staff and researchers with little role for community representatives, who may not have strong academic or government connections. Developing a data system that can help community representatives reduce racial inequality will involve positive efforts at outreach and engagement by agencies and researchers alike. The committee believes such efforts at community outreach and representation can strengthen the utility of public data for reducing racial inequality.

AN AGENDA FOR FUTURE RESEARCH

Additional research is needed to enhance understanding of racial and ethnic disparities in crime and victimization and the inequalities both in and outside the system driving those differences. Moreover, as jurisdictions across the local, state, and federal levels continue to implement policies and programs to reduce disparities in their systems, those reforms will need to be evaluated to measure performance and assess outcomes with scientific rigor. Key items for a future research agenda include the following:

• Research on how racialized inequalities in socioeconomic conditions play out to produce differences in crime, and the long-term consequences of heightened exposure to toxic neighborhood environments, including violence;
• Research that advances methodology on how to better account for the fact that Black and White populations do not reside in comparably disadvantaged places;
• Research that improves assessment of the nature of competing viewpoints and norms of control, leveraging what researchers have termed the “ecometrics” of community measurement;
• Research that moves beyond a Black/White paradigm, reflecting the increasing diversity of the U.S. population;
• Research on the resiliency of communities to help illuminate how pockets of resiliency emerge and thrive in disadvantaged communities in ways that undercut vulnerability to crime;
• Research to enhance understanding of the collateral consequences of police-citizen contact, which may be especially salient for particular neighborhoods and communities;
• Research to better understand how criminogenic forces and processes work in rural areas, as much of the country’s Indigenous population resides in nonurban places;

• Research on alternatives to the criminal justice system, including community-driven efforts and place-based strategies, that identifies factors that can contribute to or impede effective and timely uptake and scalability of programs; this includes understanding system capacity for replicating evidence-based practices (e.g., by identifying training needs of professional staff and community members and providing adequate reimbursement) and other system incentives to ensure replication of evidence-based practices with fidelity (see Chapter 6);
• Research and data collection allowing for spatial analysis to better understand patterns of behavior across places and communities;
• Research and data collection to track historical changes in crime and criminal justice contact, so that progress and regress can be measured empirically; in combination with life-course and historical perspectives, this suggests the additional utility of tracking changes in crime and criminal justice contact across birth cohorts; and
• Research on language accessibility, including how language barriers for limited-English-proficient populations limit access to fair and just outcomes (see, for example, Ballard and Rodriguez, 2022).

Moreover, as discussed in Chapter 6, there is a need to broaden the research paradigm to be inclusive of other research methods beyond randomized controlled trials (RCTs). RCTs provide indispensable insight into the effectiveness of policies and programs, but they may not be an optimal or practical approach for evaluating multifaceted, often place-based social programs that may vary greatly in implementation and in the sites of implementation. From the perspective of developing a research agenda that can help reduce racial inequality, the committee supports a pluralistic approach that includes experiments along with observational studies, ethnographic and qualitative field research, and historical analysis. With such an agenda, researchers can employ methods from public health, demographic, sociological, qualitative, and participatory-action research in partnership with community members outside the research community.

RECOMMENDATION 10-1: Federal agencies and private foundations that support criminal justice research examining racial inequalities in the criminal justice system should draw upon the best available research—quantitative, qualitative, or mixed methods—to maximize knowledge and avoid the problem of limiting scientific research where empirical methods are not possible or constrained by the lack of quantitative data.

The incorporation of lived experience through the participation of directly impacted people in research design, implementation, and evaluation is critical to the inclusion of community voices, as they are often excluded from public policy making yet have invaluable direct knowledge of how existing systems cause harm and fail to prevent future harm.

Finally, a more robust and transparent data system is needed. Such a system could be an empowering resource for communities working to reduce racial inequalities. Community organizations and actors have typically been left out of the design of data systems. Ensuring that communities are included as a key constituency for data and research is an important next step for developing a scientific program that can positively contribute to reducing racial inequality.

This page intentionally left blank.