1
Introduction and Overview
What evidence does the existing state of science offer to help advance global reforms in policing? In particular, what research can help guide efforts in a wide range of countries to enhance police capability to promote the rule of law and protection of the public? These questions are especially important during this critical juncture for policing around the world and are a priority of the U.S. Department of State’s Bureau of International Narcotics and Law Enforcement Affairs (INL). The mission of INL is to help “partner governments assess, build, reform, and sustain competent and legitimate criminal justice systems, and [develop and implement] the architecture necessary for international drug control and cross-border law enforcement cooperation.”1 To advance its own efforts to build knowledge, assess existing evidence, and improve its programs, INL asked the Committee on Law and Justice (CLAJ) of the National Academies of Sciences, Engineering, and Medicine to convene an ad hoc consensus committee to review and assess existing evidence on policing institutions, police practices and capacities, and police legitimacy in the international context.2
___________________
1 For more information about the Bureau, see https://www.state.gov/about-us-bureauof-international-narcotics-and-law-enforcement-affairs.
2 Recent legislation (The Foundations for Evidence-Based Policymaking Act of 2018) also directs federal agencies to improve the use of evidence and data to generate policies and inform programs.
THE COMMITTEE’S CHARGE
The Committee on Evidence to Advance Reform in the Global Security and Justice Sectors was assembled to identify the best available research evidence about policing that can support both the promotion of the rule of law (including human rights) and protection of the public. These two themes are defined in more detail in subsequent sections of this report. The committee comprises experts in criminology, economics, international and organized crime, law, policing, and political science, and it brings knowledge and experience from a portfolio of work that spans four continents.
Given the wide range of countries with which INL works and to which these questions might be applied—from democracies to authoritarian, corrupt, and fragile regimes that are limited in capacity or otherwise unlikely to be partners in reform—there can be a major challenge of contextual “fit.” Programs that work in some countries may not work in others. The same tactics, strategies, and interventions that could be successfully used in one state may backfire in another (or even be used to oppress people or violate human rights). Further, even where research findings about high-quality policing exist, they are not routinely used across and within country contexts that have similar policing structures. It is not only a question of “what works,” but “what works where” and “what works when.”
The committee was asked to consider these concerns and contexts while at the same time determining the applicability of existing research knowledge in policing, some of which has been reviewed by previous CLAJ consensus committees (see National Research Council [NRC], 2004, 2005; and National Academies of Sciences, Engineering and Medicine [NASEM], 2018). The committee was charged with producing a series of five reports, each addressing questions of interest to INL (see Box 1-1). To assist with this charge, the committee developed a series of five workshops to bring together both researchers who know the current and evolving state of scientific findings and practitioners who understand the decision-making environment, to address the questions in Box 1-1.
Public Workshop
This report is the first in the series, addressing the first question in the committee’s charge: What organizational policies, structures, or practices (e.g., HR and recruiting, legal authorities, reporting lines, etc.) enable a police service to promote the rule of law and protect the population?3 To
___________________
3 Each consensus report in this series of five reports will be released in PDF format in sequence of completion. A final, sixth report will compile the five reports into one published volume and be available through the National Academies Press (http://www.nap.edu) in PDF and hardcopy.
address the first question pertinent to this report, a workshop titled “Police Organizational Policies to Promote the Rule of Law & Protect the Population in the International Context” was held March 24–25, 2021. Workshop participants included members of the committee, representatives from INL, and international researchers and practitioners in the areas of recruitment, operational independence, and policing reform.
The workshop discussions were framed around a commissioned paper prepared by Peter Neyroud, Institute of Criminology, the University of Cambridge.4 The paper provided a descriptive narrative of “good practices” in a range of high-, medium-, and low-income countries and across different regions of the world. It assessed the strengths and limitations of the evidence and data for these practices and made observations on the research designs as well as the data that should be collected and reported to support and sustain the future development of the rule of law in democratic countries.
The workshop examined what is known and unknown (or untested) regarding re-constructing the policing landscape across entire countries, police recruitment policies, and the impact of police operational independence. Panelists were invited from the Northern Ireland Human Rights Commission, the United Nations Police Division, and academic institutions with expertise in researching these themes and experience in executing police reforms. Discussions at the workshop were a primary source of information for the committee’s deliberations, and speakers were identified based on the relevance of their work to the study question. Speakers with extensive experience internationally or in postconflict settings were prioritized to better ensure the utility of their comments to the report and sponsor. The committee met three times after the workshop to reach consensus on its conclusions and recommendations and to finalize its report. The committee was tasked with writing this report in a matter of months; the report therefore presents an overview of the state of research and highlights promising areas. It does not comprehensively describe all relevant research. This report does not provide a full proceedings of the workshop.5 Rather, it uses selected information from these resources (found as boxes throughout the report) as a springboard for offering conclusions and research agendas for the first question in the committee’s charge.
___________________
4 Available on the project webpage: see https://www.nationalacademies.org/our-work/evidenceto-advance-reform-in-the-global-security-and-justice-sectors.
5 Full recordings of the workshop are available: see https://www.nationalacademies.org/event/03-24-2021/evidence-to-advance-reform-in-the-global-security-and-justice-sectors-workshop-1-public-session-1.
The Committee’s Interpretation of Its Charge
The broad questions about public protection and rule of law extend through all five workshops and consensus reports, which collectively seek to accomplish the ultimate goal of the committee’s work: informing the U.S. Department of State’s (hereafter, State Department) capacity-building activities aimed at (1) strengthening the effectiveness of local, in-country law enforcement agencies, (2) building the technical skills of foreign law enforcement personnel through training and technical assistance, and (3) assisting in institutional police reform at the local level.
These questions were examined in the contemporary context of the State Department’s assistance to approximately 90 countries, as well as that of related foreign assistance efforts by the United Kingdom and Australia, working with the same or similar countries. INL programs focus on reforming or strengthening police effectiveness within host countries. The amount and type of assistance provided to specific police organizations and functions vary and are typically based on country-specific assessments. In some cases, INL police assistance programs may target areas for functional reform and capacity building, such as improving police training, internal accountability, police-community relations, civil disorder management and control, or management and supervision.6 All such programs share a general proposition that assistance from the United States can contribute to sustainable, institutional development in partner countries.
However, the committee is also keenly aware that even in the Global North, the United States in particular, high-quality knowledge from research is often not applied or heeded, and challenges to effectively promoting the rule of law and protecting the public are also prevalent. For example, during this committee’s deliberation, police agencies in the United States were being scrutinized for officer killings of unarmed people, excessive uses of force, corruption, and insurrectionists within their own ranks, and more generally they were being scrutinized for ineffectiveness in achieving everyday mandates.7 By making suggestions to INL, the committee assumes that the same recommendations could benefit all police agencies more generally.
The committee considered both the magnitude of the inquiry and the limitations of the current state of knowledge in scope and application. Given its charge to draw on literature from the international context, the committee understood that it would examine findings from research conducted in
___________________
6 Evidence to Advance Reform in the Global Security & Justice Sectors: Sponsor Profile for the Study Committee, see https://www.state.gov/wp-content/uploads/2019/03/222048.pdf.
7 It is important to note that there is much variation across police agencies in the United States, and while similar concerns about policing cross state boundaries, they do not necessarily apply to all police agencies in the United States.
multiple countries. Its default position was to identify the strong evidence that exists that might inform promoting the rule of law and protection of the public as well as the limits of applying that evidence across a highly variable landscape of policing. This task demands a scientific approach, which includes defining “rule of law” and “public protection,” applying evidence-based principles, drawing on appropriately validated research, understanding the units of analysis, examining the outcome measures for assessing effectiveness, and considering other factors that shape the outcomes measured.
RULE OF LAW
In the briefing paper written for the workshop, Neyroud (2021) offers a thorough discussion of the challenges in defining the rule of law (ROL). Several sources provide succinct definitions of ROL. For example, the U.S. State Department’s Bureau of International Narcotics and Law Enforcement Affairs defines it as:
A principle of governance in which all persons, institutions, and entities, public, and private, including the state itself, are accountable to [domestic] laws that are publicly promulgated, equally enforced, and independently adjudicated, that are consistent with international human rights norms and standards. (See https://www.state.gov/wp-content/uploads/2019/03/222048.pdf.)
The United Nations defines the concept as:
A principle of governance in which all persons, institutions, and entities, public and private, including the state itself, are accountable to laws that are publicly promulgated, equally enforced, and independently adjudicated that are consistent with international human rights norms and standards. It requires measures to ensure adherence to the principles of supremacy of law, equality before the law, accountability to the law, fairness in the application of the law, separation of powers, participation in decision-making, legal certainty, avoidance of arbitrariness, and procedural and legal transparency. (See https://www.un.org/ruleoflaw/what-is-the-rule-of-law/.)
Other dimensions and detailed conditions of the ROL are available in the broad legal and philosophical literature (e.g., Bingham, 2011; O’Donnell, 2004). From a scientific perspective, the committee’s primary concern is how any definition of ROL is empirically measured. Given its complexity, ROL is likely to vary in degree both across countries and within them over time. That feature has generated a range of cross-country measures for the ROL (Cheung, 2019; Rajah, 2012; Versteeg and Ginsburg, 2017), each of which tends to observe the kinds of features found in the existing codes
of human rights and policing practice promoted by international agencies. (See further discussion on measuring the ROL below in the section, Opportunity: Measuring the Rule of Law.)
Human Rights and Public Protection
Both the ROL definitions cited above highlight the principle that the laws of a state, as well as their enforcement, should be “consistent with international human rights norms and standards.” Adherence to human rights standards, understood as a set of normative commitments (Bottoms and Tankebe, 2017), is related to increased legitimacy in policing, including aspects such as restraining from the abuse of force and the provision of safety to the citizens to ensure they can exercise their rights and obligations. Neyroud’s paper (Neyroud, 2021) suggests that the human rights compliance required by ROL definitions can be achieved by a targeted approach to policing whereby police are encouraged to direct resources and police powers to areas of concentrated crime and harm. Such an approach requires having the information, data, and analysis to identify high-harm areas. This is termed evidence-based policing and is discussed in detail below and throughout the report.
The precursors to human rights violations are best pursued through informed policing (Hagan and Haugh, 2011; Scheffer, 2011). Human rights violations are often characterized by a small number of perpetrators harming large numbers of vulnerable victims. Strategically examining these crime contexts can help direct resources toward the disproportionately few offenders and in turn protect large numbers of the population. Discussion in Neyroud’s paper (2021) signals the importance of criminological attention to proportionality and how it might relate to law and legal theory. This is a novel perspective, because researchers in the fields of criminology and human rights law rarely engage one another. Nevertheless, it may be essential that these fields operate with mutual awareness and in mutually reinforcing ways.
International and regional mechanisms already exist to promote respect for and to defend human rights. The United Nations (UN) Office of the High Commissioner for Human Rights represents the world’s commitment to the protection of human rights as set out in the Universal Declaration of Human Rights.8 A number of charter-based and treaty-based bodies provide mechanisms for monitoring these rights in the UN system.9 For example, the Universal Periodic Review is a process in which the human rights records of UN Member States are reviewed and best practices in protecting
___________________
8 See https://www.ohchr.org/EN/AboutUs/Pages/WhoWeAre.aspx.
9 See https://www.ohchr.org/EN/HRBodies/Pages/HumanRightsBodies.aspx.
human rights and addressing violations are shared. Outcome reports are prepared for each review.10
Regional mechanisms include human rights systems in Europe, Africa, Latin America, and other regions. For example, Latin America and the Caribbean established the Inter-American human rights system in 1948, which gave birth to the Inter-American Commission for Human Rights in 1959, followed by the American Convention of Human Rights, adopted in 1969. The latter convention founded the Inter-American Court of Human Rights, whose mandate is to promote respect for and defend human rights in the region. It fulfils this mission by deploying on-site visits to observe the general human rights situation in a country, but can also investigate specific situations, and examine complaints or petitions regarding specific cases of human rights violations.
Regular reports issued by these bodies will likely provide valuable insights to agencies providing foreign assistance, like INL. Publicly highlighting priority areas for any violations may help such agencies attain adequate levels of transparency, accountability, and respect for human rights before engaging in police strengthening activities.
Police Institutions and the Rule of Law
A fundamental component of the rule of law is that the state itself be held accountable to the law. The police institution, as the entity to which the state delegates its monopoly on the legitimate use of force and with which citizens come into direct contact, has great responsibility to act in ways consistent with laws and international human rights norms and standards. The extent to which police forces adhere to constraints on the state’s coercive authority is a key indicator of the strength of rule of law in a given community or country. Promoting the rule of law therefore requires laws and policies that establish clear limits to police authority and actions—particularly regarding the use of force—as well as mechanisms for meaningful oversight and accountability. Such constraints on police officers’ authority and action are also important for protecting the citizenry from potential abuses by the police themselves. When police officers or police agencies engage in conduct that falls outside the scope of the law, they not only undermine the rule of law, they also directly contravene their obligation to protect the population.11
Measures that seek to rein in police abuses by imposing legal checks on police coercion support the rule of law and help protect the population
___________________
10 See https://www.ohchr.org/EN/HRBodies/UPR/Pages/BasicFacts.aspx.
11 In some countries, police action outside the law can result in assaults, petty corruption, and even torture and extrajudicial executions.
from potential abuses. Examples of such measures include the disbanding of Nigeria’s Special Anti-Robbery Squad, which had engaged in predatory and violent conduct against citizens; Colombia’s Constitutional Court ruling limiting police repression of protests; and the Brazilian Supreme Court’s oversight of lethal police operations in Rio de Janeiro. Restraints on policing, starting at the organizational level and working their way down to the individual officer, are therefore an essential tool in the promotion of the rule of law and the protection of the population. Moreover, according to one recent study in Liberia, police reforms in postconflict settings may also improve citizens’ perceptions of police (Blair and Morse, 2021).
EVIDENCE-BASED POLICING
The review, assessment, translation, and application of research evidence in policing has become widely known as evidence-based policing. This is an approach to police practice and management that involves using scientifically derived knowledge and processes to strengthen police departments’ decision making, actions, and overall agency functioning. In his 1998 “Ideas in American Policing” lecture for the Police Foundation,12 Sherman argued that “police practices should be based on scientific evidence about what works best” (Sherman, 1998, p. 2). In particular, Sherman focused on two dimensions of a research orientation in policing: (1) using the results of scientifically rigorous evaluations of law-enforcement tactics and strategies to guide decisions, and (2) generating and applying knowledge derived from an agency’s analysis of its own internal issues and crime problems. Such an approach may be contrasted with policing that makes decisions based on best guesses and hunches or even personal experiences, seniority, ideologies, or feelings (Lum, 2009). The prevalence of subjective decision making in policing has contributed to the use of ineffective practices, the failure to achieve sought-after outcomes,13 and the ongoing occurrence of preventable social harms by the police (Macbeth and Ariel, 2019; Sutherland and Mueller-Johnson, 2019).
At the core of an evidence-based policing approach is the idea that actions, tactics, programs, and technologies used by the police should actually deliver the outcomes expected of them. Sherman articulated the link between evidence-based policing and rule-of-law societies when he argued:
No institution is more important to that success than the police, whose
___________________
12 See https://www.policefoundation.org/publication/evidence-based-policing.
13 Such desired outcomes, particularly for the aim of promoting the rule of law and protecting the public, may include improving police legitimacy and minimizing police use of excessive force, justice-related discrimination, and disparities in criminal victimization.
competence at insuring the rule of law is constantly challenged by thousands of different opinions on how police should do their job. It is therefore essential that our society constantly improves the competence of its police not merely with our opinions, but primarily with facts derived from objective knowledge (Sherman, 2011, p. 1).14
Evidence-based policing is also aligned with a harm-reduction approach to criminal justice (Chalmers, 2003; Sherman, 2013). A harm-reduction approach emphasizes that police engage in activities that reduce all harm to society, whether those harms are crimes, injuries, civil harms, or financial harms and whether they are caused by criminals, by collisions, or by oppressive, unfair, or violent criminal justice systems and their agents.
Steps Toward Evidence-Based Policing
An evidence-based approach is applicable to all international police reform efforts. To promote the rule of law and protect the population, an evidence-based policing approach requires: (1) a reliable body of knowledge about police practices; (2) the ongoing practice of evidence-based and systematic targeting, testing, and tracking in policing; and (3) the institutionalization and implementation of knowledge into police practices (Lum and Koper, 2017; Sherman, 2013).
Reliable Body of Knowledge
There exists a robust body of policing research that can inform and advise on what is known about effective police operational tactics, strategies, and internal functions. Much of this research has been conducted in the Global North and focuses on the crime-prevention functions of policing. However, a growing body of research also addresses such concerns as police efforts to improve relationships with the communities they serve; improving trust and satisfaction in specific interactions with citizens; and strengthening internal accountability mechanisms for rule-of-law policing. This body of policing research has been systematically reviewed and compiled in several publicly available resources, including two previous National Academy of Sciences ad hoc consensus committee reports (NASEM, 2018; NRC, 2004); the Evidence-Based Policing Matrix15 (see also Lum and Koper, 2017); the
___________________
14 From his Benjamin Franklin Medal lecture to the Royal Society for the Encouragement of Arts, Manufactures, and Commerce, London, U.K., November 2011. For the lecture text, see https://www.crim.cam.ac.uk/system/files/documents/franklinfinal2011.pdf, and for the video recording, see https://www.youtube.com/watch?v=UqOq-5udMf0.
15 See https://cebcp.org/evidence-based-policing/the-matrix/.
Global Policing Database;16 and the Campbell Collaboration Crime and Justice Coordinating Group’s systematic reviews.17 The development of this knowledge has required collaborations between researchers and police agencies; objective research processes and an environment in which research findings are not suppressed; transparency and openness to data sharing and research by police agencies; and adequate investment and funding in building this research knowledge.
Ongoing Targeting, Testing, and Tracking
The practice of evidence-based policing also requires agencies—including those involved in promoting policing reform, like INL—to actively and consistently assess and evaluate their own practices against consistent standards. This requires not only consistently aligning their efforts to existing knowledge about policing, but also actively testing and evaluating their own practices to determine if those practices do, in fact, achieve sought-after outcomes (Sherman, 1998). Doing that requires police infrastructure, human resources, and information technologies that allow for accurate collection of information on crime, police activity, and citizen feedback to be able to track and understand these trends. Having high-quality information on both crime and internal police functioning can then facilitate more accurate targeting of problems with solutions supported by evidence, rather than the indiscriminate implementation of vague or non-evidence-based approaches. This capacity building equally requires a commitment to transparency, accountability, and responsible spending related to police activities and actions.
In the majority of policing agencies around the world, however, those high standards have yet to be consistently met. For example, many agencies continue to regularly carry out patrol and investigative activities that are known to be ineffective in protecting the public from crime or excessive police force. Most fail to proactively evaluate their own practices for effectiveness. Many agencies in the United States do not collect comprehensive information about their officers’ conduct and performance. This lack of systematic evidence undermines the capacity to create officer or agency accountability—such as infrastructures that would allow an agency to assess how well each officer is upholding the rule of law.
The absence of records or lack of transparency can create more challenges. In some countries, there is no written list of who has been appointed as a police officer, in turn preventing an accurate accounting of
___________________
16 See https://gpd.uq.edu.au/s/gpd/page/about.
17 See https://www.campbellcollaboration.org/contact/coordinating-groups/crime-and-justice.html.
staff numbers as well as officer fatalities. In such contexts, evidence-based policing may seem unlikely to succeed. However, raising awareness and support for the importance of record keeping and then building the capacity to create and analyze records may be precisely the place to begin a global strategy of police reform.
Institutionalization
Wherever records can be created and data generated from them, evidence-based policing can use them to implement research-backed practices. These records—with the assistance of INL or other development agencies—can inform the development and establishment of the scientific processes of targeting, testing, and tracking into the everyday systems of police organizations, including police training, supervision, management, accountability, leadership, deployment in the community, and use of technologies. Lum and Koper (2017) argue that this requires three additional activities: translation, receptivity, and institutionalization.
Translation.
Research and analysis are often presented in forms that are not practitioner friendly, emphasizing scientific aspects that may be of little interest to patrol commanders working on deployment operations or community members concerned about police activities. Therefore, research needs to be translated into digestible forms, directives, policies, and tactics that are both understandable and applicable to contemporary law enforcement and community concerns.
Receptivity.
Even when research is translated, an evidence-based policing approach requires that law enforcement agencies and their officers build receptivity to this knowledge, which in turn demands educational, structural, and cultural adjustments in law enforcement agencies that allow officers and agencies to be amenable to such knowledge (see specific examples in Lum and Koper, 2017). That is, evidence-based policing requires an environment open to change.
Institutionalization.
Embedding research into practice, in turn, requires fundamental adjustments to an organization’s systems of (and infrastructure for) incentives, accountability, deployment, supervision, management, leadership, technology, and professional development so that the organization can sustain an evidence-based approach over time (see specific examples in Lum and Koper, 2017). Evidence-based policing may also require legal frameworks and requirements to incentivize and institutionalize this approach. In theory, INL’s efforts fall under this translational and institutionalization part of evidence-based policing, although INL could also be actively involved in generating evidence as well as tracking and testing.
Challenges: Social, Cultural, and Political Factors
A key question for evidence-based policing reforms, then, is what political-social conditions afford the best opportunities to use scientific knowledge and generate and apply such knowledge. Police reforms, and any associated technical assistance and training, can often be challenged by social, cultural, and political factors. (See identified challenges from the INL Guide to Police Assistance in Box 1-2.) Given the variations in policing history, as well as in legal systems and regime types, the starting point for police reform is likely to vary greatly from one country to the next, and perhaps between regions within a country. Resistance to the rule of law by any elements of national governance that would lose power or money can
derail policing reforms, even when the knowledge and skills are ready to deploy. In some places, that resistance can be violent, with attacks on police stations and the murdering of police who endeavor to achieve reform. In other settings, resistance can be less visible but equally powerful. In addition to political resistance, social and cultural histories can shape the level of receptivity to reform. For example, the conditions under which slavery (or apartheid) was abolished likely influence the cultural landscape for police reform and may set countries apart in the ways in which they should approach reform.
Social factors, such as corruption, also impact the possibility of reform. Corruption and informal practices—both internal and external to the police organization—may have more sway over formal policies, structures, or practices and can fundamentally alter how police organizations function. Related conditions—such as when police are barely paid or resourced, where police work with directives that include political repression, or where there is overlap between police and political elites and organized crime—greatly affect what can be achieved (for example, see Gerber and Mendelson, 2008).
Examples from reform efforts in Africa demonstrate these contexts. For example, Hills (2020, p. 1546) describes Somali policing as “reflecting a series of social and political influences and relationships within fields of power characterized by inequality, with the emphasis on coercion, exploitation and accommodation.” Such characteristics might suggest that donors supporting policing reform should invest in providing the minimal requirements for state-based policing to address the challenges of stabilization and development, such as context-specific vetting, basic training courses, stipends for officers, and the use of activities and buildings to signal presence.
While it can be daunting to consider such challenges, initiatives can work well when they are customized to a context, and donors can play a catalytic role. The idea of “barrier mapping,” from the organizational change management literature (Lewin, 1951), may provide a path forward, as it allows planners to think about bypassing some barriers rather than confronting them directly. Such mapping, in the policing context, would consider how imported ideas may be received by local law enforcement and political elites, and how such ideas may be transformed once filtered through local interests and dispositions as well as the political will necessary to ensure not only that knowledge is accepted but that it is also effectively implemented (Hills, 2012).
In any case, identifying the barriers to change should be a part of every evaluation of efforts to reform.18 Once those barriers are identified,
___________________
18 “Pre-mortem” examinations of reform plans are similar and aim to generate ideas to consider and then avoid results on the back end, causing reform efforts to have failed (Kahneman, 2011).
along with the relevant actors, incentives, and resources, this can illuminate where and when it is feasible to proceed, where it is best not to proceed, and where it may be feasible to proceed but only with a plan for evading or overcoming any obstacles.
Challenges: Research and Measurement Issues
A fundamental question and challenge for implementing an evidence-based approach for INL’s reform efforts is how progress can be measured. There are questions about what kinds of data to collect, what units of analysis to use,19 how to manage data, and how comparisons should be made. It is important to recognize that not all police agencies are nationally governed, so country-level data may obscure the information being sought. There are many possible measures of citizen security and the rule of law, and many measures that extend beyond traditional crime statistics and measures for police use of deadly force. Presently, there is a lack of consensus over how the success of police reforms can best be measured. Even methods of counting homicides vary widely across countries (Andersson and Kazemian, 2018), as do measures of less extreme dimensions of citizen safety. Because the rendering of INL assistance can vary so widely across countries, it may not be necessary for a single measurement system of reform effectiveness to be used across them all. What may be most useful in each country is country-specific indicators. However, if a set of globally reliable measures could be developed to pass some global tests of reliability principles, there would be far greater benefits for guiding police development. Whether reliable measurement across countries and within them over time becomes more likely by improving existing measures (see discussion below) or by developing new ones is a question that an expanded research agenda for global police reform can address (see further discussion of the research agenda in Chapter 5).
Opportunity: Measuring the Rule of Law
Given the multifaceted and complex nature of the concept of the ROL, it is difficult to measure a country’s commitment to it and, more specifically and relevant to the committee’s charge, to measure a police service’s
___________________
19 In the context of providing foreign assistance, it seems reasonable to use the state (country) as the unit of analysis for measuring success or progress in police reforms. However, the disadvantage of using the state as a unit of analysis is that it limits the statistical power for measuring outcomes. There may also be advantages to using the police force itself as the unit of analysis, and in many countries those police forces exist at the subnational level. Where the police forces are administered subnationally (e.g., in Brazil, Argentina, and Mexico), there is a clear substantive and methodological reason for the police force to be the unit of analysis.
commitment to it. Notwithstanding this challenge, there have been several attempts to quantify and index ROL (Cheung, 2019; Rajah, 2012; Versteeg and Ginsburg, 2017). For example, beyond the two discussed here and in Appendix A, both the World Bank20 and the United Nations21 have projects to develop governance and ROL indicators. This section discusses and compares two of the most comprehensive indices which use different methodologies and sources of data to rank countries: (1) The World Justice Project (WJP)22 and (2) the Varieties of Democracies (V-DEM) Project.23 Both projects provide a composite ROL Index through a multidimensional set of outcome indicators, each of which reflects a particular aspect of this complex concept. The two measures are highly correlated. A comparison of these two indices is available in Appendix A.
The evidence reviewed by the committee suggests that research can, in principle, measure whether police reforms can support basic tenets of and potentially improve ROL. The committee identified candidate measures that satisfy requirements of basic face validity and are regularly produced for multiple countries over multiple years. The fact that such evaluations do not appear to be a part of current practice in overseas aid is no impediment to doing so in the future. Nor are the metrics currently available—including those reviewed here—the only possible way to assess reform effects on ROL. Rather, the evidence today demonstrates a proof of concept for such measures in general. Improved measures can and should be sought to enable evaluations of the benefits achieved by various strategies of police reform.
It is also possible to measure ROL below the country level. For more populous or sharply divided states, it may be essential to deploy surveys or other measures at a subnational level to measure, for example, the number of people who are killed by the police. With the growth in internet publication of local newspapers, it may even be possible to construct web-crawling tools to collect such data. Several newspapers have done this for tracking incidents involving U.S. police, for example, without incurring substantial costs.24 Nongovernmental organizations can also be commissioned to run observatories to document cases of police abuse, such as the Lethal Force Monitor in Latin America (Bergmann et al., 2019) and Campaign Zero in the United States.25
___________________
20 See https://info.worldbank.org/governance/wgi.
21 See https://www.un.org/en/events/peacekeepersday/2011/publications/un_rule_of_law_indicators.pdf.
22 See https://worldjusticeproject.org/our-work/research-and-data/wjp-rule-law-index-2020.
23 See https://www.v-dem.net/en.
24 See, e.g., https://www.washingtonpost.com/graphics/investigations/police-shootings-database.
The World Justice Project.26
Covering 128 countries from 2012 to 2020, the WJP’s ROL Index provides scores and rankings based on eight factors: (1) constraints on government powers, (2) absence of corruption, (3) open government, (4) fundamental rights, (5) order and security, (6) regulatory enforcement, (7) civil justice, and (8) criminal justice. In constructing the Index, the WPJ relies on national surveys of more than 130,000 households and 4,000 legal practitioners and experts. These primary sources are used to measure how the rule of law is experienced and perceived.
The WJP defines the rule of law as a durable system of laws, institutions, norms, and community commitment that delivers (1) accountability (the government, as well as private actors, are accountable under the law); (2) just laws (the laws are clear, publicized, and stable; are applied evenly; and protect fundamental rights, including the security of persons and contract, property, and human rights); (3) open government (the processes by which the laws are enacted, administered, and enforced are accessible, fair, and efficient); and (4) accessible and impartial dispute resolution (justice is delivered timely by competent, ethical, and independent representatives and neutrals who are accessible, have adequate resources, and reflect the makeup of the communities they serve). The four universal principles are operationalized using the eight factors listed above.
The Varieties of Democracies Project.27
V-DEM defines the ROL as “the extent to which laws are transparently, independently, predictably, impartially, and equally enforced, and the extent to which the actions of government officials comply with the law.” Methodologically, V-DEM’s measures are based on more than 3,000 country experts who provide their judgment on different concepts and cases. According to V-DEM, they typically gather data from five experts for each observation.
The index is formed by taking the point estimates from a Bayesian factor analysis model of the following 15 indicators: (1) compliance with high court; (2) compliance with judiciary; (3) high court independence; (4) lower court independence; (5) executive respects constitution; (6) rigorous and impartial public administration; (7) transparent laws with predictable enforcement, (8) access to justice for men; (9) access to justice for women; (10) judicial accountability; (11) judicial corruption decision; (12) public-sector corrupt exchanges; (13) public-sector theft; (14) executive bribery
___________________
26 The following is a snapshot of information on the World Justice Project and its ROL Index; for reference and additional information, see https://worldjusticeproject.org/our-work/research-and-data/wjp-rule-law-index-2020.
27 The following is a snapshot of information on the Varieties of Democracies Project and its ROL Index; for reference and additional information, see https://www.v-dem.net/en.
and corrupt exchanges; and (15) executive embezzlement and theft. The measure is, therefore, a composite interval scale, from low to high (0–1).
One of the advantages of using V-DEM’s Rule of Law Index is its large spatial coverage relative to the WJP (179 countries compared to 128 countries in WJP data) and temporal coverage (from the year 1789 to 2020). The V-DEM measure is less focused on street crime and public safety; many of the indicators in the V-DEM survey reflect specific components of indicators in the WJP indices. However, there are no explicit questions in the V-DEM survey distinguishing civil from criminal justice, or order and security. In contrast, the WJP survey does not explicitly ask about gendered rights or white-collar crime by government and private officials, and it does not distinguish between higher and lower courts. The analysis included in Appendix A compares these two indices in greater detail, reflecting the wide variety of ROL rankings, particularly in countries where INL operates.
ORGANIZATION OF THE REPORT
Following this introduction, Chapter 2 outlines the structures and governance within policing organizations, summarizing the various ways policing institutions are organized and the role that structures play in supporting the ability of policing to promote the rule of law and protect the population. Chapter 3 looks at the types of internal and external policies that promote accountability within policing agencies (to include internal recruitment and retention, reporting and supervision, disciplinary action, and technology). Chapter 4 examines practices of evidence-based, proactive policing, notably those related to problem-oriented policing, community-oriented policing, and the use of discretion to reduce harm. Chapter 5 lays out the committee’s conclusions and recommendations.
While findings in this report provide guidance on organizational structures, policies, and practices that should be considered to enable a police service to promote the rule of law and protect the population, the report also serves as a call to researchers and funding agencies to advance the research necessary to inform U.S. foreign assistance and the capacity building of criminal justice systems and police organizations in developing countries of need. Appendix A offers a validation exercise of two ROL correlates, and Appendix B provides biographical sketches of committee members and study staff.
