Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics (2009) / Chapter Skim
Currently Skimming:
5 Principles and Practices: BJS as a Principal U.S. Federal Statistical Agency
5 Principles and Practices: BJS as a Principal U.S. Federal Statistical Agency
Pages 209-276
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 209... ...
One important filter through which to view the priorities and operations of BJS is its role as one of the principal statistical agencies in the U.S. federal statistical system.
|
From page 210... ...
As shown in Figure 5-1, the three largest statistical agencies -- the Census Bureau, the Bureau of Labor Statistics, and the National Center for Education Statistics -- dominate the others in terms of resources even though the subject-matter portfolios of the smaller agencies -- justice, transportation, agriculture, and so forth -- are undeniably important. It is appropriate, in the panel's judgment, to evaluate BJS in the context of the larger federal statistical system, especially the principal statistical agencies whose primary mission is the collection and dissemination of statistical information.
|
From page 211... ...
5–A PRINCIPLES OF A FEDERAL STATISTICAL AGENCY 5–A.1 Trust Among Data Providers A federal statistical agency must have the trust of those whose infor mation it obtains. Data providers, such as respondents to surveys and custodians of administrative records, must be able to rely on the word of a statistical agency that the information they provide about them selves or others will be used only for statistical purposes.
|
From page 212... ...
oblige BJS to violate the principle of trust among its institutional data providers. Specifically, the provision of information to a statistical agency is fundamentally different from the provision of information to a regulatory or enforcement agency.
|
From page 213... ...
The final act is lengthy, including specification of grant monies targeted at reduction strategies, the establishment of a national commission, and adoption of national standards. However, in this section, we focus on the specific demands put on BJS by a section of the act covering "national prison rape statistics, data, and research" -- reporting requirements that, in certain respects, run counter to the proper and accepted role of a federal statistical agency.
|
From page 214... ...
1435 -- consistent with7 the revised House language, but now bearing the name "Prison Rape Elimination Act." Upon introduction, the bill was immediately passed by unanimous consent without debate or amendment; the House took up the Senate bill on July 25 and passed it without objection; and the bill was signed on September 4, becoming Public Law 108-79. Text of the Act and Reporting Requirements Box 5-1 shows the alterations to the section of PREA concerning BJS data collection between its original introduction in the 107th Congress and final passage.
|
From page 215... ...
(3) SOLICITATION OF VIEWS- The Bureau of Justice Statistics shall solicit views from representatives of the following: State departments of correction; county and municipal jails; juvenile correctional facilities; former inmates; victim advocates; researchers; and other experts in the area of sexual assault.
|
From page 216... ...
, there is established, within the Bureau Department of Justice, the Review Panel on Prison Rape (in this section referred to as the ‘Panel')
|
From page 217... ...
and the Review Panel, with respect to prison rape, for the pre ceding calendar year to– (A) Congress; and (B)
|
From page 218... ...
. The critical difference in the legislative texts in Box 5-1 lies in the reporting requirements to support public hearings by the Review Panel.
|
From page 219... ...
BJS Data Collections and Reports in Support of the Act In response to the enactment of PREA, BJS organized a series of data collection efforts, summarized in Bureau of Justice Statistics (2004c) , that have been characterized as "a quantum leap in methodology and our knowledge about the problem" of prison rape (Dumond, 2006)
|
From page 220... ...
. Cognizant of BJS's legal reporting requirements, both the prison and jail releases from the National Inmate Surveys identified the names of institutions with high rates of offending; however, both have explicitly described an inability to identify the three highest-rate and two lowest-rate facilities as prescribed by the law.
|
From page 221... ...
b Number of respondents selected for the National Inmate Survey on sexual victimization. c Weights were applied so that inmates who responded accurately reflected the entire popula tion of each facility on selected characteristics, including age, gender, race, time served, and sentence length.
|
From page 222... ...
held 7 days of hearings in March 2008, in Washington, DC, and Houston, Texas, to obtain testimony from each of the adult federal and state prisons identified in Table 5-1. The National Prison Rape Elimination Commission issued press releases on the occasion of the BJS report releases and the start of the Review Panel hearing.
|
From page 223... ...
For many facilities the decision to label them as Tier A or Tier B is quite arbitrary and made purely by chance. This amply illustrates the problems of using statistical data for regulatory purposes.
|
From page 224... ...
Assessment Whatever the reasons for the change in reporting requirements, both the original and the final versions of the PREA bill violated the expected principles and practices of a federal statistical agency. BJS directly contributed to regulatory activities affecting individual data providers: explicitly singling out individual facilities to receive a summons to public hearings.
|
From page 225... ...
Again, once data providers lose trust that cooperation with BJS will not lead to individual harmful actions on them, the agency faces large problems. 5–A.2 Strong Position of Independence A federal statistical agency must have a strong position of indepen dence within the government.
|
From page 226... ...
BJS, through its administrative placement in OJP is not the most heavily , layered of statistical agencies, but it ranks among them. In the panel's judgment, the principle of a strong position of independence of a statistical agency was seriously violated in BJS's recent past by the circumstances surrounding the release of data from the 2002 PPCS.
|
From page 227... ...
So, in that respect, we have been functioning as a kind of semiautonomous statistical agency quite well. However, the BJS press releases -- I use the term "BJS press releases," but, actually, they are Department of Justice press releases of ficially, and they have always gone to the Department for clearance.
|
From page 228... ...
Attorney General OJP Assistant Attorney General Press Release DOJ BJS Dissemination Director, Office of Director Public Affairs Disseminate to Congress, media, and Post to BJS executive department website press offices Figure 5-2 Review, approval, and dissemination process for BJS survey press releases, 2007 NOTE: BJS, Bureau of Justice Statistics. DOJ, Department of Justice.
|
From page 229... ...
PRINCIPLES AND PRACTICES 229 Figure 5-3 Example summary and links to report and data on Bureau of Justice Statistics website Figure 5-4 Excerpt from example Office of Justice Programs press release accompanying new Bureau of Justice Statistics data release
|
From page 230... ...
A press release simply is not a statistical product and thus should not be treated as a statistical product at all -- let alone one that is somehow covered by the [CNSTAT guidelines in Principles and Practices for a Federal Statistical Agency.] A press release, rather, is a public relations announcement issued to encourage media coverage.
|
From page 231... ...
In the March 7, 2008, Federal Register, OMB published Statistical Policy Directive 4 on the release and dissemination of products from the federal statistical agencies. Defining a "statistical press release" as one of the product types covered by the directive, OMB "encouraged" agencies to issue press releases to accompany the issuance of new data and reports.
|
From page 232... ...
Immediately following the dispute over the 2002 PPCS press release, BJS Director Greenfeld resigned. As the narrative description by Lichtblau (2005a)
|
From page 233... ...
for a statistical agency to highlight its findings from the data, any methodological concerns that the new data may raise, and to promote accurate reporting and publicity of new results. Accordingly, press releases should share the same protections from interference as other BJS reports and releases.
|
From page 234... ...
We also recognize that there exists no organizational arrangement that -- on its own -- can completely shield a statistical agency from threats to its independence and guarantee freedom from political or structural interference (or the appearance thereof)
|
From page 235... ...
under the Justice Assistance heading. The Committee recommendation retains the ac count structure used in previous years and funds State and local law enforcement programs under seven appropriation accounts.
|
From page 236... ...
This classification drew protest from several social science organizations including the American Society of Criminology, whose executive board passed a resolution in November 2002 arguing that "the compilation, analysis, interpretation, reporting, monitoring, and management of crime and justice statistics .
|
From page 237... ...
as commercial, with the reason for classification as commercial listed as "pending an agency approved restructuring decision (e.g., closure, realignment) ." In our assessment, the collection and analysis of statistical data by federal statistical agencies is an essential government function; that OJP has not more fully realized this point suggests a continued incompatibility of functions between BJS and its administrative parent.
|
From page 238... ...
Though it left intact language from BJS's creation in 1979 giving the BJS director "final authority for all grants, cooperative agreements, and contracts awarded by the Bureau" (93 Stat.
|
From page 239... ...
(5) provide staff support to co ordinate the activities of the Office and the Bureau of Justice Assistance, the National Institute of Justice, the Bureau of Justice Statistics, and the Office of Juvenile Justice and Delinquency Prevention.
|
From page 240... ...
In our assessment, the inherent conflicts between the priorities of a program office such as OJP and a statistical agency such as BJS -- and the too-fine line between synergistic work by OJP offices and attempts to make those offices "speak with one voice" -- makes the status quo untenable in the long run. On the basis of these arguments, we conclude that BJS's administrative placement in OJP is detrimental: Finding 5.3: The placement of BJS within the Office of Justice Programs has harmed the agency's ability to innovate in data collections and expand the efficiency of achieving its statistical mission.
|
From page 241... ...
A separate office including both a research agency and a statistical agency would also be uniquely poised to develop research programs in justice-related issues that have received relatively little rigorous empirical treatment, such as the ex tent to which forensic evidence (e.g., fingerprints or firearm-related toolmarks) are introduced in judicial proceedings (and the effective ness of that evidence)
|
From page 242... ...
the Bureau of Justice Statistics would continue all of the functions currently carried out by BJS.
|
From page 243... ...
; emphasis in the original. Finding 5.4: Under current law, the director of the Bureau of Justice Statistics serves at the pleasure of the president; the di rector is nominated to an unspecified term by the president, with the advice and consent of the Senate (42 USC § 3732(b)
|
From page 244... ...
The events of 2005 demonstrated that BJS can be and has been harmed by the current arrangement by which the BJS director serves strictly at the pleasure of the administration. The circumstances of Director Greenfeld's dismissal -- in the immediate aftermath of refusing to alter a press release to address political concerns -- fostered the appearance of formal and structural interference in BJS's operations.
|
From page 245... ...
Only with such substantive expertise can the department's statistical agency produce optimally relevant statistical information to the policy makers of the department. Statistical agencies are part of the management information system for policy making in program departments.
|
From page 246... ...
5–A.4 Credibility Among Data Users A federal statistical agency must have credibility with those who use its data and information.
|
From page 247... ...
For some panel members in the audience of the workshop, some of the law enforcement community were asking for almost real-time event data -- a goal that is difficult for any statistical agency to achieve. Despite these types of critiques of BJS, panel after panel at the workshop expressed great belief that the BJS data series were credible, valued, and relevant to their work.
|
From page 248... ...
. It was not corrected in the version of the bill that finally passed the Senate; in the final consolidated appropriations bill that included DOJ, BJS funding came closer to the House mark than the Senate mark.16 As before, the panel concludes that a clear separation between BJS and OJP and placement of BJS elsewhere in the DOJ hierarchy would help clarify the mission of BJS and strengthen its profile as a principal statistical agency.
|
From page 249... ...
, much as the Bureau of Transportation Statistics has done for its re lated fields. BJS's role in such a journal or statistical publication -- and knowledge of strengths and weaknesses in BJS data -- could be enhanced by encouraging BJS staff or grantees to seek publication in
|
From page 250... ...
McEwen (1996) summarized the 1995 workshop on police use of force that contributed to the development of PPCS, and BJS partnered with SEARCH, the National Consortium for Justice Information and Statistics, on a series of workshops on law enforcement databases such as criminal history records and sex offender registries (Bureau of Justice Statistics, 1995, 1997b, 1998a)
|
From page 251... ...
Recommendation 5.8: BJS should establish an Advisory Group under the Federal Advisory Committee Act to provide guidance to BJS on the addition of new data collection efforts and the modification of current ones in light of needs identified by the group. Membership in the group should include, at a minimum, leaders and practitioners from each of the major subject matters covered by BJS data, as well as those with statistical and other types of academic expertise in these subject matters.
|
From page 252... ...
, and gain critical support for new and continuing data collections. Recommendation 5.9: DOJ should take steps to ensure that con gressional staff are aware of BJS data that could be used in devel oping legislation; DOJ and BJS should learn from congressional staff how their data are needed to inform/support legislation so that they can improve the utility of their current data and so that they can develop new data sets that could enhance policy development.
|
From page 253... ...
BJS shares with other federal statistical agencies a fundamental problem that it has insufficient numbers of technical staff whose primary job is to focus on evaluation of the quality of data collected by and for BJS. Because of this absence, the outside user of BJS data has no set of working papers, methodological briefs, or quality profiles that may be consulted to inform themselves of the characteristics of particular data sets or the potential strengths and weaknesses for their specific uses of the data.
|
From page 254... ...
As noted in Chapter 4, the state SAC network also provides a means for the dissemination of BJS data and products (and SAC analyses thereof) to local audiences.
|
From page 255... ...
Maximizing the use of BJS data requires that it be released in a timely and equitable fashion and in formats that facilitate its use, while protecting the confidentiality of the data and furthering the goals of the agency. These
|
From page 256... ...
; opportunities to present BJS data in similar structures should be considered. Recommendation 5.11: BJS should evaluate each of its data pro grams to inquire whether more timely estimates might be ob tained by (a)
|
From page 257... ...
This search logic may be obvious to some but not to others who visit the BJS website, and it is not clear that the formats and product lines currently available have a coherent and integrated dissemination plan or strategy. Increasing sophistication of the public with regard to electronic access to information may warrant a reevaluation of the mix of media used to disseminate BJS data.
|
From page 258... ...
As suggested by our comments earlier in this section, BJS recognizes the importance of its Web presence to the spread of its information. Former BJS Director Jeffrey Sedgwick (2008:2)
|
From page 259... ...
Some of the larger federal statistical agencies -- notably the Bureau of Labor Statistics and the Census Bureau (the latter through its "American FactFinder" interface) -- have made considerable efforts in permitting website users to tabulate (and even to plot
|
From page 260... ...
BJS's performance is certainly within the norms of other principal statistical agencies and we suggest that it could be improved still further through the recommendations we offer in the earlier section. 5–B.6 Fair Treatment of Data Providers [Fair treatment practices include]
|
From page 261... ...
Violating this practice can have consequences that take decades to undo. 5–B.7 Commitment to Quality and Professional Standards of Practice A statistical agency should: • use modern statistical theory and sound statistical practice in all technical work.
|
From page 262... ...
BJS staff are frequent participants in interagency working groups of staff from the range of federal statistical agencies. Several of these activities are topic working groups of the Federal Committee on Statistical Methodology, itself an interagency working group coordinated by OMB.
|
From page 263... ...
Public versions of the ICRs are browseable online at http://www.reginfo.gov by searching for data collections listed under OJP . On one hand, preparation of ICR supporting statements could be seen as no more and no less than clearing a bureaucratic hurdle.
|
From page 264... ...
They are also first drafts of the technical documentation for new data series and templates for actual data collection efforts. On these dimensions, neither new nor continuing BJS data collections are helped by having weak and deficient supporting statements made for them in a public (if not widely viewed)
|
From page 265... ...
is a useful example. The abstract mentions that collection is a "part of the BJS Law Enforcement Management and Administrative Statistics program," and the statement on the necessity of the collection references 2003 LEMAS data: It is estimated that about 250 law enforcement aviation units are in operation among State and local agencies in the United States.
|
From page 266... ...
BJS's request for clearance of the 2007 Survey of Law Enforcement Gang Units (ICR 200705-1121-001) shared some gross features of the aviation unit ICR, again using the "survey" nomenclature but describing the effort as a "nationwide census of all law enforcement gang units operating within police agencies of 100 or more officers." The supporting statement for the gang unit study does not explain whether any other data sources besides previous LEMAS returns are to be used to build the frame of dedicated gang units, leaving it unclear whether the collection is indeed a census (a canvass of all known gang units)
|
From page 267... ...
A statistical agency should be among the most intensive and creative users of its own data, both to formally evaluate the quality and properties of its data series but also to understand the findings from those data and shape future refinements. BJS's "Special Reports" series have, in the past, gone into depth on topics not routinely studied by the agency's standard reports or have taken unique looks at BJS data, such as age effects in intimate partner violence (Rennison, 2001)
|
From page 268... ...
. This is a laudable reaction that is a step toward laying out more concrete options for and future activities related to 18 "In addition," the plan notes shortly thereafter, "BJS staff meet regularly with Federal, State, and local officials to identify emerging data needs or desirable modifications to existing collection and reporting programs" (Bureau of Justice Statistics, 2005a:32)
|
From page 269... ...
It follows that this guidance can be applied to changes to other BJS data collections, and that such evaluative studies are not possible without the resources necessary to make innovative research a priority for the agency. Congress and the administration cannot reasonably expect BJS to shoulder daunting data collection requests without the agency engaging in ongoing research, development, and evaluation.
|
From page 270... ...
However, greater use of model-based estimates must be done with caution, for several reasons. One is the challenge of interpretation: Modeling may not be understood by many consumers of BJS data.
|
From page 271... ...
(National Research Council, 2009:47, 48) The practice of instituting a strong internal and external evaluation program is a new addition to the fourth edition of Principles and Practices of a Federal Statistical Agency.
|
From page 272... ...
This work should be done in partnership with other statistical agencies or data users, such as we describe below in Section 5–B.11 for comparing BJS's prison and jail censuses with the data quality and resolution provided by the Census Bureau's ACS. Other examples for multiple-source evaluation include: • Examination of differences between homicide rates computed from the UCR data and those from the cause-of-death data coded in the vital statistics that are compiled by the National Center for Health Statistics; • Reconciliation of the number of gunshot victims known to the police (or measured in emergency room admissions data)
|
From page 273... ...
When possible and appropriate, federal statistical agencies should cooperate not only with each other, but also with state and local statis tical agencies in the provision of data for subnational areas. (National Research Council, 2009:13)
|
From page 274... ...
. Of course, BJS's most intensive relationship with another statistical agency is with the Census Bureau.
|
From page 275... ...
5–C SUMMARY The panel believes that BJS and DOJ should conduct continual examination of BJS's fulfillment of the principles and practices of a federal statistical agency. Our panel's review found that the perceived independence of the agency was severely shaken by recent events.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.