Strengthening Peer Review in Federal Agencies That Support Education Research (2004) / Chapter Skim
Currently Skimming:
1 Setting the Stage
1 Setting the Stage
Pages 9-19
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 9... ...
science policy; outlines current policies and initiatives with implications for peer review in federal agencies; highlights the inherent tensions between political and scientific values in federal peer review systems (in education in particular) , and describes our sources of evidence for, and approach to, setting forth our conclusions and recommendations.
|
From page 10... ...
As such, peer review is a tool through which researchers develop, sustain, and communicate their professional culture, suggesting the need for a buffer from political influences. Peer review in federal agencies, then, reflects both the principles of democratic accountability and the principles of scientific merit.
|
From page 11... ...
the structure and use of peer review across a broad swath of agencies, including those that fund education research. Some of these policies invoke peer review as a way to promote particular objectives, including enhancing the technical quality and credibility of information disseminated by federal agencies (e.g., U.S.
|
From page 12... ...
, the input of the scientific communities critiquing an early draft of the bulletin raised important questions and issues about the roles and purposes of peer review in general, many of which we address in this report. Later in this chapter, we highlight some of these commonalities as prologue to our consideration of peer review of education research proposals in federal agencies.
|
From page 13... ...
Recent federal initiatives in education research and the critical role of peer review in it also illustrate the often controversial negotiations and highstakes debates that characterize peer review in the federal government, providing further context for our consideration of peer review with respect to education research funding in the federal government. Most notable in this regard are a set of federal education policies aimed at applying "scientifically based research" to improve policy and practice.
|
From page 14... ...
In addition to providing context for our consideration of peer review of education research proposals, this snapshot of recent events illustrates the complexities inherent in peer review in general and relevant activities in education research specifically, laying the foundation for our discussion of the use of peer review in federal agencies that support education research. IMPLICATIONS AND THEMES Taken together, the conversations that have swirled around this array of activity go to the core issues involved in developing effective peer review systems and the proper role of researchers, elected officials, and other stakeholders in it.
|
From page 15... ...
We tackle the tough question of what counts as expertise in the diverse and applied field of education research. Specific issues of whether and how to involve stakeholders in the process and how to identify and deal with potential conflicts of interest and bias among candidates for participating in peer review panels are also addressed, picking up on the issues Graham and members of the scientific community raised about assembling panelists.
|
From page 16... ...
To help frame the event, we commissioned Edward Hackett and Daryl Chubin, authors of the highly influential book Peerless Science: Peer Review and U.S. Science Policy (Chubin and Hackett, 1990)
|
From page 17... ...
Our collective experience on panels is substantial, including service for such federal agencies as the IES and its predecessor agencies, several directorates of the NSF, a number of study sections in the NIH, the Veteran's Administration, the Department of Defense, and the Department of Energy; state agencies, such as the Massachusetts Department of Health; participation in panel reviews of research proposals to be funded by other national governments, such as those in the United Kingdom, Canada, Japan, Australia, and Israel; research reviews for philanthropic foundations and professional associations, such as the American Educational Research Association, the Association of Teachers of Preventative Medicine, the March of Dimes, the Spencer Foundation, the Markle Foundation, the Russell Sage Foundation, the Sloan Foundation, the All Kinds of Minds Foundation, and the McDonnell Foundation; and in peer review deliberations across a range of disciplines and fields, including sociology, history, psychology, cognition, epidemiology, statistics, cultural anthropology, nuclear physics, and an array of subfields in education research. OBJECTIVE AND APPROACH Our main objective in issuing this report is to inform decision makers charged with developing or maintaining peer review systems for education research proposals in a rapidly changing policy context.
|
From page 18... ...
Thus, we designed the workshop and wrote this report to apply to the range of federal agencies that use peer review (or will in the future) to aid education research funding decisions -- including, but not isolated to, the IES.
|
From page 19... ...
Building on that discussion, Chapter 3 contains our proposals for strengthening and improving peer review systems used to aid decision making about the federal funding of education research.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.