Evaluating Federal Research Programs Research and the Government Performance and Results Act (1999) / Chapter Skim
Currently Skimming:
1 Statement of the Problem
1 Statement of the Problem
Pages 13-24
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 13... ...
To that end, the law requires each agency to produce three documents: a strategic plan, which sets general goals and objectives over a minimal 5-year period; a performance plan, which translates the goals of the strategic plan into annual targets; and a performance report, which demonstrates whether the targets were met. Agencies delivered the first required strategic plans to Congress in Septem ber 1997 and the first performance plans in the spring of 1998.
|
From page 14... ...
OMB allowed considerable agency experimentation with different approaches to similar activities, waiting to see what ideas emerged. The expectations of and thus the guidance from the various congressional and executive audiences for strategic and perfor mance plans have not always been the same and that has made it difficult for agencies to develop plans agreeable to all parties.
|
From page 15... ...
One of the industry participants said that the objective of their industrial research is "knowledge generation with a purpose." The industry representative indicated that the company must first support world-class research programs that create new ideas; second, relate the new ideas to an important need within the organization or project; and third, build new competence in tech nologies and people. With respect to performance assessment, many industry participants noted that results of applied research and development programs are more easily quantified than results of basic research.
|
From page 16... ...
The second COSEPUP workshop focused on the strategic and performance plans of 10 federal agencies: the Department of Defense, the Department of Energy, the Department of Transportation, the Department of Agriculture, the National Aeronautics and Space Administration, the National Institutes of Health, the National Science Foundation, the Environmental Protection Agency, the National Institute of Standards and Technology, and the National Oceanic and Atmospheric Administration. As might be expected, most of these organizations use different approaches to translate the goals in their strategic plans into performance goals for scientific and engineering research.
|
From page 17... ...
Peer review of projects, grants, and contracts differs from peer review of programs and of intramural and extramural research. Those differences led COSEPUP to hold a third workshop focused on peer review and other methods for evaluating research.
|
From page 18... ...
with peer review when both methods are used. The primary argument against bibliometric anaylsis is that bibliometric measurements treat all citations as equally important.
|
From page 19... ...
. variance across agencies; concerns regarding use of "old boy network"; results depend on involvement of high-quality people in process Happenstance cases not comparable across programs; focus on cases that might involve many programs or fields making it difficult to assess fecleral-program benefit Retrospective Useful for identifying linkages Not useful as a short-term evaluation analysis between federal programs and tool because of long interval innovations over long intervals between research and practical of research investment outcomes Benchmarking Provides a tool for comparison Focused on fields, not federal research across programs and countries programs 19
|
From page 20... ...
Peer Review Peer review is the method by which science exercises continuous self-evaluation and correction. It is the centerpiece of 20
|
From page 21... ...
However, those distortions can be minimized by the rigor of peer selection, the integrity and independence of individual reviewers, and the use of bibliometric analysis and other quantitative techniques to complement the subjective nature of peer review. Peer review is not equally appropriate across the wide span of research performed by federal agencies.
|
From page 22... ...
This method is most appropriate for assessing a particular type of accountability question (for example, impact of National Science Foundation funding on mathematics research)
|
From page 23... ...
and S Cole, Peer Review in the National Science Foundation, Washington, D.C.: National Academy Press, 1981; GAO, Peer Review; Reforms Needed to Ensure Fairness in Federal Agency Grant Selection, June 1984.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.