Measuring the Impacts of Federal Investments in Research A Workshop Summary (2011) / Chapter Skim
Currently Skimming:
2 THE USES AND MISUSES OF PERFORMANCE MEASURES
2 THE USES AND MISUSES OF PERFORMANCE MEASURES
Pages 7-18
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 7... ...
. One objective of performance measures is to guide public decision making.
|
From page 8... ...
Will these scientific advances lead to the achievement of societal objectives? Finally, performance measures can benchmark accomplishments against historical or international measures and advocate for particular actions.
|
From page 9... ...
Researchers and policymakers sometimes refer to the "black box" of innovation - the complex process of turning knowledge into applications - and much research done in economics and related disciplines tries to explain what goes on inside the black box. Finally, performance measures can help policymakers avoid "fads" that direct attention in unproductive ways.
|
From page 10... ...
With regard to the allocation of money across fields, performance measures lead to multiple answers and therefore to multiple possible decisions. For example, bibliometric studies among journals might point toward the importance of biochemistry, economic research might point to
|
From page 11... ...
Do No Harm It is critically important to "do no harm," Feller emphasized. A major goal of developing performance measures is to improve the quality of decision making.
|
From page 12... ...
As an example, Sarewitz cited a statement by former NIH Director Harold Varmus that some cancer treatments are "unreasonably effective" but that it is hard to fund research on these treatments because such research is considered high risk. "I was stunned by this, because my view of the complexity of the innovation system is that if we understand that technologies and practices themselves are sources of problems that research can address, then one ought to see unreasonably effective cancer treatments as an incredibly potent attractor of research." However, the predominant model of research pursued at NIH is to understand the fundamental dynamics of a disease, which then will lead rationally toward the best treatments to use.
|
From page 13... ...
More generally, research performance measures may reveal little about the value and contextual appropriateness of the full array of science policy tools. For example, tools like demonstration and procurement, especially as done by the Department of Defense, have been enormous drivers of innovation in the past, yet they are outside the domain of research performance measures.
|
From page 14... ...
Case studies, for example, can produce synthetic systems-oriented insights that can have a powerful and enriching impact on policy making and "hopefully, change the narrative." Second, the science policy research community can do a better job of coming up with diverse performance criteria and measures that can support rather than displace qualitative insights. An interesting recent example involved the public policy analogues of market failures, which could be used to drive public investments in the same way that market failures have in the past (Bozeman and Sarewitz, 2005)
|
From page 15... ...
He said that most people working in his field would conclude that greater emphasis on patenting would reduce the rate of innovation. "Most faculty agree that patents in computer science basically are almost always a bar that reduces the rate of innovation by creating rigidities and without the benefits of the economic incentives that are supposedly being provided.
|
From page 16... ...
However, Feller added that it may be possible to document the need for transformative research. For example, NSF has been funding Science and Technology Centers that are focused on emerging scientific opportunities with important societal implications, such as hydrological research or the atmospheric sciences, that can have difficulty obtaining funding through conventional channels because they are too risky or large.
|
From page 17... ...
Measures of research performance can help agencies "get their house in order," said Feller, since many allocation decisions are still internal to agencies. However, measures demonstrating positive research outcomes do not necessarily guarantee that Congress will continue to allocate funds for those programs.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.