Frontiers in Agricultural Research Food, Health, Environment, and Communities (2003) / Chapter Skim
Currently Skimming:
6. Quality and Impact Assurance in the REE Agencies
6. Quality and Impact Assurance in the REE Agencies
Pages 119-145
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 119... ...
Both quality assurance and impact assurance are necessary follow-up activities for relevance assurance, which was discussed in Chapter 4. The REE agencies have implemented several changes to strengthen quality assurance during the last decade.
|
From page 120... ...
To ensure quality in the workforce, all ARS employees, including the scientific employees, are subject to annual performance reviews, and permanent scientists undergo a review of their progress on a 3- to 5-year cycle (discussed later in this chapter)
|
From page 121... ...
Review of quality of individual scientists annual performance reviews for all ARS employees and 3- to 5-year Research Position Evaluation System peer reviews for senior scientists · Solicitation of input on quality of ARS science from peer scientists and users Cooperative State · Annualreview of individual projects Research, Education, · Annual review of research programs and Extension Service · Peer review of research proposals for competitive programs (CSREES) · Review of output from and input into special grants (earmarks)
|
From page 122... ...
(See Chapter 4 for a discussion of relevance assurance mechanisms.) NASS reports that it uses historical track records to compare crop estimates and forecasts published during the growing season with the end-of-season final estimates to evaluate the accuracy of crop estimates and forecasts.
|
From page 123... ...
Peer Review Internal peer reviews of individual projects are conducted annually and at project completion in all the REE agencies. Research programs receive periodic reviews, generally at 5-year intervals.
|
From page 124... ...
The program appears to have encouraged the development of improved research proposals, which does not necessarily guarantee that the quality of research in a given unit will be improved but suggests that the quality of the overall research program should increase with time. However, the low rate of project termination resulting from either proposal review or performance evaluation suggests limited accountability of the system.
|
From page 125... ...
Intramural Research reported to the committee that NIH has a rigorous, largely retrospective, review system in place in its intramural research programs. In contrast with the review of extramural grants, which mainly assesses the quality of proposed research, the work of all principal investigators is reviewed mainly in retrospective fashion, in which the research program is evaluated in toto for its overall goals, quality of research, and long-term objectives, based in specific criteria (Box 6-1)
|
From page 128... ...
nature of the peer-review process, the extremely low rates of project termination, and the lack of impact of poor performance in proposal peer review on personnel grade level. RECOMMENDATION 7: The REE intramural research system should strengthen quality control for poor research performance.
|
From page 129... ...
A National Research Council review of the NRI concluded that the quality of scientific work in this program was high (NRC, 2000~. Special Grants CSREES special grants are added to the annual budget of the agency by congressional action.
|
From page 130... ...
Quality of Research Conducted for Action Agencies As discussed in Chapter 4, the REE agencies provide information to several action agencies, including the Animal and Plant Health Inspection Service, the Food and Nutrition Service, the Farm Services Agency, the Food Safety and Inspection Service, and the Natural Resources Conservation Service. In general, the committee's interviews with key stakeholders in client agencies in USDA indicated a high quality of research for short-term needs, although some agencies reported that some technologies developed were too complex to be user-friendly or were not relevant to the needs of the agency.
|
From page 131... ...
. Internal and External Recognition of Research Quality The committee found evidence that REE scientists produce research of high quality.
|
From page 132... ...
Onassis Public Benefits Foundation ARS American Society of Agronomy ARS Crop Science Society of America ARS American Dietetic Association ARS Pittsburgh Conference on Analytical Chemistry and Applied Spectroscopy ARS Washington State University, Pfizer Animal Health ARS American Oil Chemists Society ARS American Peanut Research and Education Society ARS American Veterinary Medical Association ARS American Society of Civil Engineers ARS Agronomic Science Foundation, Soil Science Society of America ARS American Agricultural Economics Association ERS Department of Energy ARS Rhone-Poulenc Award and Organization of Nematologists of Latin America ARS 1 Mississippi Chapter of the American Society of Agronomy ARS Strategic Environmental Research and Development Program (Department of Defense, Department of Energy, the Environmental Protection Agency) ARS National Academy of Sciences ARS American Society of Agricultural Engineers ARS National Science and Technology Council (White House Presidential Early Career Awards for Scientists and Engineers)
|
From page 133... ...
and the Office of Management and Budgetresearch that may yield low rates of scientific publication. In this section, the committee reviews the impact of REE research by using a variety of metrics focused on some dimension of the real output or payoff of research, including publications, citation frequency, patenting, longer-term quantitative measures, and social rate of return.
|
From page 134... ...
. In 2000, the REE agencies produced 2,035 scientist-years of research, and the federally supported state research institutions (state agricultural expenment stations [SAESs]
|
From page 135... ...
Rates of return are high but vary widely. Evenson's summary of rates of return on public aggregates agricultural research investments shows a median real social rate of return from 126 research studies of 45%; 66% were in the range of 21-80%.
|
From page 136... ...
(1992) Statistical method 1937-1942 1947-1952 1957-1962 1957-1972 1949-1979 1949-1989 1949-1959 1949-1959 1949-1959 1939-1948 1937-1942 1947-1957 1957-1962 1967-1972 1938-1972 1939-1972 1868-1926 1927-1950 1948-1971 1948-1971 1948-1971 1949-1972 1929-1977 1949-1959 1949 1954 1959 1964, 1969, 1974 1969 1943-1977 1959-1982 1950-1982 1930-1990 1930-1990 Pre-1930 1930-1990 1931-1985 1987 50 51 49 34 58 58 25-40 n.s.b 47 41-50 56 51 49 34 24-31 23-30 65 95 130 93 95 28-47 28-37 66-100 100 79 66 37 55 7-36 47 43 17-31 28 27 93 Negative 11.6 49-58 30 VA VA South North West aThe internal rate of return is a discount rate at which the present value of a series of investments is equal to the present value of the returns on those investments.
|
From page 137... ...
describes how a continuous investment in public agricultural research of 1% of output per year will contribute 0.76% per year to total factor productivity growth when the real rate of return is 40%. 9A total-productivity or multifactor productivity index measures the quantity of output produced compared with the quantity or cost of all the measurable inputs used to produce it (Ahearn et al., 1998)
|
From page 138... ...
Social rate of return has limitations in accounting for full environmental and social costs of research. Environmental, Economic, and Health Outcomes Aside from the long-run benefits of productivity enhancement measured through social rate of return, documented quantitative examples of the impact of REE research on other outcomes are scarce.
|
From page 140... ...
These are considered in detail in Appendix G This report has identified a need to broaden the REE agency focus on needs of and impacts on new, nontraditional stakeholders and to be more strategic in
|
From page 141... ...
ERS's recent redesign of its Web site to be more accessible and understandable to the publicthrough provision of links to nontechnical summaries, technical abstracts, data, and publications is an excellent model for how positive change could occur in this regard throughout the REE mission area. RECOMMENDATION 8: REE agencies should develop and adopt ways of measuring the national, long-term impacts of their research on the environment, human health, and communities.
|
From page 142... ...
SUMMARY This chapter has considered quality-assurance and impact-assurance processes and their outcomes in the REE agencies and has provided recommendations for improving the effectiveness of these processes. The use of peer review as a quality-assurance mechanism for research inputs and outputs was discussed with regard to intramural research, formula-funded research, special grants, and competitive grants.
|
From page 143... ...
l999b. Nonfasting plasma total homocysteine levels and all-cause and cardiovascular disease mortality in elderly Framingham men and women.
|
From page 144... ...
2001c. National Agricultural Statistics Service FY 2002 and Revised FY 2001 Annual Performance Plans.
|
From page 145... ...
Washington, DC: National Agricultural Statistics Service, US Department of Agriculture. Available online at http: //www.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.