Strengthening Science at the U.S. Environmental Protection Agency Research-Management and Peer-Review Practices (2000) / Chapter Skim
Currently Skimming:
Research Management at EPA
Research Management at EPA
Pages 35-98
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 35... ...
In fiscal year 1999, ORD had 1,976 staff members at 12 geographically dispersed laboratory facilities, three field stations, three assessment offices, and the headquarters office in Washington, DC. ORD also had an extramural research budget of about $315 million—more than half of its $559 m~lion total budget in fiscal year 1999—for grants, cooperative and interagency agreements, contracts, and fellowships.
|
From page 37... ...
in response to a recommendation from our companion committee In this NRC study (NRC 1997) , ORD and the EPA Science Policy Council, with the assistance of other EPA offices, began to develop in 1998 an agency-wide "inventory of science activities." The inventory is intended to become an "evergreen" interactive planning too]
|
From page 38... ...
And, of course, the academic community and the private sector conduct much of the research relevant to EPA's mission. In Safeguarding the Future: Credible Science, Credible Decisions (EPA 1992)
|
From page 39... ...
As recommended in Future Risk: Research Strategies for the 199Os (EPASAB 1988) and Building a Foundation for Sound Environmental Decisions (NRC 1997)
|
From page 40... ...
ORD's strategic plan (EPA 1996a, 1997a) states, "While all of EPA uses science for policy and regulatory decision-making, and various EPA offices contribute to the scientific underpinnings of the Agency's decisions, the responsibility for leadership in science at EPA and for the bulk of EPA's research and development work resides in the Office of Research and Development." Yet, EPA's statutory mandates and regulatory programs have historically been problem-driven, and consequently so has most of ORD's program.
|
From page 41... ...
, our companion committee in this NRC study recommended that EPA's research program maintain a balance of roughly equal proportions between problem-driven research, targeted at understanding and solving particular, identified environmental problems and reducing the uncertainties associated with them, and core research, which aims to provide broader, more generic information to help improve understanding relevant to environmental problems for the present and future. It described problem-driven research as the kind of research and technical support activity that ORD has pursued most in the past, efforts that are largely driven by current or anticipated regulatory efforts of other EPA offices.
|
From page 42... ...
, as well as new approaches to managing current problems. Safeguards will continue to be needed to ensure that the important scientific needs of EPA's regulatory programs and regional offices are not unduly compromised.
|
From page 43... ...
Regulatory officials, on the other hand, often argue that EPA's limited resources, including ORD's, are provided to support the agency's existing statutory mandates and regulatory programs, so ORD should provide the regulatory offices with more technical assistance and short-term, quick-payoff, applied work. Our committee is convinced, as was our companion committee (NRC 1997)
|
From page 44... ...
ORD's technical assistance provides the regulatory offices with competent scientific support, and it enables the ORD research scientists to keep abreast of regulatory and policy developments elsewhere in EPA. ORD should meet the continuing challenge to lead the agency through research while continuing to assist its client regulators, who have variable levels of understanding and appreciation of science but a strong say in ORD's budget and priorities.
|
From page 45... ...
Among its findings, the MITRE report expressed concern about the lack of clear, agreed-upon mission statements for EPA, ORD, or the laboratories. It also commented on the excessive use of contract personnel at EPA laboratories, various facility and equipment problems, and the need for improvement in quality-assurance and research-planning practices.
|
From page 46... ...
It also expressed concern about the possibility that decreased ORD involvement in short-term, applied research and technical assistance to the regulatory and regional offices might lead to the expansion of separate research programs in the regulatory offices, potentially damaging the overall quality and effectiveness of ORD's research role in the agency. The NAPA review also generally supported the MITRE recommendations but emphasized that ORD's mission and goals should be clarified before any laboratory reorganization took place.
|
From page 47... ...
To increase its interactions with the academic community by expanding its competitive extramural research grants and fellowship programs. · To expand and strengthen its peer-review practices.
|
From page 48... ...
identified the following benefits and limitations: Benefits . Aligns laboratory missions with EPA's unique research responsibilities for reducing uncertainty associated with risk assessment, with research priorities directly related to needs of the environmental management decision process.
|
From page 49... ...
Creates the opportunity for greater scientific career development in the staff. Limitations Science leadership and innovation by national laboratories must be guided by a strengthened research planning and decision process that includes effective participation of program and regional offices in establishing needs, priorities, and accountability.
|
From page 51... ...
With the restructuring of the ORD research enterprise, it was considered essential that ORD laboratory personnel be capable of performing state-ofthe-art research techniques, that they take leadership roles in national and international environmental research initiatives, and that they interact effectively with those com portents of EPA that need technical information. Before 1995, roughly 30% of ORD's resources were devoted to what ORD called fundamental and anticipatory research.
|
From page 52... ...
Now, the committee views the 1995 reorganization of ORD and the agency's peer-review practices to be still a work in progress but sufficiently mature to be assessed in the following pages. RESEARCH PLANNING One of the principal findings in Safeguarding the Future: Credible Science, Credible Decisions (EPA 1992)
|
From page 54... ...
recommended the following in regard to scientific plar~Iiing in EPA: Most sorely needed are a coherent scientific and technical strategy statement for EPA, a strategic plan for OR D, anti a strategic plan for each ORD laboratory anal center that is consistent with the agency and ORD plans. Each strategic plan should consist at a minimum of a vision statement, a mission statement, and a plan for achieving them.
|
From page 55... ...
The ORD strategic plan (EPA 1996a, 1997a) set forth a simple vision: "ORD will provide the scientific foundation to support EPA's m~ssign." It defined ORD's mission in four statements: · Perform research and development to identify, understand, and solve current andfuture environmental problems.
|
From page 56... ...
· Through an innovative and effective human resources development program, nurture and support the development of outstanding scientists, engineers, anal other environmental professionals at EPA.
|
From page 57... ...
· To provide credible, state-of-the-art risk assessments, methods, modeZs, and guidance. · To exchange reliable scientific, engineering, and risk assessment/risk management information among private and public stakehoZders.
|
From page 58... ...
2EPA Program Offices and Regions may sell choose to fund. using ORD labs.
|
From page 59... ...
59 o c .~ ~ _ i i · ~ ~ · · ~ - - -\ ~~ -uo.~ o~m u-~:o C U 0 ~ U U (I3,U U C c,C {y ~ e ~ ~ ~ ~ | ~ ~ ~ ~ .
|
From page 60... ...
Of the six highest-priority topics chosen, three are targeted at specific environmental problem areas—safe drinking water, with an initial focus on microbial pathogens, disinfection by-products, and arsenic; high-priority air pollutants, with an initial focus on particulate matter; and emerging environmental issues, with an initial focus on endocrine disrupters. The other three high-priority topics address broader questions of methodology and approaches—research to improve ecosystem risk assessment; research to improve health risk assessment; and research on pollution prevention and new technologies for environmental protection.
|
From page 61... ...
; the development of budget operating plans and laboratory implementation plans for inhouse work or appropriate extramural mechanisms, such as Operants, cooperative agreements, or contracts; and the process of planning for research-information management. Our committee commends ORD for the progress it has made to date in developing its strategic and research plans.
|
From page 62... ...
During the committee's site visits and interviews, the staff of some EPA regulatory program offices expressed concern that ORD's support of their programs appears to have diminished since the 1995 reorgani
|
From page 63... ...
As a result, they acknowledged to the committee that EPA regulatory offices engage in some research without ORD involvement, even though those other offices are unable to afford much research. Dissemination and Technology Transfer Safeguarding the Future: Credible Science, Credible Decisions (EPA 1992)
|
From page 64... ...
At the same time, the committee noted during its site visits and interviews with managers and staff of EPA's regulatory offices that not a single individual was in favor of moving the research program outside the agency or even moving the scientific and technical-service functions and resources from ORD to the control of the regulatory program offices. Each group of regulatory officials was asked that question, and the predominant answer was that such moves would weaken, not strengthen, the scientific foundation of EPA's actions and decisions.
|
From page 65... ...
The plans and strategies contained little information about the resource levels required and measurable results or timetables associated with the anticipated research. The plans and strategies generally give little insight as to how the research activities will be orchestrated and coordinated among ORD laboratories and other agencies and organizations.
|
From page 66... ...
General Accounting Office (GAO) criticized ORD's performance in reporting the progress of some of the other research plans listed above, noting that one of the agency's regulatory program offices so acutely needed information on ORD's work, well beyond what ORD was providing, that the regulatory office paid for the development of a system to track ORD's work.
|
From page 67... ...
projects. Commenting further on the lack of transparency and progress reporting in ORD's research planning and budgeting process, the GAO report stated, .
|
From page 68... ...
, and the ORD national laboratories and centers had each developed mission statements, the BOSC found that none of the laboratories and centers had developed its own strategic plan. Recently, EPA advised our committee that all the laboratories and centers have drafted such plans, but they were developed too late for consideration by the BOSC in its first program reviews, and they were not provided to our committee.
|
From page 69... ...
individual research program and project priorities should be developed by individual laboratories and their divisions in response to the overall strategic goals and m~ssions defined from above. In other words, the mission and goals should largely be defined in a top-down process, but the laboratory programs and projects should be defined in a more bottom-up process by the researchers who will implement them.
|
From page 70... ...
was formed by combining former ORD laboratories in Research Triangle Park, NC, Cincinnati, OH, Teas Vegas, NV, and Athens, GA, with headquarters in Research Triangle Park, NC (see Figure 2-4~. In fiscal year 1999, the Exposure Laboratory had 448 employees and a total budget of $109 million, including a $45 million extramural budget.
|
From page 71... ...
was formed by combining former ORD laboratories in Cincinnati, OH; Research Triangle Park, NC; Ada, OK; and Washington, DC, with central administration in Cincinnati (see Figure 2-4~. In fiscal year 1999, the Risk Management Laboratory had 393 employees and a total budget of $108 million, including a $62 million extramural budget.
|
From page 72... ...
Principal investigators at the laboratory reported to BOSC that the research environment within the Effects Laboratory had substantially improved in recent years. The BOSC concluded that the Exposure Laboratory was "conducting some high-quality, peer-reviewed science in high-priority areas for EPA" (EPABOSC 199Sb)
|
From page 73... ...
. It expressed concern that only 168 of the 407 staff at the Exposure Laboratory had research degrees at the masters or doctoral level, whereas 103 staff positions were in administrative jobs, and about 45 of those were strictly management.
|
From page 74... ...
Noting that the Risk Management Laboratory's research priorities are heavily influenced by statutory requirements or court orders, as well as ORD, the BOSC expressed concern about the vagueness and lack of clarity of the {aboratory's understanding of its research scope, how it sets its research priorities, how much flexibility it has in sewing such priorities, and how it makes decisions about the allocation of available resources to various research activities. The BOSC also questioned whether the Risk Management Laboratory was preparing adequately to meet the part of its research mission dealing with the management of ecological risks.
|
From page 75... ...
. When ORD shifted the emphasis from extramural management to in-house research, the shift had a major impact on the Risk Management Laboratory, despite the fact that the Risk Management Laboratory still had 111 active cooperative agreements with 75 academic institutions at the time of the BOSC review (EPABOSC 199Sc)
|
From page 76... ...
However, the committee cautions that excessive use of external review panels can undermine the sense of responsibility of laboratory managers for identifying and resolving problems Each of the ORD laboratories (and the Assessment Center) has conducted a competitive internal research-grant program.
|
From page 77... ...
Until 1995, funding for the grants program fluctuated between $5 million and $25 million per year. In Safeguarding the Future: Credible Science, Credible Decisions (EPA 1992)
|
From page 78... ...
First, recognizing that the immense need for scientific and technical knowledge about environmental problems far exceeded any realistic assessment of the budgets and capabilities of EPA's in-house laboratories, the agency wanted to increase greatly the collaboration and potential contribulions of the nation's academic community through an expanded and strengthened, competitive, investigator-~n~tiated grants program. Second, as discussed in the next section of this chapter, the agency wanted to reverse the trend of previous years in which many in-house research scientists and engineers in ORD laboratories were increasingly spending their time as administrators of extramurally funded projects at the cost of doing less and less in-house research.
|
From page 79... ...
Topics for STAR program solicitations are developed in a process that begins with agency-wide research-coordination teams that include representatives from ORD's national laboratories and centers, as wed as EPA regulatory and regional offices. After considering the ORD strategic plan, the needs of other EPA offices, and input from external groups, decisions about potential solicitation topics are made on the basis of such criteria as the need for improving risk assessment or risk management in a topic area, the suitability of the topic for the grants process, the extent to which the topic may complement or be per
|
From page 80... ...
Today, peer reviews for scientific merit in the EPA STAR grants program are conducted under policies and practices that are generally modeled after those of NSF (1998) and are also similar in many ways to the extramural grants programs of the National Institutes of Health (NTH)
|
From page 81... ...
One important difference between the EPA grants program and the NSF and NIH programs in terms of peer-review practices derives from the different agency missions. NSF and NTH are science agencies, so their reviews focus mainly on technical merit and potential unpacts of the proposed research.
|
From page 82... ...
Merit review criteria in the EPA grants program address both the research proposal and the investigator. For the proposal, reviewers are asked to assess factors such as originality, creativity, potential scientific knowledge contributions, appropriateness and adequacy of methods, technical merit of the proposed approach, feasibility, and quality assurance plans.
|
From page 83... ...
The center seeks to involve ORD laboratory personnel and promote communication with regulatory program staff in the grants program through its web site, special reports, progress-review workshops, and other mechanisms, including informal communication. In 1997-199S, ORD's BOSC conducted program reviews of the national centers, as they also did for the national laboratories (see previous section)
|
From page 84... ...
In view of the agency's vast needs for risk assessments, the many risk-assessment activities being performed by the agency's regulatory program offices, and the limited resources available to the Assessment Center, the BOSC urged the Assessment Center to rethink its role and become more of a risk-assessment leader, catalyst, and resource service center for the other offices of the agency—a source of advice, guidance, and methodology—rather than a primary performer of individual risk assessments. The BOSC suggested that the Assessment Center could be most effective by focusing not on performing or trying to "own" or control individual risk assessments, but rather on improving and supporting the scientific underpinnings of the risk assessments performed across the agency through the development, acquisition, testing, and maintenance of state-of-the-art methods and information that support the agency's risk-assessment activities.
|
From page 85... ...
The Assessment Center should focus on being a research organization dedicated to advancing the state of practice in risk assessment, not a performer of individual risk assessments that could be done by EPA's regulatory offices. We commend the Extramural Center for developing and conducting the STAR research-grants program in an open, careful, and credible process of national competition and independent merit review.
|
From page 86... ...
These · ~ are w~n-w~n ventures. The committee also commends the Extramural Center for developing and maintaining its excellent internet site, which makes available to everyone the abstracts of research applications funded by the Extramural Center and its interagency partners, the annual and final reports from grantees, and reports of workshops on the integration of research results and their relevance for decision-making.
|
From page 87... ...
o -- r _ ~ ~ c~ _ Although the committee supports the strengthened and expanded STAR grants program, it also recognizes that the funds used to increase the program approximately fourfold, from $23 million in 1993 to approximately $100 million today, came from reductions in other ORD programs. Much of it came from funds that the laboratories had previously used for interagency agreements, cooperative agreements, and contracts with other research organizations in government, the academic community, and the private sector.
|
From page 88... ...
concluded that an inadequate infrastructure and lack of long-term support have limited EPA's ability to attract and retain outstanding scientists, and that EPA did not yet have the critical mass of such scientists needed to make EPA science generally credible to the broader scientific community. The 1992 report recommended continued attention to appropriate science and science-management career tracks research career tracks for scientists in ORD, and career tracks for scientists in the agency's program and regional offices that are similar to those for agency attorneys.
|
From page 89... ...
In 1983, this program and the resources supporting it were transferred to EPA's policy office; the budget for that program diminished considerably and was recently eliminated. ORD has continued to fund extramural economic and social science research at a modest level through its competitive grants program, and the policy office and EPA regulatory offices perform some economic studies, but EPA's inhouse program in economics and social-science research has diminished from approximately 30 ORD staff members in the 1970s to an almost entirely grants-based extramural program today.
|
From page 90... ...
In its 1995 reorganization, ORD sought to reverse the trend of previous years in which many in-house research scientists and engineers in ORD laboratories were increasingly spending their time as administrators of extramurally funded projects at the cost of doing less and less in-house research. The primary concern was that EPA's laboratories were losing their expertise and abilities to perform first-rate research.
|
From page 91... ...
In addition, concerns were heard about favoritism and poor oversight by ORD laboratory personnel in the administration of some externally funded projects. To change that, the agency centralized much of its extramural research funding in a rigorously competitive STAR grants program, and it sent a strong message to in-house laboratory staff that they would be expected to do more research and less administration, and that they would be judged mainly by research accomplishments in Me future.
|
From page 92... ...
At times, however, many ORD staff have been discouraged and pessimistic about the future of ORD and frustrated and uncertain about prospects for their own professional careers within ORD. The concerns heard by our committee were many: too much disruptive change in budgets, priorities, and policies, often crisis-driven; excessively bureaucratic procedural hurdles; too many scientifically underqualified administrators instead of research scientists managing laboratory programs; failure to replace departing scientists and technicians with new talent in a timely manner; lack of trust up and down the management chain; institutional faultfinding and paranoia; inadequate travel funds and other infrastructure support for nonmanagerial scientists; lack of explanations for decisions; unkept promises; criticism from Congress and others; pessimism that anyone will listen or be able to help.
|
From page 93... ...
A lack of stability in goals, priorities, practices, structure, or funding can be especially harmful to a research organization. ORD's historical lack of stability and sometimes disruptive changes have been attributed to growth in EPA's legislative mandates and priorities; specific directives from Congress in the appropriations process; changes in political administrations; changes in public attitudes; lawsuits and court decisions affecting regulatory programs and associated scientific needs; pressures from public groups or regulated parties; inadequate budgets to meet competing demands; recommendations from outside groups; and changes in the leadership of ORD.
|
From page 94... ...
Continuing career development for ORD's research staff is critical to the quality and productivity of their research. Opportunities for professional development are especially important for ORD scientists and engineers who, in the 1995 reorganization of ORD discussed earlier in this chapter, were asked to return to research after functioning as managers of extramurally funded projects.
|
From page 95... ...
The committee recommends that ORD increase sabbatical assignmeets for ORD researchers to gain experience in other scientific organizations, and that ORD bring more scientists from universities, other government agencies, and private organizations to ORD laboratories and centers for visiting appointments. Recruitment The long-term success of EPA's research and development program depends on a staff of well-trained, creative scientists, engineers, and other professionals.
|
From page 96... ...
in 1999, ORD received 1,061 applications for 50 available positrons. ORD placed 47 applicants in the laboratories, including 15 at the Effects Laboratory, 21
|
From page 97... ...
Research Adanagemenf of SPA 97 at the Exposure Laboratory, and 11 at the Risk Management Laboratory; and 3 went to ORD's Assessment Center. The 50 included 28 women and 10 minority fellows.
|
From page 98... ...
Research managers in ORD should be selected on the basis of scientific competence and the personnel skills needed to lead and nurture professional development of their staff scientists and eng~neers. Our committee's vision for the future of EPA's research program requires leaders who have technical competence; managerial abilities; communication skills; knowledge and skills in research planning and administration and in the public decision process, including its political dimensions; and the ability to marshal constituencies for an effective research program.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.