National Academies Press: OpenBook

Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories (2014)

Chapter: Appendix D: Relevant Findings and Recommendations From Previous Reports

« Previous: Appendix C: Presenters at the Committee's Information-Gathering Sessions
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

Appendix D

Relevant Findings and Recommendations From Previous Reports

This appendix presents relevant findings and recommendations from past reports of EPA’s Board of Scientific Counselors, EPA’s Science Advisory Board, National Research Council, and U.S. Government Accountability Office. In carrying out its study, the committee sought to build upon the results of these past reports as well as others cited in the chapters of its report.

1. Strengthening Science at the U.S. Environmental Protection Agency: Research-Management and Peer-Review Practices (NRC 2000)

Recommendation: The committee concurs with the recommendations of the 1997 report of its companion committee — the Committee on Research Opportunities and Priorities for EPA — that ORD should maintain approximately an even balance between core research and problem-driven research (p. 138).

Recommendation: The committee recommends that ORD place greater emphasis on maintaining awareness of research conducted by other organizations. ORD should develop and implement a proactive, structured, and visible strategy for stimulating, acquiring, and applying the results of research conducted or sponsored by other federal and state agencies, universities, and industry, both in this country and abroad (p. 139).

Recommendation: The committee recommends that ORD continue and expand its multiyear research planning approaches in both problem-driven and core research areas (p. 137).

Recommendation: The committee recommends that the National Center for Environmental Research, in concert with ORD’s national laboratories, develop additional mechanisms to promote and facilitate research interactions among STAR grantees and ORD research staff (pp. 140-141).

Recommendation: The committee recommends that ORD substantially improve the documentation and transparency of its decision-making processes for setting research and technical-assistance priorities, making intramural and extramural assignments, and allocating funds (p. 143).

Recommendation: The committee recommends that the administrator direct the deputy administrator for science and technology to expand upon the agency’s recently initiated science inventory by conducting, documenting, and publishing a more comprehensive and detailed inventory of all scientific activities conducted by agency units outside ORD. The results of the inventory should be used to ensure that such activities are properly coordinated through the agency-wide science-planning and budgeting process and are appropriately peer reviewed (p. 144).

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

Recommendation: The committee recommends that EPA change its peer-review policy to more strictly separate the management of the development of a work product from the management of the peer review of that work product, thereby ensuring greater independence of peer reviews from the control of program managers, or the potential appearance of control by program managers, throughout the agency (p. 145).

Recommendation: The committee recommends that the numbers and skill mix of the staff of ORD’s National Center for Environmental Research be reassessed to ensure they are consistent with the needs of the current program of research grants, centers, and fellowships (p. 140).

Recommendation: The committee recommends that EPA substantially increase its efforts to disseminate actively ORD’s research products and ongoing projects, to explain their significance, and to assist others inside and outside the agency in applying them (p. 141).

Recommendation: The committee recommends the establishment of a new position at EPA: deputy administrator for science and technology. This position would require authorization from Congress, appointment by the President, and confirmation by the Senate (p. 130).

Recommendation: To foster greater continuity in the management of EPA’s research program, the committee recommends that the position of assistant administrator for ORD be converted to a statutory term appointment of 6 years (p. 132).

Recommendation: The committee recommends that ORD make a concerted effort to give its research managers a high degree of flexibility and accountability. They should be empowered to make decisions at the lowest appropriate management level consistent with EPA policy an ORD’s strategic goals and budget priorities (p. 133).

Recommendation: The committee recommends that ORD continue to place high priority on its graduate fellowship and postdoctoral programs (p. 134).

Recommendation: The committee recommends that ORD create the equivalent of endowed academic research chairs in the national laboratories (p. 135).

2. The Measure of STAR: Review of the U.S. Environmental Protection Agency’s Science to Achieve Results (STAR) Research Grants Program (NRC 2003)

Recommendation: EPA should continue its efforts to attract “the best and the brightest” researchers to compete for STAR funding (p.143).

Recommendation: The committee recommends that STAR and ORD continue to work to produce state-of-the-science and research-synthesis documents. These are important for identifying critical information gaps and communicating the state of knowledge on a particular issue to the many users and audiences interested in this information (p. 142).

Recommendation: Given the nation’s continuing need for highly qualified scientists and engineers in environmental research and management, the STAR fellowship program should be continued and funded (p. 143).

Recommendation: STAR program funding should be maintained at 15-20% of the overall ORD budget, even in budget-constrained times. However, budget planners should clearly recognize the constraints of not having inflation escalators to maintain the level of effort of the entire program (p. 143).

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

3. Evaluating Research Efficiency in the U.S. Environmental Protection Agency (NRC 2008)

Finding: The key to research efficiency is good planning and implementation. EPA and its Office of Research and Development (ORD) have a sound strategic planning architecture that provides a multi-year basis for the annual assessment of progress and milestones for evaluating research programs, including their efficiency (p. 59).

Finding: All the metrics examined by the committee that have been proposed by or accepted by OMB to evaluate the efficiency of federal research programs have been based on the inputs and outputs of research management processes, not on their outcomes (p. 59).

Finding: Ultimate-outcome-based efficiency metrics are neither achievable nor valid for this purpose (p. 59).

Finding: EPA’s difficulties in complying with PART questions about efficiency (questions 3.4 and 4.32) have grown out of inappropriate OMB requirements for outcome-based efficiency metrics (p. 59).

Finding: An “ineffective” (OMB 2007a) 3 PART rating of a research program can have serious adverse consequences for the program or the agency (p. 59).

Principle: The process efficiency of research should not be evaluated using outcome-based metrics (p. 61).

Principle: The efficiency of R&D programs can be evaluated on the basis of two metrics: investment efficiency and process efficiency (p. 62).

Recommendation 1: To comply with PART, EPA and other agencies should only apply quantitative efficiency metrics to measure the process efficiency of research programs. Process efficiency can be measured in terms of inputs, outputs, and some intermediate outcomes but not in terms of ultimate outcomes (p. 65).

Finding: The most effective mechanism for evaluating the investment efficiency of R&D programs is an expert-review panel, as recommended in earlier reports of the Committee on Science, Engineering, and Public Policy and the Board on Environmental Studies and Toxicology. Expert-review panels are much broader than scientific peer-review panels (p.60).

Principle: Investment efficiency is best evaluated by expert-review panels that use primarily qualitative measures tied to long-term plans (p. 63).

Principle: Process efficiency, which may be evaluated by using both expert review and quantitative metrics, should be treated as a minor component of research evaluation (p. 64).

Recommendation 2: EPA and other agencies should use expert-review panels to evaluate the investment efficiency of research programs. The process should begin by evaluating the relevance, quality, and performance of the research (p. 66).

Finding: Among the metrics proposed to measure process efficiency, several can be recommended for wider use by agencies (see recommendation 1) (p. 60).

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

Principle: Despite the wide variability of research activities among agencies, all agencies should evaluate their research efforts according to the same criteria: relevance, quality, and performance (p. 60).

Recommendation 3: The efficiency of research programs at EPA should be evaluated according to the same overall standards used at other agencies (p. 67).

4. SAB Comments on EPA’s Immediate Science Needs (EPASAB 2009, pp. 2-3)

Finding: EPA has long worked with other organizations on environmental issues.

Recommendation: EPA should develop more robust partnerships and innovative approaches to supporting cutting-edge research and development, both domestically and internationally.

Finding: Human health and environmental problems are interrelated, are often associated with multiple stressors, and often involve exposures from more than one medium.

Recommendation: EPA should increase its efforts to address issue evaluation and management in an integrated manner that recognizes the complexity of the world in which the problems occur.

Finding: EPA has already begun to design a program to conduct integrated multidisciplinary research on complex environmental issues.

Recommendation: EPA should support new research frameworks to overcome barriers that now limit development of knowledge of integrated environmental problems and their solutions.

Finding: EPA must commit to establishing a research base that will make it possible for the nation to acquire the knowledge needed to address the difficult environmental problems that we now face and which will only grow in complexity and magnitude in the future

Recommendation: EPA should move to restore the budget for research and development in order to maintain the U.S. as an international leader in environmental protection.

Finding: Decision, Behavioral and Social Sciences are critical to framing, designing and implementing EPA decision processes and to the effective and credible resolution of environmental problems.

Recommendation: Research and operational capacity in the social sciences should be augmented.

Finding: Energy and climate change issues stand out in their importance to the nation’s and the world’s well-being.

Recommendation: EPA should take the lead in assessing the environmental and health implications of energy and climate change policies.

5. Computational Toxicology Review (EPABOSC 2010)

Finding: The BOSC members believe that the CTRP has made substantial progress toward meeting the original long-term goals, and that the progress is appropriate given the duration of the Program’s existence and the resources involved (p. 2).

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

Finding: One of the challenges that the CTRP has taken on is the assembly and integration of the vast quantities of existing, available toxicological and toxicogenomics data (p.2).

Recommendation: These projects need to continue to build on things that are in place, drilling deeper into the data and continuing the problem of structuring, standardizing, and organizing the data so that they can be more easily subjected to comprehensive meta-analyses (p.3).

Recommendation: Several CTRP projects have undertaken structuring, standardizing, and organizing the data so that they can be more easily subjected to comprehensive meta-analyses. At this point, the CTRP should obtain some public feedback on how people are using and interpreting the available data (p. 5).

Finding: A major part of the modeling effort focuses on interrogating the databases. The BOSC noted that a substantial part of these efforts utilizes machine-learning methods (p. 4).

Recommendation: BOSC encourages the Program to consult with biostatisticians early and often to assure they can address any objections [to machine learning]; the CTRP also should consider attempting some additional methods (p. 4).

Finding: There also is a need to interact more extensively with the broader scientific user community in the process of developing and rolling out tools and software (p. 4).

Recommendation: This could be achieved through an annual or biannual conference by bringing together the data generators, the data users, and the risk assessors/managers—the ultimate users of these alternative methods/models (p.4).

Recommendation: Acceptance of products, methods, and databases by the risk assessment community is the key to success. Hence, the NCCT should organize an annual or biannual conference that brings together the data generators, data users, and risk assessors/managers—the ultimate users of these alternative methods/models (p. 5).

Finding: There were concerns expressed by some BOSC members that associations are not causation and this should be recognized by the EPA management, both at the CTRP level and at the level of the Office of the Administrator (p. 4).

Recommendation: The results of a computer-generated association should be carefully examined through traditional testing and careful scientific study (p. 4).

Recommendation: As more data from high-throughput assays and computer models become available, the NCCT should provide guidance on how to interpret this information in the context of more traditional testing and scientific examination so that risk assessment practitioners in the EPA program offices can apply these findings (p. 5).

Recommendation: Continue to interact with other scientific bodies, regulatory agencies, and universities both in the United States and globally so as to insure that work conducted elsewhere can be “built upon.” In addition, it is recommended that the group interact with the toxicology groups within pharmaceutical and major chemical companies (p. 7).

Recommendation: Routinely (perhaps biannually) sponsor some sort of exchange of information with risk assessment practitioners both inside and outside EPA (corporations, consultants, and government scientists) to be sure that the end products of the Program’s work are both reliable and of use to the future users (p. 7).

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

Recommendation: For the next BOSC review, develop a table that presents the level of effort dedicated to specific projects, by year. This table would contain the number of CTRP FTEs, as well as the approximate level of “collaborative” effort (from other EPA laboratories and other partners and consultants). In kind support and “hard” dollars also should be presented (p.7).

Recommendation: Keep the statisticians and mathematical modelers involved in assay evaluation so that they can move from qualitative prediction to quantitative prediction of outcomes from exposure data (p. 8).

Recommendation: Conduct an unbiased evaluation of the usefulness of particular assays to achieve prediction beyond a single class of compounds and to define knowledge gaps for new assay design (p.9).

Recommendation: Develop case studies that demonstrate a strategy for incorporation of CTRP tools/research into the risk assessment process (p. 9).

Recommendation: Be more integrative, both internally and externally, to ensure all parties are working from common assumptions, data development schedules, and deliverable planning (p. 10).

Recommendation: Expand outreach to the broader community, both within EPA and in the extramural community. This is not to say that the CTRP has not been effective in building a strong outreach program, but only that this needs to be a priority, and possibly a higher priority (p. 13).

Recommendation: Detail specific roles for the STAR Centers as part of the integrated approach to managing the Program’s mission (p. 13).

Recommendation: Place a higher priority on incorporation of ecological receptors and greater focus on assessment of exposure factors (p. 13).

Recommendation: Develop a forward, longer term plan to incorporate the field of ecological risk assessment as part of the CTRP (p. 13).

Recommendation: Expand the ExpoCast program to include real exposure and outcomes data, as well as the additional development of software resources to take advantage of these data for exposure and outcome predictions. This should be a priority of the Center (p. 13).

Recommendation: Continue training postdoctoral fellows because these scientists have the potential to be ambassadors to the rest of the community to help extend the understanding and acceptance of the types of computational tools the CTRP is trying to develop, and in doing so, ultimately help to improve those tools and their efficacy (p. 13).

Recommendation: Highlight quality assurance for software and models with a specific testing approach augmented with a sophisticated evaluation approach that probes how the systems produced work in the hands of users (p. 13).

Recommendation: Promote “user-centered design”, an approach that grounds the process of design in information about the people who will use the product (p. 13).

Recommendation: Establish performance metrics that track the development of tools and resources for informing chemical prioritization, toxicity testing, and risk assessment (p. 15).

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

Recommendation: Continue to meet with customers, clients, and stakeholders on a regular basis to ensure that the Program is meeting the needs of the risk assessors and risk managers in the Agency (p. 15).

6. ORD Strategic Research Directions and Integrated Transdisciplinary Research (EPASAB 2010)

Finding: ORD’s research direction largely misses strategic opportunities related to social and behavioral sciences. It also misses the opportunity to improve ORD research programs by incorporating social and behavioral sciences (p.5).

Recommendation: EPA needs to reorient its research agenda to recognize that many environmental threats stem from the actions, decisions, and behaviors of individual Americans (p.6).

Finding: Due to the nature of the challenges and scientific capacity within EPA, there is strong justification for EPA to provide leadership in establishing multi-agency partnerships that leverage resources and provide comprehensive solutions (p. 6).

Finding: There is no systematic communication between ORD and states regarding research needs (p.3).

Recommendation: A more systematic process is needed for states and tribes to identify, organize, prioritize and communicate their immediate and anticipated requirements for science support into the ORD research planning and implementation process (p. 3).

Finding: ORD’s management structure currently provides the ORD Executive Committee and Laboratory Directors with primary control of resources, while research planning is the responsibility of National Program Directors (p. 5).

Recommendation: Integrated transdisciplinary research requires alignment of research resources with Agency priority needs and is more likely to succeed with true matrix management that recognizes those priorities and addresses resource allocation decisions (p. 5).

Finding: ORD demonstrated linkages between ORD research contributions and EPA accomplishments under the key priorities (p. 3).

Recommendation: We recommend that EPA make these linkages when planning future research programs (p. 3).

Finding: It will be essential for EPA as a whole, and not just ORD alone, to adopt a systems approach to research planning (p. 3).

Recommendation: A systems approach that incorporated human health concerns into global change analysis could be used to break down artificial barriers between human health and ecological assessment (p.4).

Recommendation: Systems approaches, if applied to air research or to ORD’s “one hydrosphere” vision, could help EPA better understand the root causes of environmental problems that may be related to energy usage, transportation, and local planning and zoning (p. 4).

Finding: Planning and conducting a systems-based and integrated transdisciplinary research program requires mechanisms to encourage scientists to think outside their traditional disciplines or re-

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

search programs, to seek connections and questions that cross research programs and media, and to look for “systems effects” related to a research question (p.5).

Recommendation: We recommend that ORD consider and implement as soon as possible strategies to 1) encourage systems approaches to research and, 2) support and provide leadership for integrated transdisciplinary research teams (pp. 4-5).

7. The Use of Title 42 Authority at the U.S. Environmental Protection Agency: A Letter Report (NRC 2010, pp 23-24)

On the basis of its evaluation and review, the committee offers the following findings and recommendations:

The committee agrees with previous expert panels and committees that a science and engineering workforce that is capable of performing and conducting research at the highest level is essential for EPA to protect public health and the environment.

On the basis of the committee’s review of ST, SL, and SES positions, the committee concludes that no other hiring mechanisms or authorities available to EPA serve the function of Title 42 to recruit and retain world-class scientists and engineers.

The selection of particular research fields that would benefit most from Title 42 appointments is of paramount importance. The committee recommends that ORD focus its Title 42 appointees in fields deemed most critical by its research priority-setting process.

The committee notes that the number of Title 42 appointments is not limited at NIH and CDC, other federal agencies that fill scientific positions using Title 42 authority. The numbers of Title 42 appointments in those agencies are substantially larger than at EPA.

All world-class scientists and engineers do not necessarily have doctoral-level degrees, and EPA should be flexible in its requirement that all Title 42 appointees have such degrees.

EPA has approached the use of Title 42 authority prudently. For example, a position was not filled when highly qualified candidates could not be identified, and EPA has not awarded the maximum compensation allowed under Title 42 to appointees. The committee concurs with EPA’s approach.

In developing its Title 42 program, EPA has used various techniques to recruit candidates. To identify the most qualified candidate, the committee recommends that EPA adhere to the following procedure: (1) establish a search committee to oversee recruitment, promote diversity in the process, evaluate applicants’ credentials, and recommend the most qualified applicants to a selection committee; (2) advertise widely on appropriate Web sites, in appropriate journals, through scientific and engineering societies, and by contacting highly competent people in the relevant disciplines; and (3) form a selection committee to determine the best candidate and forward the recommendation to ORD management, ultimately the ORD AA or designee, for approval. Both search and selection committees should include members who are outside EPA. The entire search and selection process should be as open as feasible to ensure that the best practices are followed, that a broad and diverse search has reached the most qualified potential candidates, and that fairness prevails.

The Title 42 program at EPA is small and still evolving, but it has worked well. Outstanding candidates have been identified and hired, and top scientists have been retained. Furthermore, the BOSC

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

and EPA indicate that the Title 42 program has helped the agency to achieve its mission. For example, the NCCT has, in its few years of existence, conducted important research and made substantial progress in developing new tools based on advances in molecular biology and genomics.

The committee recommends that permanent Title 42 authority be granted to EPA.

The committee recommends that EPA use the BOSC or the SAB to review the Title 42 program every 5 years to ensure that it is being used for the intended purposes of creating a critical mass of world class scientists and engineers, that Title 42 hires are in the fields identified as having the highest priority by the agency, and that it is implemented in a manner that ensures selection of the best candidates.

The committee recommends that EPA be granted expanded authority to define the number of Title 42 positions on the basis of its programmatic needs and available budget.

8. To Better Fulfill Its Mission, EPA Needs a More Coordinated Approach to Managing Its Laboratories (GAO 2011, pp. 27-28)

EPA labs: function and capabilities for meeting current needs

Recommendation: Improve physical infrastructure and real property planning and investment decisions by:

managing individual laboratory facilities as part of an interrelated portfolio of facilities;

ensuring that master plans are up-to-date and that analysis of the use of space is based on objective benchmarks; and

improving the completeness and reliability of operating-cost and other data needed to manage its real property and report to external parties.

EPA Labs Management Process

Recommendation: Develop an overarching issue-based planning process that reflects the collective goals, objectives, and priorities of the laboratories’ scientific activities.

Recommendation: Establish a top-level science official with the authority and responsibility to coordinate, oversee, and make management decisions regarding major scientific activities throughout the agency, including the work of all program, regional, and ORD laboratories.

Recommendation: If EPA determines another independent study is needed, the agency should include alternative approaches for organizing the laboratories’ workforce and infrastructure, including options for sharing and consolidation.

EPA Labs Ability to Meet Future Challenges

Recommendation: Develop a comprehensive workforce planning process for all laboratories that is based on reliable workforce data and reflects current and future agency needs in overall number of federal and contract employees, skills, and deployment across all laboratory facilities.

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

9. ORD New Strategic Research Direction (EPASAB/BOSC 2011)

EPA science needs (challenges); future

Recommendation: One area where ORD can increase its capacity to address future critical environmental issues involves the exploration of opportunities offered by computational analysis and modeling of complex environmental data (p.10).

Finding: Sustainability goals and all the systems of interest to EPA include human behavior (p. 12).

Recommendation: Increased emphasis on social, behavioral and decision sciences within ORD is needed for the new research programs to be successful. Social science research should be integrated in all of the programs in explicit ways. (p. 12).

EPA labs: function and capabilities for meeting current needs

Recommendation: It would be helpful for all research frameworks to include a list of definitions of key sustainability terms that would be consistent across ORD’s programs (p. 5).

Recommendation: SAB and BOSC recommend that ORD revise each [of the six] research frameworks to include sustainability explicitly in its research vision, invoke a definition of sustainability shared across ORD, and demonstrate clearly how planned research relates to the key components of sustainability (the environment, the economy, and society) (p. 6).

Extramural Research funded by EPA

Recommendation: ORD should set defined goals to catalyze and complement environmental science programs outside EPA and seek BOSC review and assessment related to this topic every two years. (p.10)

Recommendation: Innovation could be enhanced by emphasizing innovation in EPA’s extramural grant programs and by making EPA data easily accessible to the outside community of scientists who could use these data in creative ways (p. 11).

Recommendation: The SAB and BOSC recommend that ORD explore mechanisms for industry-government collaboration (p. 27).

Recommendation: ORD should evaluate existing mechanisms for inter-agency collaboration and build on them to maximize the potential to catalyze and complement environmental science programs outside EPA (p. 23).

Recommendation: ORD should explore new opportunities to partner with the National Science Foundation to support extramural research in [the social science] area, such as the Foundation’s Sustainability Research Networks Competition (SRN) and its Dynamics of Coupled Natural and Human Systems (CNH) program (p. 17).

Recommendation: ORD should consider programs to sponsor senior academic researchers for one-year visiting sabbaticals to seek their suggestions about how to transform the Air, Climate and Energy program into a program fully integrating sustainability (p. 21).

Recommendation: The Homeland Security model of coordination within and outside the EPA can be a model for other research programs (p. 30).

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

EPA Labs Management Process

Finding: The increase in the amount of communication among ORD’s National Program Directors and Directors of Laboratories and Centers in the development of ORD’s research frameworks is readily apparent and very positive (p. 4).

Recommendation: ORD should seek to expand formal mechanisms to promote networking among internal researchers to improve research coordination throughout the research process in the least time-intensive manner (pp. 4-5).

Recommendation: ORD should identify priority cross-program research topics such as nitrogen and climate as vehicles for research coordination and building of interdisciplinary culture (p. 5).

Recommendation: Regarding prioritizing programs for increased or decreased emphasis, the SAB and BOSC recommend that ORD conduct analyses to help develop criteria for prioritization (p. 26).

EPA Labs Ability to Meet Future Challenges

Finding: ORD’s involvement of stakeholders in EPA program and regional office and other federal partners in research planning provides a good mechanism to identify environmental issues and prioritize among them (p. 9).

Recommendation: It may be helpful for ORD to form an internal committee of cross-program futurists, with representatives from each research program to identify emerging issues and to consult regularly with the SAB, BOSC and other EPA groups and external stakeholders (p. 9).

Finding: The EPA has thought seriously and operationally about ways of energizing the creative nature of ORD scientists and has begun to explore ways of enhancing innovation as a fundamental part of ORD programs (p. 11).

Recommendation: ORD should develop metrics to evaluate the contributions of the Chief Innovation Officer and programs such as Pathfinder. ORD should define “failure” and “success” as it further develops its innovation program and reach agreement on an acceptable failure rate for innovation efforts (p. 11).

Recommendation: The SAB and BOSC recommend that ORD undertake research to define the benefits of moving from a more technology-based regulatory system to a performance-based regulatory system that provides incentives for sustainable solutions (p. 21).

EPA Labs Leadership in Environmental Research and Other Science-Related Activities

Recommendation: ORD should continuously stimulate interactions between EPA and outside scientists. One mechanism could involve a program of roundtables with outside experts. Visiting scientists could be brought into the laboratories and centers for longer periods (e.g., one year) to cross-fertilize ideas on how to make sustainability an organizing principle at EPA (p.10).

10. Science Integration for Decision Making at the U.S. Environmental Protection Agency (EPASAB 2012)

Finding 1a: Over 6,000 EPA employees are involved in scientific assessments, research, and related activities, with approximately 1,300 full-time scientific staff in the Office of Research and Development (ORD) and approximately 4,700 full-time scientific staff in program and regional offices (p.2).

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

Finding 1b: An overarching barrier to consistent science integration is the lack of strong, coordinated management at EPA to support the scientists in the regional and program offices. Currently, the EPA does not have a single entity responsible for managing and strengthening EPA’s scientific workforce so that it functions as a resource for the agency as a whole (p.8).

Finding 1c: The EPA has not developed a coordinated human resource strategy for building this science base within ORD and beyond. Effective science integration requires the recruitment, retention, and development of leading scientists from many fields across EPA programs and regions, as well as in ORD (p. 8).

Recommendation 1: The EPA should increase and improve support and training for scientists and managers across the agency, especially in programs and regions, to strengthen capacity for science integration. Traditional rewards and recognition for scientific excellence focus on discovery, peer reviewed publication, and national and international recognition by peers. As a result there are few professional incentives for scientists to focus on support of regulatory decision making. The SAB recommends that scientists throughout the agency be encouraged to participate actively in developing improved approaches to integrate science into agency decisions and be rewarded for their valuable contributions (p. 11).

Finding 2a: Science integration practices vary across the agency. Some managers actively promote science integration, but more could be done in most program and regional offices. Time and resource constraints are important barriers to science integration across the EPA, but notably some leaders and managers make science integration a priority. The need for improving science integration is most acute in the regions and program offices on the front line for addressing environmental issues. Currently, the EPA does not have a single entity responsible for managing and strengthening the EPA’s scientific workforce so that it functions as a resource for the agency as a whole (pp.7-8).

Recommendation 2: Managers should be engaged in and accountable for integrating science into decision making, starting with problem formulation and science assessment, in their own organizations and across the EPA. The SAB recommends that EPA managers consistently devote attention to implementing all the components of science integration. Management should be accountable for problem formulation to martial integrated thinking about complex environmental problems as they occur in the real world (p.10).

Finding 3a: Science integration is an integral component of many decisions at EPA. The SAB interviews confirmed that agency staff and managers view science as an important component of decision making at the EPA, whether decisions involve regulatory, enforcement or voluntary programs. Science Integration is a three-part process:

  1. 1) problem formulation – asking the right questions;
  2. 2) assessment – combining information and analyses from different scientific fields to address the problem; and
  3. 3) decision making and evaluation – application of the science and ongoing evaluation of the outcome of the decision (pp. 3-4).

Finding 3b: No EPA program has fully implemented all the steps of science integration. The SAB envisions a framework for science integration with three major components: problem formulation; analysis and decision making; and implementation and performance evaluation. The first step, problem formulation, may be the most important. Problem formulation is a systematic planning step, linked to the regulatory and policy context of an environmental problem, which identifies the major factors to be considered, developed through interactions among policy makers, scientists and stake-

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

holders. The analysis and decision-making step often includes the assessment of existing science (p.4).

Finding 3c: Regulatory program and disciplinary “silos” remain significant barriers to science integration. Narrow interpretations of legislative mandates and the organizational structure of the EPA’s regulatory programs often have posed barriers to innovation and cross-program problem solving. Rigidity within scientific disciplines also can pose an obstacle to science integration. Interdisciplinary work is difficult; experts often use different terminology and methodologies. These differences can become intellectual silos when the science integration is not formally facilitated (p. 5).

Finding 3d: There is a critical need for more high quality assessments translating existing science on a broad range of topics important to decision making at the EPA. Regional and program offices emphasized the importance of science assessments that evaluate the state of existing science. However, interviewees noted that scientific literature reviews published in peer-reviewed journals generally do not provide assessment information that meets the EPA’s regulatory needs. The EPA has a continuing need to develop capacity for trans-disciplinary scientific assessment, translation, and integration (the cover letter).

Recommendation 3: The EPA should explicitly plan for science integration to support environmental decisions. For each decision requiring scientific information, science integration will require an initial problem formulation step, with the following components:

  1. 1) Involvement of the responsible decision-maker to define the initial questions that will look broadly at the physical, economic, and social context of specific environmental problems;
  2. 2) Identification of options for intervention and risk management;
  3. 3) An assessment plan that discusses the appropriate level and types of science required for the decision;
  4. 4) Expectations regarding the required timeline and resources; and
  5. 5) An appropriate balance of public and stakeholder engagement (p.9).

11. Implementation of ORD Strategic Research Plans (EPASAB/BOSC 2012)

EPA science needs (challenges): current

Sustainability (p. 4)

Recommendation: Each ORD program should define more specifically what sustainability means within the program context, and identify how each plan incorporates ecological and human health into the definition of sustainability.

Recommendation: ORD should collaborate with other partners in the EPA, including the National Center for Environmental Economics, to develop a plan to develop the social, behavioral and decision science needed to support sustainability research and other goals identified in ORD’s six major research programs. A useful first step would be for ORD to plan a workshop on this topic and seek SAB and BOSC advice in workshop planning.

Extramural Research funded by EPA (p. 9)

Recommendation: ORD should use solicit and support innovation research projects in communities and utilities across the country.

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

EPA Labs Management Process

Recommendation: ORD should consider including a more detailed timeline with deliverables for planned activities for each research program with specific milestones and/or intermediate deliverables (p. 3).

Recommendation: ORD should develop individual “roadmaps” with goals and an outline of paths to those goals for each of the integrated research topics, similar to the roadmap being developed for ORD’s nitrogen topic (p. 6).

Recommendation: ORD should develop a graphical framework for each integrated research topic that identifies and discusses the responsibilities and relationships of the various participating EPA programs and external agencies and groups (p. 6).

[see also individual ORD program recommendations below]

EPA Labs Ability to Meet Future Challenges

Recommendation: In future action plans, ORD should provide a comprehensive mapping of projects to goals, and not just provide examples [emphasis added] (p. 3).

Recommendation: ORD should develop a structured approach (e.g., through a risk portfolio or decision science-based analysis) to assess the relative priorities of emerging issues vis à vis existing and legacy research activities (p. 5).

Recommendation: ORD should make training and development for ORD staff a priority and seek new ways to interact with scientists outside the EPA through partnerships with other agencies and academic institutions to keep staff on the frontier of science and alert to emerging issues (p. 5).

Recommendation: ORD should strive wherever possible to craft its research such that it fulfills the dual goals of meeting specific programmatic goals while also maintaining and expanding the agency’s core capabilities in critical research areas (p. 5).

Innovation (p. 9)

Recommendation: When assessing potential innovation projects and impacts of innovation projects, ORD should consider multiple benefits of such projects, and identify and focus its metrics on the goals of the EPA’s organizations and their specific need rather than on conventional business performance metrics.

Recommendation: Innovative activities and support of those activities should be prioritized to reflect the EPA’s most pressing needs.

Recommendation: ORD should provide more information on the guiding principles that govern how Pathfinder Innovation Projects grants are awarded and how questions for challenges are chosen.

Recommendation: ORD should undertake additional efforts to identify and leverage the top innovators via mentoring of others and/or assembling the top innovators in small teams to promote further breakthroughs.

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

Recommendation: ORD should provide as much encouragement for social and sociotechnical innovations as for purely technological ones.

Recommendation: ORD should develop an award system that would align with the desired behavioral changes in moving the ORD culture to one of innovation.

Recommendation: ORD should sponsor a focused workshop on metric development for innovation that would result in a set of metrics that represents a reasonable fit with the ORD mission and desire for innovation.

Air, Climate, and Energy Program (p.12)

Recommendation: ORD should more explicitly map the long list of individual projects and project outputs in the Strategic Research Action Plans to strategic research themes and the overarching vision.

Recommendation: The Strategic Research Action Plan should include a plan for energy research and indicate how this research will integrate with the plans for climate and air quality research.

Recommendation: To support this additional systems-level focus on energy, ORD should identify senior leadership to provide necessary systems science expertise and ensure that the connections between energy research projects are drawn and made explicit.

Recommendation: The Strategic Research Action plan should include a description of how ORD’s ACE activities are positioned within the portfolio of other research activities at the EPA and the research of other federal agencies.

Recommendation: The Strategic Research Action Plan needs more comprehensive and greater depth in planned social science and behavioral research.

Chemical Safety for Sustainability Program (p. 17)

Recommendation: Clearly demonstrate how CSS research impacts upon end users (e.g., risk managers, policy makers) and how it brings value for informing decisions.

Recommendation: Increase focus on the refinement and verification of proximal and consumer exposure models, including both external and internal dosimetry.

Recommendation: In the effort to transition toward EDSP21, place greater attention on the challenges involved in using reductionist approaches (e.g., ToxCast) in evaluating highly integrated physiological networks, such as the endocrine system.

Recommendation: Frame the research on EDSP21 as a precedent for addressing analogous challenges for evaluating other complex integrated biological systems (e.g., nervous system).

Recommendation: Define ORD’s unique niche within the broader landscape of nanotechnology research.

Recommendation: Clearly and transparently describe the proposed approach for verification of new computational toxicology tools for their intended purpose and with respect to risk assessment, and present to BOSC for review.

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

Recommendation: Define the typical range of intra- and inter-individual variation in biological control pathways in order to distinguish between adaptive vs. adverse changes. Address how the program will dovetail with higher tier targeted testing.

Recommendation: Place greater emphasis on integration of toxicokinetics (ADME) and physiologically-based pharmacokinetic models.

Human Health Risk Assessment Program (pp. 25-26)

Recommendation: The EPA should broadly examine the diverse venues where risk assessment activities reside within the agency and seek to establish connections and integration that will foster ongoing enhancement of methodologies that are common to risk practitioners throughout the Agency.

Recommendation: ORD leadership should elaborate a strategic vision that enhances linkages among the thematic areas of the HHRA and with the other research programs, particularly the CSS program, and that emphasizes the way that the HHRA program contributes to sustainability research. This vision will be needed for revising the HHRA strategic plan.

Recommendation: A wide- reaching plan is needed for incorporating data from emerging technologies, e.g., “omics” and high throughput testing, into EPA risk assessment approaches and for evaluating the utility of these data for decision-making. This activity needs emphasis in Theme 4.

Recommendation: While progress by HHRA has been on pace during its first year, the agenda needs to be set for the longer-term with priorities given to the most critical topics for decision-making, particularly as resources may decline.

Recommendation: Exposure sciences need greater emphasis within the activities of the HHRA and further expertise is needed in this cross-cutting area.

Recommendation: The addition of further social, behavioral, and decision scientists to HHRA would benefit many of its activities and enhance integration with other programs. This recommendation echoes prior reports and speaks to the broad, multidisciplinary nature of decision-making and communication with regard to risk in the face of uncertainty. Long-standing gaps in expertise within the Agency should be addressed.

Recommendation: Concerted and sustained efforts are needed to assure that scientists with HHRA and elsewhere in EPA and decision-makers are fully versed in the latest risk assessment approaches and the interpretation and application of their findings.

Recommendation: EPA risk managers should also be educated about new data and approaches to risk assessment, leading to greater confidence in decisions based on these approaches. They need to be kept aware of advances made under Theme 4.

Recommendation: Peer reviews of HHRA documents and assessments could be made more efficient. The plans for changes in the IRIS assessments should benefit the peer review process. Additionally, the intensity of peer review should reflect the complexity and importance of the product. For extensive peer reviews, it is important to evaluate and improve the process to triage comments so that effort is directed at the points of criticism that are most important and that have significant implications for overall risk estimates and decision-making. This may be facilitated by an independent “monitor” or “editor.”

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

Safe and Sustainable Water Resources Program (p. 29)

Recommendation: ORD should include specific tasks and milestones in the SSWR Strategic Research Action Plan.

Recommendation: The SSWR program should further clarify what is the agency’s focus vs. the focus of other agencies regarding SSWR sustainability-related research.

Recommendation: The SSWR program should develop a structured way to assess emerging issues in establishing priorities.

Recommendation: The SSWR program should consider the magnitude and distribution of risks associated with not pursuing emerging SSWR research issues that could benefit certain communities such as environmental justice communities.

Recommendation: ORD should transparently communicate its efforts to prioritize research and conduct outreach and actively engage with communities when developing SSWR research priorities.

Recommendation: EPA should invest more in assessing use of market mechanisms for nutrient control, and identify metrics for nutrient management.

Recommendation: The SSWR program should be engaged with and knowledgeable about research on mechanisms and forms of nutrient delivery in agriculture.

Recommendation: ORD should identify and seek opportunities for leveraging research related to nutrients with other federal agencies and utilize ORD’s strengths in areas such as monitoring, data analysis, and modeling within such leveraged efforts.

Recommendation: ORD should assess and encourage opportunities for innovation in nutrient research.

Recommendation: The SSWR program should take a leadership role in conducting green infrastructure research and incorporate natural infrastructure into its research.

Recommendation: The SSWR program should inventory best practices and innovation activities, and seek partnership opportunities to assess lessons learned related to green infrastructure.

Recommendation: The SSWR program should develop tools to encourage/improve how states help communities address Combined Sewer Overflow consent order requirements.

Recommendation: ORD should support competitions that solicit innovation in storm water monitoring and modeling.

Homeland Security Program (p. 32)

Recommendation: ORD should develop metrics for measuring progress and success at project conception.

Recommendation: The HSRP should document its impact by identifying the multiple benefits of its products. It should concurrently expand its communication about the broad applicability and many benefits of HSRP products and expertise; outline the value proposition to stake-holders; and market HSRP expertise to additional partners to increase resource leveraging.

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

Recommendation: The HSRP, as a valuable national resource, should adopt an “all-hazards” approach to enhance its value. Current products should be assessed and mapped to the needs of potential new partners. HSRP is strongly encouraged to conduct research portfolio analysis and road mapping to elucidate their current and future research needs.

Recommendation: HRSP should continue to enhance its relationships with other federal agencies where there is synergy.

Sustainable and Healthy Communities Program (pp. 38-39)

Recommendation: Integrating ecological and human health. The SAB and the BOSC commend EPA for recognizing the importance of bringing together human health and ecosystem services. Although this integration requires considerable effort, it is an important area that is worthy of investment. Moreover, EPA is the one agency that is positioned to do this. Although the communication flow among the different experts (e.g., ecosystem scientists and, human health scientists) does not always occur at the level needed, ORD is attempting to foster these interactions. Sustained efforts to promote interaction and integration are needed. ORD should outline the barriers to this integration and think creatively about strategies to help overcome them.

Recommendation: Inclusion of social, behavioral and decision sciences. Social, behavioral and decision sciences are an essential component of the SHC program because they contribute to understanding human actions driving environmental, social and economic change, the value of ecosystem services, development of decision-support tools, the design of policies, and behavioral responses to policy changes. SHC has taken a step in the right direction but much work remains to be done. The SAB and the BOSC would like to see future efforts expanded.

Recommendation: Distinguishing research from implementation. Throughout the action plan, it was difficult to separate (a) research from implementation, and (b) client from partner from community. The SAB and the BOSC suggests that SHC articulate more clearly its plan for research and how this plan fits in terms of interacting with local communities, state environmental agencies, and regional offices, and distinguish research from implementation in the text.

Recommendation: Focusing the science questions and research. There was some concern that there were too many science questions, with most too broad in scope. The SAB and the BOSC recommend that the Strategic Research Action be edited to explain how each of these science questions will be answered given the research that will be undertaken. This task would help SHC bring its stated research objectives into sharper focus, especially in light of resource constraints. The SAB and the BOSC also recommend that, at the very least, the program should prioritize the science questions.

Recommendation: Engaging communities and building partnerships. The SAB and the BOSC commend the SHC program for engaging stakeholders in community listening sessions. However, more structured and guided methods will allow for a better understanding of community values, needs/wants, and constraints. There also remained some confusion about what SHC means by community engagement. The SHC program should clarify its view of what community engagement, participatory research, and community self-assessment mean for the program. The SHC program should draw upon the previous work in this area.

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

12. Best Practices in Assessment of Research and Development Organizations (NRC 2012a)

WHAT CONSTITUTES AN EFFECTIVE R&D ORGANIZATION? (pp. 21-22)

  • A clear and substantive mission,
  • A critical mass of assigned work,
  • A highly competent and dedicated workforce,
  • An inspired, empowered, highly qualified leadership,
  • State-of-the-art facilities and equipment,
  • An effective two-way partnership with customers,
  • A strong foundation in research,
  • Management authority and flexibility, and
  • A strong linkage to universities, industry, research institutes, and government organizations.

SOME QUESTIONS TO CONSIDER DURING ASSESSMENT (pp. 9-10)

Assessing Management

Answers to the following questions will be useful in the assessment of organizational management:

Does the organization’s management understand its mission and its relationship to that of its parent? Does the vision statement of the organization align with that of the parent organization?

Is there a long-range plan for implementing the strategy by specific technical programs?

Does the organization have an explicit strategy for its work and for securing the necessary resources?

Do the program plans reflect a model for balance—that is, amount of basic versus applied and development research, and short-, medium-, and long-term work?

Does the organization have a clear champion within the parent organization?

Does management have an aggressive recruiting plan with well-defined criteria for new hires? Is there a set of practices for retaining, promoting, and recognizing the staff?

Does the organization have a process for forecasting likely future technical developments in areas appropriate to its mission?

Does the organization’s management have discretionary authority to invest in new programs on its own initiative? Does management solicit ideas from the staff for new work?

Does management regularly assess facilities and equipment for adequacy? Does it have a fiscal plan for updating or replacing laboratory equipment?

Is there a process for regularly reviewing the organization’s research portfolio for its alignment with the mission?

What is the management climate, and how does one assess it? Is there enough flexibility to work across organizational lines?

How does the structure of the organization support its mission?

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

How much collaboration is there with outside organizations? How many staff exchanges are there?

Does management have a well-defined process and criteria for determining what work is performed inhouse versus what work is sponsored via grants, contracts, or other mechanisms with external entities?

Does the management support a culture of creativity, diversity, and entrepreneurship?

Assessing the Quality of Scientific and Technical Work

Answers to the following questions will be useful in the assessment of the quality of an organization’s technical work:

Does the assessment include the quality of the staff, equipment, and facilities?

Does the assessment include the nature of the research portfolio as to alignment with the mission and the balance in regard to basic, applied, and development work and short-, intermediate, and long-term research?

Does the organization have a set of indicators that can serve as parameters when the time frame precludes immediate assessment? Does the organization benchmark itself against premier organizations?

Who is the expected audience for the assessment?

Is the review done by technical peers?

What are the criteria for ensuring the credibility and validity of the assessment?

What is the scope of the assessment? Does it include proposals for new work? Does it include assessment of completed work—internal review and authority to release a report, publications, patents, invited lectures, awards, and the like?

Who designs and manages the assessment?

Assessing Relevance and Impact

Addressing the following questions will be useful in the assessment of an organization’s relevance and impact:

Does the organization have a process for identifying its stakeholders and customers?

Does it have a regular process for reviewing its programs and plans with its stakeholders?

Does the organization have a process for learning of its customers’ current and likely future needs and expectations for the organization?

Does the organization have an explicit process for tracking the utilization of its results (e.g., is transition to the next R&D stage actively managed and measured)?

Does it have a formal program for recording the history of its work from concept to final utility or impact?

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

Does the organization have a program to conduct retrospective studies of its earlier work?

13. Science for Environmental Protection: The Road Ahead (NRC 2012b)

Finding: EPA can maintain its global position by staying at the leading edge of science (p. 10).

Finding: Effective science-informed regulation and policy aimed at protecting human health and environmental quality rely on robust approaches to data acquisition, modeling, and knowledge development (p. 8).

Finding: Maintaining leading-edge science requires the development and application of systems-level tools and expertise for the systematic analysis of the health, environmental, social, and economic implications of individual decisions (p. 10).

Recommendation: Maintaining leading-edge science requires the development of tools and methods for synthesizing scientific information and characterizing uncertainties. It should also integrate methods for tracking and assessing the outcomes of actions (that is, for being accountable) into the decision process from the outset (p. 10).

Finding: Although EPA has periodically attempted to scan for and anticipate new scientific, technology, and policy developments, these efforts have not been systematic and sustained. The establishment of deliberate and systematic processes for anticipating human health and ecosystem challenges and new scientific and technical opportunities would allow EPA to stay at the leading edge of emerging science (p. 200).

Recommendation: The committee recommends that EPA engage in a deliberate and systematic “scanning” capability involving staff from ORD, other program offices, and the regions. Such a dedicated and sustained “futures network” (as EPA called groups with a similar function in the past), with time and modest resources, would be able to interact with other federal agencies, academe, and industry to identify emerging issues and bring the newest scientific approaches into EPA (p. 200).

Finding: EPA has recognized that innovation in environmental science, technology, and regulatory strategies will be essential if it is to continue to perform its mission in a robust and cost-effective manner. However, to date, the agency’s approach has been modest in scale and insufficiently systematic (p. 202).

Recommendation: The committee recommends that EPA develop a more systematic strategy to support innovation in science, technology, and practice (p. 202).

Finding: Environmental problems are increasingly interconnected. EPA can no longer address just one environmental hazard at a time without considering how that problem interacts with, is influenced by, and influences other aspects of the environment (p. 189).

Recommendation: The committee recommends that EPA substantially enhance the integration of systems thinking into its work and enhance its capacity to apply systems thinking to all aspects of how it approaches complex decisions (p. 189).

Finding: It is difficult to understand the overall state of the environment unless one knows what it has been in the past and how it is changing over time. Typically this can only be achieved by examining high-quality time series of key indicators of environmental quality and performance. Currently at EPA, there are few long-term monitoring programs, let alone programs that are systematic and rigorous (p. 201).

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

Recommendation: The committee recommends that EPA invest substantial effort to generate broader, deeper, and sustained support for long-term monitoring of key indicators of environmental quality and performance (p. 201).

Finding: Research on environmental issues is not confined to EPA. In the United States, it is spread across a number of federal agencies, national laboratories, and universities and other public-sector and private-sector facilities. There are also strong programs of environmental research in the public and private sectors in many other nations (p. 198).

Recommendation: The committee recommends that EPA improve its ability to track systematically, to influence, and in some cases to engage in collaboration with research being done by others in the United States and internationally (p. 198).

Finding: Expertise in traditional scientific disciplines—including but not limited to statistics, chemistry, economics, environmental engineering, ecology, toxicology, epidemiology, exposure science, and risk assessment—are essential for addressing the challenges of today and the future. The case of statistics is one example where the agency is facing significant retirements and needs to have, if anything, enhanced expertise. EPA is currently attuned to these needs, but staffing high-quality scientists in these areas of expertise who can embrace problems by drawing from information across disciplines will require continued attention if EPA is to maintain its leadership role in environmental science and technology (p.195).

Recommendation: EPA should continue to cultivate a scientific workforce across the agency (including ORD, program offices, and regions) that can take on transdisciplinary challenges (p. 195).

Finding: EPA’s economic, social, behavioral, and decision science staff consists almost entirely of economists. The agency is without strong expertise in social, behavioral, and decision sciences, though it does support some research in these areas through outside grants, collaborations, and procurement (p. 196).

Recommendation: The committee recommends that EPA add staff who have training in behavioral and decision sciences and find ways to enhance the existing staff capabilities in these fields (p. 196).

Finding: The need for improvement in the oversight, coordination, and management of agencywide science has been documented in studies by the National Research Council, The Government Accountability Office, and the agency’s own SAB as a serious shortcoming and it remains an obstacle at EPA. The committee’s own analysis of challenges and opportunities for the agency indicates that the need for integration of systems thinking and the need for enhanced leadership at all levels is even stronger than it has been in the past (p. 192).

Recommendation: The committee recommends that the EPA administrator continue to identify ways to substantially enhance the responsibilities of a person in an agency-wide science leadership position. That person should hold a senior position, which could be that of a deputy administrator for science, a chief scientist, or possibly a substantially strengthened version of the current science advisor position. He or she should have sufficient authority and staff resources to improve the integration and coordination of science across the agency. If this enhanced leadership position is to be successful, strengthened leadership is needed throughout the agency and the improved use of science at EPA will need to be carried out by staff at all levels. The committee specifically recommends that the person in this position and his or her staff create, implement, and periodically update an integrated, agency-wide multiyear plan for science, its use, and associated research needs (pp. 192-193).

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

Recommendation: Strengthening its scientific capacity. This can be accomplished by continuing to cultivate knowledge and expertise within the agency generally, by hiring more behavioral and decision scientists, and by drawing on scientific research and expertise from outside the agency (p. 13).

Finding: If EPA is to provide scientific leadership and high-quality science based regulation in the coming decades, it will need adequate resources to do so. Some of the committee’s recommendations, if followed, will allow EPA to address its scientific needs with greater efficiency. But the agency cannot continue to provide leadership, pursue many new needs and opportunities, and lay the foundation for ensuring future health and environmental safety unless the long term budgetary trend is reversed (p. 203).

Recommendation: The committee recommends EPA create a process to set priorities for improving the quality of its scientific endeavors over the coming decades. This process should recognize the inevitably limited resources while clearly articulating the level of resources required for the agency to continue to ensure the future health and safety of humans and ecosystems (p. 203).

REFERENCES

EPABOSC (U.S. Environmental Protection Agency Board of Scientific Councilors). 2010. Computational Toxicology Review: Letter Report. April 2010 [online]. Available: http://www.epa.gov/osp/bosc/pdf/ctox1004rpt.pdf [assessed May 2, 2014].

EPASAB (U.S. Environmental Protection Agency Science Advisory Board). 2009. SAB Comments on EPA’s Immediate Science Need. EPA-SAB-09-013. May 2009 [online]. Available: http://yosemite.epa.gov/sab/sabproduct.nsf/95eac6037dbee075852573a00075f732/379eb5d91c593045852575ad00727b52/$FILE/EPA-SAB-09013-unsigned.pdf [accessed May 2, 2014].

EPASAB (U.S. Environmental Protection Agency Science Advisory Board). 2010. ORD Strategic Research Directions and Integrated Transdisciplinary Research. EPA-SAB-10-010. July 2010 [online]. Available: http://yosemite.epa.gov/sab/sabproduct.nsf/E989ECFC125966428525775B0047BE1A/$File/EPA-SAB-10-010-unsigned.pdf [assessed May 2, 2014].

EPASAB (U.S. Environmental Protection Agency Science Advisory Board). 2012. Science Integration for Decision Making at the U.S. Environmental Protection Agency. EPA-SAB-12-008. July 2012 [online]. Available: http://yosemite.epa.gov/sab/sabproduct.nsf/cf0020ec3f99320a85256eb4006b6bd1/8aa27aa419b1d41385257a330064a479/$FILE/EPA-SAB-12-008-unsigned.pdf [accessed May 2, 2014].

EPASAB/BOSC (U.S. Environmental Protection Agency Science Advisory Board and Board of Scientific Councilors). 2011. ORD New Strategic Research Direction. EPA-SAB-12-001. October 2011 [online]. Available: http://epa.gov/osp/bosc/pdf/StratResDir111021rpt.pdf [accessed May 2, 2014].

EPASAB/BOSC (U.S. Environmental Protection Agency Science Advisory Board and Board of Scientific Councilors). 2012. Implementation of ORD Strategic Research Plans. EPA-SAB-12-012. September 2012 [online]. Available: http://yosemite.epa.gov/sab%5CSABPRODUCT.NSF/3822EB089FCCB18D85257A8700800679/$File/EPA-SAB-12-012-unsigned.pdf [accessed May 2, 2014].

GAO (U.S. Government Accountability Office). 2011. To Better Fulfill Its Mission, EPA Needs a More Coordinated Approach to Managing Its Laboratories. GAO 11-347. July 2011 [online]. Available: http://www.gao.gov/assets/330/321850.pdf [accessed May 2, 2014].

NRC (National Research Council). 2000. Strengthening Science at the U.S. Environmental Protection Agency: Research-Management and Peer-Review Practice. Washington, DC: National Academy Press.

NRC (National Research Council). 2003. The Measure of STAR: Review of the U.S. Environmental Protection Agency’s Science to Achieve Results (STAR) Research Grants Program. Washington, DC: National Academies Press.

NRC (National Research Council). 2008. Evaluating Research Efficiency in the U.S. Environmental Protection Agency. Washington, DC: National Academies Press.

NRC (National Research Council). 2010. The Use of Title 42 Authority at the U.S. Environmental Protection Agency. Washington, DC: National Academies Press.

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×

NRC (National Research Council). 2012a. Best Practices in Assessment of Research and Development Organization. Washington, DC: National Academies Press.

NRC (National Research Council). 2012b. Science for Environmental Protection: The Road Ahead. Washington, DC: National Academies Press.

Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 75
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 76
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 77
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 78
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 79
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 80
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 81
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 82
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 83
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 84
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 85
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 86
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 87
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 88
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 89
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 90
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 91
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 92
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 93
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 94
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 95
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 96
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 97
Suggested Citation:"Appendix D: Relevant Findings and Recommendations From Previous Reports." National Research Council. 2014. Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories. Washington, DC: The National Academies Press. doi: 10.17226/18950.
×
Page 98
Next: Appendix E: Summary of EPA Personnel by Laboratory Type for 2013 »
Rethinking the Components, Coordination, and Management of the U.S. Environmental Protection Agency Laboratories Get This Book
×
Buy Paperback | $48.00 Buy Ebook | $38.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Environmental Protection Agency (EPA) applies scientific results that have been provided by various parts of its own organization and by external organizations. The agency requires substantial high-quality inhouse scientific expertise and laboratory capabilities so that it can answer questions related to regulation, enforcement, and environmental effects of specific chemicals, activities, and processes. It is also usually faced with situations in which research or analytic work is time-critical, so it maintains dedicated laboratory staff and facilities that can respond quickly to such needs. In recent years, EPA has made several changes to improve the efficiency and effectiveness of its laboratories, such as the designation of national program directors to align the work of research laboratories with the needs of the agency's regulatory program offices. The agency is currently undertaking an integrated evaluation of it laboratories to enhance the management effectiveness and efficiency of its laboratory enterprise and to enhance its capabilities for research and other laboratory-based scientific and technical activities. The results of EPA's evaluation are expected to include options for colocation and consolidation of laboratory facilities.

Rethinking the Components, Coordination, and Management of U.S. Environmental Protection Agency Laboratories assesses EPA's highest-priority needs for mission-relevant laboratory science and technical support, develops principles for the efficient and effective management of EPA's laboratory enterprise to meet the agency's mission needs and strategic goals, and develops guidance for enhancing efficiency and effectiveness now and during the next 10 years. EPA's laboratories play a vital role in the agency's work. The findings and recommendations of this report will help EPA to develop an implementation plan for the laboratory enterprise.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!