Research and Technology Coordinating Committee Letter Report: March 2012 (2012)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

500 Fifth Street, NW Washington, DC 20001 Phone (202) 334-2934 Fax (202) 334-2003 www.TRB.org March 5, 2012 Victor Mendez Administrator Federal Highway Administration 1200 New Jersey Avenue, SE Washington, DC 20590 Dear Mr. Mendez, On December 15–16, 2011, the Research and Technology Coordinating Committee (RTCC) met with the Federal Highway Administration’s (FHWA’s) Research, Development, and Technology (RD&T) staff at the Keck Center in Washington, D.C. The roster of the committee, indicating members in attendance, appears in Attachment 1. For this meeting FHWA staff posed two broad questions on which they were seeking guidance: whether FHWA’s RD&T program is (a) investing in the right things and (b) carrying out its program in the right ways. RTCC’s charge is to monitor and review FHWA’s research and technology activities and advise FHWA on (a) the setting of a research agenda and coordination of highway research with states, universities, and other partners; (b) strategies to accelerate the deployment and adoption of innovation; and (c) areas where research may be needed. RTCC’s review includes the process of research agenda setting, stakeholder involvement, the conduct of research, peer review, and deployment. The committee’s role is to provide strategic, policy-level advice on topical priorities, processes, and strategies to accelerate the adoption of innovation. At FHWA’s request, this letter addresses FHWA RD&T priority setting, program management, and performance measurement. The content of the letter was developed in closed- session deliberations and subsequent correspondence among the members. The letter was then subject to the National Research Council’s peer-review process. The letter is organized as follows: the first section addresses FHWA RD&T priority setting, the second addresses program management topics raised at the meeting, and the third addresses performance measurement. In each of these sections, a brief overview of the corresponding FHWA presentation is provided, followed by RTCC commentary on the subject. The final section lists suggested follow-up items on which FHWA has requested RTCC guidance. FHWA RD&T PRIORITY SETTING Context To set the stage for discussion about RD&T priority setting, Michael Trentacoste, Associate Administrator for RD&T, and Debra Elston, Director, Corporate Research, Technology, and

2 Innovation Management, explained that the absence of RD&T earmarks in the extensions of current authorizing legislation has provided FHWA with flexibility in choosing projects and allocating funding within the designated categories of the Safe, Accountable, Flexible, Efficient Transportation Equity Act: A Legacy for Users (SAFETEA-LU). The Senate reauthorization bill, Moving Ahead for Progress in the 21st Century, provides FHWA with even more discretion within broad guidance laid out in the bill. The bill passed in the House of Representatives, the American Energy and Infrastructure Jobs Act of 2012, provides similar flexibility to FHWA. Both bills are free of research project earmarks. Thus, if these provisions are maintained in a final bill approved by Congress, FHWA could be resuming its role of setting priorities and allocating resources across what may be a substantial program for the first time in many years. The fixed shares of funding across infrastructure, safety, planning and environment, operations, and policy that have prevailed during SAFETEA-LU need not be continued. Thus, FHWA RD&T staff asked RTCC for guidance on research priorities and identification of possible gaps in its RD&T program. To aid the committee in its discussion, the R&D priority-setting process of the Office of Infrastructure was described as one example of how various FHWA offices determine what RD&T to invest in. This process is discussed next. Infrastructure RD&T Strategic Planning Peter Stephanos, Director, Office of Pavement Technology, described the Office of Infrastructure’s comprehensive research and technology strategic planning and programming process, which aligns proposed research projects and deployment efforts with the office’s strategic objectives. The process gathers input from technical staff throughout the office and provides information that office directors can use in setting priorities and allocating resources. Included within the process is the development of a strategic plan, research road maps, 1 detailed work plans, and a project reporting system for monitoring progress. The committee is impressed by the comprehensive and systematic planning process followed by the Office of Infrastructure but sees room for improvement in one area. With regard to stakeholder input, individual Office of Infrastructure staff members have apparently consulted with stakeholders and have selected projects from comprehensive road maps developed by the pavement and concrete industries. [Stakeholders interested in FHWA’s RD&T programs include state departments of transportation (DOTs), metropolitan planning organizations (MPOs), local governments, highway industry groups, and researchers, among others.] Mr. Stephanos acknowledged, however, that stakeholder engagement in the development of the plan has not been as systematic as it could have been. The sharing of FHWA research road maps and plans with the wider community is an important next step in opening up the RD&T planning process. As plans are refined and developed, an earlier and more systematic outreach process would be helpful in soliciting stakeholders’ views on the issues they face and on their strategic goals for research. To develop buy-in for FHWA’s program, this process needs to be transparent to external stakeholders. Many staff members are involved in the development of the infrastructure research and technology strategic plan, road maps, and work plans, and external stakeholders are interested in these documents. Thus, communication of the extensive planning process to internal and external stakeholders is important and challenging. In this regard, FHWA will need a simplified, 1 Research road maps are generally multiyear plans for research within a discrete area or subarea that lay out objectives, projects, benchmarks, timelines, and expected deliverables.

3 higher-level presentation of its research plans and related documents to external stakeholders. (RTCC understands that the Office of RD&T has engaged the Volpe National Transportation Systems Center to assist in the development of an accessible report.) Nevertheless, the documents will need to be sufficiently detailed to allow stakeholders to identify the strategic goals for R&D and the major initiatives being planned to meet those goals. Some stakeholders will also be interested in specific R&D projects; to the extent that they are defined, it may be necessary to include appendix material that lists proposed projects and the rationale for including them. Two additional items are offered for FHWA’s consideration with regard to priority setting. First, although a transparent and systematic process is needed for developing a research strategic plan and related implementation plans, the process should not become so elaborate that the cost and effort of engaging in the process begin to erode its benefits. Second, it will be helpful for FHWA to view the agency’s investments in research as a portfolio, with the understanding that relatively few ideas explored through research, whether in government or in industry, ever mature into usable products. As with any portfolio of investments, the goal is to have enough investments pay off in ways that justify the entire effort. Preliminary Plans for Allocating Resources to Deployment On the assumption that a future authorization will specify an amount for deployment and give FHWA broader discretion over particular initiatives to pursue, Jack Jernigan, Team Director, Research and Technology Program Development and Partnership Team, shared FHWA staff’s preliminary thinking about how resources would be devoted to deployment and technology transfer. The proposed strategy, which would distribute some funds to program offices by formula and award other specific funds to projects on the basis of merit, appeals to RTCC. As part of the process of determining merit, solicitation of the views of stakeholders and customers concerning the kinds of products they need and the support required to implement them would be important. The proposal to allocate some funds for further pilot testing and evaluation of promising products is also logical, since it is important to identify products for deployment that are truly “market-ready.” Additional RTCC Observations on Priority Setting With regard to priority setting, FHWA’s RD&T can be conceived of as a federal, rather than a national, highway research program. The federal program would be one that serves FHWA’s goals and objectives; the national program would include highway research funded by other modal administrations, agencies, states, and universities. Clearly, FHWA must invest in some research to carry out its role as a mission agency. FHWA also has an important coordination role in the entire national program, including highway research carried out by other modal administrations. The distinction between federal and national, however, breaks down beyond FHWA’s regulatory role because FHWA depends on others, primarily state DOTs, MPOs, counties, and cities, to implement innovations coming out of FHWA’s R&D program. Thus, a substantial element of FHWA’s program must be responsive to the needs of these stakeholders, and a major part of its deployment program must be devoted to delivering products that they need. RTCC is pleased that FHWA recognizes the challenges it faces in allocating resources to its R&D and technology delivery programs. How much to invest in infrastructure as opposed to

4 safety, mobility, or any other functional area of FHWA RD&T is unclear. Any guidance that RTCC might offer would reflect the biases and the areas of expertise of the current members. The committee can, however, comment on the process that FHWA follows. Clearly, an important element of this process is early and regular engagement of stakeholders at the right levels. CEOs, chief engineers, and heads of planning and operations of state DOTs are the appropriate audiences to engage with regard to what the strategic goals should be for FHWA RD&T. Similarly, transportation directors at MPOs could help inform FHWA’s R&D strategic goals. Technical staff at DOTs and other organizations can also provide input, particularly on the potential payoffs from specific R&D initiatives. Most important is to understand what FHWA’s customers would value. Similar engagement with private industry is also important; however, the kinds of RD&T identified need to be appropriate for the federal government. If authorization passes in the next few months, FHWA may be handed substantially more discretion over RD&T resource allocation before it has fully developed its RD&T plans. If that occurs, it may be advisable to maintain some consistency, at least initially, with how resources have been allocated across infrastructure, operations, safety, planning and environment, and policy. Future shares of funding across functional areas can be driven by customer and stakeholder needs as they are better articulated and by decisions about specific research investments based on merit. PROGRAM MANAGEMENT To provide context for possible RTCC guidance on program management, FHWA staff briefed RTCC on the Every Day Counts (EDC) initiative and the Exploratory Advanced Research (EAR) Program. Every Day Counts Michael Trentacoste provided the committee with a status report on EDC, which RTCC views as a valuable technology deployment program. Mr. Trentacoste noted that success in implementing the initial round of market-ready technologies promoted in EDC has resulted in a broad-based agency solicitation of other innovations that are appropriate for a high-level effort to secure widespread adoption. RTCC acknowledges the benefits of soliciting suggestions about specific market-ready innovations. It would also be beneficial for FHWA to ask stakeholders about areas where they are most in need of assistance. A clearer focus on the problems that stakeholders are addressing would feed back to the R&D priority-setting process described above to inform the kinds of research that FHWA should invest in to yield useful products. RTCC was pleased to learn that FHWA’s division offices have become involved in the delivery of EDC. Federal staff at the state level could also be used to gather input on what states, MPOs, and local governments need most with regard to future research and product development. Exploratory Advanced Research David Kuehn, Program Manager for the EAR Program, presented a status report to the committee, which is pleased with the program’s progress. RTCC has long held the view that FHWA’s special niche in highway research is to carry out investigations that are further

5 upstream, and therefore riskier, than the kinds of research carried out by the National Cooperative Highway Research Program and the states through their individual programs. Investing in riskier, long-term initiatives where the payoffs are the least certain but the benefits are potentially the largest is a role best filled by the federal government. PERFORMANCE MEASUREMENT FHWA staff asked for RTCC guidance on developing performance measures for its R&D and technology delivery efforts. As Michael Trentacoste and Jack Jernigan indicated, they are interested in advice about measures that would be appropriate at the initiative and program levels rather than at the project level. Many federal RD&T programs have developed program-level performance measures. Most of them are appropriate for basic research, which is the largest area of federal investment in science, engineering, and health care research. Whereas they would not be effective for FHWA’s applied R&D, they could be informative for the EAR Program. Most of FHWA’s R&D is applied, and for these efforts, performance measures can be more easily related to outputs and outcomes than can performance measures for basic research. The attached white paper prepared by Transportation Research Board staff identifies potential performance measures for applied research and criteria for selecting them (Attachment 2). With regard to criteria for selection, choosing measures that are meaningful to policy makers within the U.S. Department of Transportation, Congress, and key stakeholders is particularly important. Performance measures are useful in communicating program progress to those who allocate and influence the allocation of resources. R&D program managers themselves also have to be concerned about the technical merits of the initiatives they fund and whether the initiatives are being accomplished. On this point, the learning opportunities offered by projects that do not reach their anticipated goals can be valuable. Even research projects that fail to reject the null hypothesis can be meaningful. Program managers should evaluate RD&T results to gain from the lessons learned and improve their processes regardless of how individual projects turn out. Finally, as FHWA identifies possible measures, it would benefit from considering measures appropriate for each step of the innovation cycle, recognizing that different measures will be appropriate for different steps. The committee would be pleased to assist FHWA staff in identifying appropriate performance measures. Among the important issues to be addressed are the difficulties of obtaining appropriate and reliable measures at an affordable cost; the general lack of careful evaluation of innovations introduced by highway agencies and MPOs; the challenges of evaluating the benefits of innovations, particularly long-lived assets; and incentives to choose measures that are readily obtained rather than those that would be most appropriate. NEXT STEPS As planning begins for the next meeting, RTCC suggests the following topics for consideration: The committee would like learn more about how FHWA plans to carry out communicating its RD&T plans, both externally and internally. Once FHWA’s road maps and program plans are posted online, the committee is interested in how FHWA will solicit stakeholder input and revise its plans.

6 RTCC would benefit from hearing how other program offices within FHWA set priorities for RD&T. The committee would like to continue the discussion of FHWA’s coordination role, both with the states and with other agencies funding highway research. The committee is particularly interested in how FHWA will communicate with the new university transportation centers concerning its overall national research agenda and strategy and how it plans to coordinate with them. RTCC would be pleased to continue to assist in performance measurement and welcomes the opportunity to comment on measures that the staff proposes to adopt. On behalf of RTCC, I offer my thanks to Michael Trentacoste and his staff for excellent presentations that set the stage for a useful, productive discussion. I hope you find this letter to be similarly useful as the RD&T programs move forward. Sincerely, Michael D. Meyer, Chairman and Professor, Department of Civil and Environmental Engineering, and Director, Georgia Transportation Institute, Georgia Institute of Technology Attachments Participants, December 2011 Meeting Staff White Paper on Identifying Potential Performance Measures for FHWA RD&T