Performance Measurement Tool Box and Reporting System for Research Programs and Projects (2008)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

33 CHAPTER 5 – CONCLUSIONS AND RECOMMENDATIONS CONCLUSIONS 1. The research performance measures perceived to be of most value among state transportation agency administrators and research program managers are the number of lives saved, the number of crashes avoided, and the amount of dollar cost savings realized from the implementation of research products. 2. Wide variability currently exists among state transportation agencies regarding the monitoring of research program and project performance. There is little consistency in the performance measures being used and few tools exist to provide meaningful assistance during the process. 3. Consensus use of the three outcome measures defined in the RPM System – number of lives saved, number of crashes avoided, and dollar cost savings to the agency – will likely require both coordination and encouragement from AASHTO RAC leadership and the strong support of AASHTO SCOR. RECOMMENDATIONS TO RPM SYSTEM USERS 1. The three outcome measures defined in the RPM System – number of lives saved, number of crashes avoided, and dollar cost savings to the agency – are recommended for use by every state transportation agency. 2. It is recommended that every state transportation agency seriously consider tracking research project and program performance, even if only on several highly successful research projects each year, and that this information be loaded into the RPM-Web database. While determining research benefits in this manner will provide, at a minimum, strong anecdotal evidence of justification for the program’s budget, a compendium of similar entries from a broad number of state transportation agencies will result in meaningful information being derived from the national summary report. Nationwide summary information should prove valuable at the time of the next federal transportation budget re-authorization. 3. Research program and project performance measures should be carefully selected. It is recommended that only research performance measures tracking the highest priority performance areas be officially selected and reported for a given performance period. Other performance metrics may well be monitored, at the discretion of and as needed by the research program manager. Wise and limited selections followed by thorough tracking are believed to compose the formula for success in research performance measurement. 4. Credible determination of estimated research benefits requires three rules to be strictly followed. a. Consult top agency experts whenever a factor in the benefit calculation must be an estimate based purely or primarily on an individual’s experience. b. Document both the sources of information used in the calculation of estimated benefits as well as the method of performing the calculation. c. Estimations should always be made on the conservative side of probability. This must be made clear to the experienced personnel asked to provide an estimated factor.

34 5. Consider requiring that contract researchers provide an estimate of expected benefits for the sponsoring agency if products from the research project are fully implemented by the research sponsor. This would be the final deliverable of the researcher’s project. Not only will this provide the state transportation agency an early estimate of benefits, but it is likely that this requirement will also tend to better focus the researcher throughout the project on obtaining the desired benefits. SUGGESTED RESEARCH 1. A suggested major next step in research performance measurement is the development of one or more additional performance measures focusing on the impact of research on traffic congestion. As traffic congestion and associated travel time delays are a major concern of the traveling public, a performance measure including quantitative means for characterizing reduced traffic congestion would substantially improve the set of available research performance measures. Tools to assist users as well as example benefit estimations will be necessary complements to the new measure or measures. 2. As research office staff resources are usually stretched thinly, it is suggested that an immediate effort be organized to identify and pursue means for facilitating use of the new system in state research offices. Improvements might include development of additional tools, modified screen designs or functionalities, or any other manner of improving system ease of use, intuitiveness, and efficiency. A group of selected states planning early implementation would possibly compose an ideal team to meet, share ideas and brainstorm, and then feed desired or needed changes back to NCHRP, AASHTO, or another body for funding consideration. There may be great value in including several states on this team who have not been involved with the system’s development since these states may bring additional perspectives, needs, or ideas which have eluded both the research team and the panel.