Highway Safety Research Agenda: Infrastructure and Operations (2013)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

8Background on Prioritization Methodologies in Use for Assessing Research Needs The process of developing a national prioritization meth- odology began with a review of the literature to identify exist- ing methods used for prioritizing research and projects both nationally and internationally. Specifically, the literature review focused on prioritization methods within the transportation field while considering general prioritization issues faced in other fields. Special attention was given to existing methods for prioritizing highway safety research. The section below pro- vides an overview of the salient and common factors involved in the prioritization processes from the literature. The sources used in the literature review are provided in the reference list at the end of this report. Additionally, more information is given in two appendices: Appendix A. Prioritizing Projects: General Consider- ations – includes literature related to project prioritization in general. Appendix B. Prioritizing Projects: Specific Examples from the Transportation Field – features reports and litera- ture related to specific prioritization processes employed in the transportation field. Note: These appendices are not published herein, but can be accessed on the TRB website at the follow- ing URL: http://apps.trb.org/cmsfeed/TRBNetProjectDisplay. asp?ProjectID=2727. Overarching Issues Observed in Previous Prioritization Methods 1. Minimize the power of politics involved in the process and clearly define and justify federal roles and responsibilities. 2. Develop a formal, consistent, and visible prioritization process based on a data-driven approach. 3. Encourage collaboration among stakeholders and bottom- up initiation of projects. Must have committed champions and should encourage joint funding so resources can be leveraged for common purposes. 4. Support overall organizational goals and strategies. 5. Focus on large, multi-modal research efforts of national importance. 6. Grant certain projects “immunity” to ensure that projects late in the development process are not reprioritized with- out good cause (applies to future prioritization cycles). 7. Encourage communication throughout the strategic pri- oritization process with and among stakeholders. 8. Ensure visibility of results to all stakeholders. Specific Processes Observed 1. Develop selection criteria and weighting scheme. a. Based on strategic fit. i. Contribution to strategic vision, mission, and goals. ii. What is the appropriate perspective (national, regional, or local)? iii. Does the project fill a gap in existing literature or build onto areas where current information is lacking? b. Based on stakeholder input. i. Was the project ranked as a high priority, relative to other topics, by experts from a range of backgrounds? ii. Was the project previously identified as an issue of concern by other organizations (including TRB committees and federal agencies)? c. Based on urgency. i. Is it a short-term, intermediate-term, or long-term need? ii. How soon can the research be expected to contribute? d. Based on feasibility. i. Is it possible with existing skills and resources? ii. Does it deliver value (economically or directly to the road user)? C H A P T E R 2 Developing the Prioritization Methodology

9 e. Based on potential benefits. i. Does the project have the potential to address safety issues for a wide range of communities, crash types, and citizens? f. Based on cost. i. What is the level of financial investment needed for research and development and for successful implementation of the results? g. Based on risk. i. Is it likely to go over budget or over time? ii. Is it so complex that it is likely to fail? iii. Are there other risk factors including environmen- tal challenges, stakeholder opposition, or harm to reputation? h. Consider the impact and implementation prioritiza- tion matrix (see Figure 1 below, from Transformation Management Team, Final Report, Volume 5: Strategic Planning and Prioritization, North Carolina Depart- ment of Transportation [NCDOT]). i. Used to assist in ranking the projects. 2. Provide/publish project selection criteria (make available to stakeholders). a. Allows self-evaluation of projects before submitting for consideration. 3. Develop a form/template of information to be completed for each proposed project by the submitting agency/person (similar to RNSs). a. These data will be considered for project selection. b. Data may include purpose, benefits, cost, and technical merits, along with subjective elements such as inter- modal connectivity, geographic balance, economic importance, and relevance to a particular objective. c. Any risks or potential risks associated with the project should also be included in the template. 4. Identify potential list of projects. a. Incorporate input from all stakeholders (including the public, if applicable). b. Where there are inputs from multiple stakeholders, each group should prioritize their own needs prior to submitting for consideration in the overall list. c. Use a funnel approach to receive bottom-up input. i. Where there are several layers within an organiza- tion, let the organization receive input from all lev- els within (and prioritize) prior to submitting for consideration in the overall list. 5. Identify a priority team. a. Define the roles and responsibilities of the priority team. i. Review proposal submittals. ii. Assign scores based on selection criteria. b. Consider trade-offs among projects. i. Can a project be partially funded to achieve partial results or does it need to be fully funded in order to achieve any meaningful results? c. Develop prioritized list based on the proposal scores and overall vision/mission. i. Top-down approach to be consistent with overall goals. ii. Bottom-up approach by receiving input from stakeholders. Figure 1. Impact and implementation prioritization matrix.

10 A discussion of the prioritization methods for CMF research and non-CMF research is presented below. Rating Crash Modification Factor (CMF) Research Topics Summary of Value of Research (VOR) Method This section provides a brief overview of the method that has been developed for prioritizing research to develop CMFs. It is referred to herein as the VOR method. More information about this method is available in Appendix C, which is not pub- lished herein. The objective of the VOR method is to provide a more objective means by which to rank candidate research projects. The underlying basis for the proposed method is that all research projects have one goal – to develop information for making a better safety-related decision (i.e., information that will reduce the chances of making an inferior decision). The decision to be made is whether or not to implement a given treatment on a specific set of roadway locations. The standard deviation of the CMF is a measure of the uncertainty of the CMF. The smaller the standard deviation of the CMF, the lower the chance of making an inferior decision. If, by conducting a research project, it is possible to substan- tially reduce the standard deviation of the CMF, then one could argue that such a research project has good value because it provides a sounder basis for decision. The VOR method com- putes a monetary value based on the extent to which a research project can reduce the standard deviation of a CMF and thus increase the chances of making the correct decision. Overview of VOR Concept This subsection describes the concepts underlying the VOR method. For this discussion, the CMF is defined using the variable q. If q is less than 1.0, then the treatment is expected to reduce crashes, whereas if q is greater than 1.0, the treat- ment is expected to increase crashes. For example, if q is 0.8, then the treatment is expected to reduce crashes by 20 per- cent. The safety benefit of a treatment at a roadway segment can be expressed as follows: = × Safety Benefit (Predictedreductionin targetcrashesper year) ($ value of the target crash) where the predicted reduction in target crashes per year equals the expected number of target crashes times (1- q). For the purposes of an example, say that for some roadway sections where an implementation decision must be made, the annual cost of implementing and maintaining the treatment is $9,000, the expected number of target crashes per year is 8.0, and the average cost of a target crash is $10,000. Each agency d. Develop a prioritized list of unrestricted scenarios. i. Unrestricted with respect to financial and legal restrictions – investments that need to be made. e. Develop a prioritized list of restricted scenarios. i. Restricted with respect to financial and legal restrictions – investments that can be made. 6. Document, communicate, and market final results. a. Include partners, stakeholders, the public, and the media. b. Document to provide visibility of the prioritization process by describing the input, the scenarios evalu- ated, the decisions made, the anticipated effects of those decisions, and the need for additional funding and/or funding flexibility. Perceived Weaknesses of Previous Prioritization Methods The team noted several weaknesses in the previous prioriti- zation methods. These previous methods were observed to be: 1. Ad-hoc. 2. Reactive. 3. Not supported by data. 4. Involving too many decision-makers. 5. Not visible to stakeholders and the general public. 6. Lacking input from stakeholders, particularly from aca- demia and industry. 7. Selecting projects with an emphasis on local priorities and external inputs at the expense of systematically addressing long-term needs (not achieving overall goals). Based on the limitations identified in current methods for prioritizing research, there is a need for a more consistent and rigorous prioritization process. The project team has devel- oped a new prioritization methodology that incorporates the key elements identified in the literature review and addresses many of the shortcomings of existing methods. The method- ology includes two methods – one for prioritizing research related to the development of CMFs and a second related to non-CMF research. As discussed in Chapter 1, other types of research needs were identified but were not included in the prioritization process because they were not defined as research according to the scope of the project. Those projects that would not generate new knowledge (e.g., guidebooks, syntheses, implementation tools, and scans) were not consid- ered “research” and were not included in the prioritization. In order to prioritize these types of “non-research implementa- tion” projects, an agency may use subjective assessment by decision-makers, or develop a priority ranking method com- parable to the method developed herein for the fundamental and non-CMF applied RNSs.

11 In Figure 2, mean q represents the mean value of q based on previous research, and each shaded block represents one standard error of q. The figure illustrates four scenarios regarding relative values of mean q and qb. In the top two sce- narios, the mean values of q and qb are relatively far apart, and qb is outside the 95% confidence interval of the mean value of q (the 95% confidence interval is approximately mean q ± 2 standard errors). Hence, in these two scenarios, the likelihood of making an incorrect decision is low, the cost of making an incorrect decision is low, and the value of conducting further research to refine the existing CMF is relatively low. On the other hand, in the bottom two scenarios, the mean values of q and qb are closer to each other, and qb is within the 95% con- fidence interval of the mean value of q. Hence, in the bottom two scenarios, the likelihood of making an incorrect decision will be higher, and the value of conducting further research to refine the existing CMF is relatively high. If new research on a treatment substantially reduces the standard deviation of q, then it is possible to reduce the cost of making an incorrect decision. The value of a research proj- ect is computed as the cost of making an incorrect decision based on current standard deviation of q minus the cost of making an incorrect decision based on the revised standard deviation of q that is expected after the research is completed. Appendix D, which is not published herein, provides a dis- cussion of how the current standard deviation of q can be estimated from prior research. If no prior research is avail- able to estimate the current mean and standard deviation of q, then Appendix D describes how these parameters can be estimated using an expert panel or other means. may have its own policy regarding the minimum benefit cost ratio that is necessary before a treatment can be implemented at a particular site. If the benefit cost ratio is assumed to be 2.5, then in order to justify the treatment, the reduction in crash harm due to the treatment has to be at least 2.5 × 9,000 = $22,500. Since the average cost of a target crash is $10,000, the proposed treatment should reduce the frequency of tar- get crashes by 2.25 (22,500/10,000) from the current expected number of 8.0 in order to be justified. This represents a reduc- tion of 28.125% (2.25/8.0), i.e., a q of about 0.72. So, if q of the treatment is greater than 0.72 (i.e., the estimated treatment effectiveness is less than 28%), then the treatment is not justi- fied, but if it is less than 0.72, it is justified. The q of 0.72 can be considered to be a breakeven point (denoted by qb). From previous research (for example, from the CMF Clear- inghouse or the Highway Safety Manual [HSM]), an estimate can be made for q and the standard deviation of the q for the particular treatment being considered. It is important to note that q is not a universal constant, but a random variable with a mean and a standard deviation. It is an estimate of the true treatment effect which will never be known and which may change with different roadway characteristics. Since the topic is whether or not to implement a treatment in the future, the interest is in the q for a future implementation. The exact q for the future implementation is unknown, but the best assumption is that it will be a value from a distribution that can be derived from the mean and standard deviation of the value of q based on previous research. For the example discussed above, if the mean value of q based on previous research is 0.6 (less than qb), then the deci- sion will be to implement the treatment. However, since q is not a universal constant, q for a future implementation could be more than qb, and in that case, the decision to implement will be an incorrect decision. On the other hand, if the mean value of q based on previous research is 0.9 (greater than qb), then the decision will be to not implement the treat- ment. Again, since q is not a universal constant, q for a future implementation could be less than qb, and the decision not to implement will be an incorrect decision. Based on the mean and standard deviation of q (based on information from pre- vious research) and assuming that q is gamma distributed, one can derive the probability of making an incorrect decision and the cost of making an incorrect decision (Appendix C provides further details about the method for computing this). If the standard deviation of q based on the previous research is very low, then the chances of making an incor- rect decision will be low and the cost of making an incorrect decision will be low as well. Similarly, if the mean values of q (based on previous research) and qb are very far from each other, then the likelihood of making an incorrect decision will be low and the cost of making an incorrect decision will be low as well. This phenomenon is illustrated in Figure 2. Figure 2. Relative positions of  and b and the impact on the VOR.

12 Step 4 - Determine the Expected Standard Deviation of the CMF After the New Research is Completed The objective of this step is to estimate the expected standard deviation of the subject treatment’s effect on crash frequency based on the proposed research. This standard deviation must be smaller than that estimated in Step 3 if the research is to have value. This result can be achieved by using a study design that accounts for the likely sources of systematic variation that underlie the standard deviation. Appendix D provides a discus- sion of the methods that can be used to estimate the expected standard deviation. Step 5 - Apply the Procedure to Estimate the Expected Value of the Research The objective of this step is to use the information from previous steps with the VOR method to compute the expected value of the information obtained from the proposed research project. An Excel-based software tool was developed to facili- tate the implementation of the VOR method. It is described in Appendix E and F. The User Manual for the VOR tool is given in Appendix G. These appendices are not published herein, but are available online at: http://apps.trb.org/cmsfeed/ TRBNetProjectDisplay.asp?ProjectID=2727. Implementation of the VOR Method This section summarizes the process followed to develop an implementation-ready version of the VOR method. This ver- sion of the method is intended to facilitate the cost-effective evaluation of proposed research projects. Projects suitable for local, regional, or national application can be evaluated with equal ease. It is envisioned that the implementation- ready version of the VOR method can be used by a SAC on an annual basis to update and maintain a prioritized list of proposed research projects. As alluded to in the previous section, the VOR method requires some input data that can be challenging to acquire. It also includes some analytic components that are not amena- ble to manual calculation. The manner by which these issues have been addressed is described in this section. This section consists of three subsections. The first subsec- tion summarizes the input data needed for the VOR method and the likely sources of these data. The second subsection summarizes the procedure used to estimate the target crash distribution. The third subsection summarizes the role of expert opinion as a source of the information needed for the VOR method. Appendix E provides more detail about the development of the implementation-ready version of the VOR method. Application Steps This subsection describes the steps followed when apply- ing the VOR method to evaluate one CMF-based research project: Step 1 - Identify Target Sites (Segments or Intersections) Where a Treatment Can Be Implemented The objective of this step is to determine the types of seg- ments or intersections at which the subject treatment can be applied. If a treatment is applicable to segments, then the num- ber of treatable miles is determined. If a treatment is applicable to intersections, then the number of intersections is determined. The segments can be categorized by functional class (e.g., rural principal arterial) or by facility type (e.g., rural two-lane high- way). The intersections can be categorized by area type, control type, number of legs, etc. If a treatment can be implemented nationwide, then it is important to be able to estimate the num- ber of target segments or intersections nationwide. Step 2 - Determine the Distribution of Target Crashes for These Sites The objective of this step is to quantify the parameters of the distribution of crashes that occurred at the target sites. These parameters are then used with the gamma distribu- tion to define the target crash distribution. If the treatment is applicable to a given state, then an existing safety performance function (SPF) derived for that state can be used to estimate the mean and variance of the target crash frequency. If the treatment is applicable to several states, then an existing SPF for each state can be used in a similar manner to estimate the mean and variance of crash frequency for each state. These parameters are then combined to determine the mean and variance of the crash frequency for all states. If the treatment is applicable to several states and an existing SPF is not avail- able for every state, then the SPF for one state can be calibrated to the other states using ratio of the fatal crash rate of the other state to that of the base state. Further discussion about this is provided in Appendix E. This appendix is not published herein, but is available on the TRB website at: http://apps.trb. org/cmsfeed/TRBNetProjectDisplay.asp?ProjectID=2727. Step 3 - Determine the Mean and Standard Deviation of the CMF Based on Previous Research The objective of this step is to quantify the mean and stan- dard deviation of the subject treatment’s effect on crash fre- quency based on the findings of past research. FHWA’s Crash Modification Factor Clearinghouse is a good starting point for extracting this information.

13 can be used to estimate the number of candidate units using its sample expansion factors. Other options are available in this case, and are discussed in Appendix E. The “number of affected units” is computed by multiply- ing the number of candidate units by the proportion of those units that will likely be considered for treatment. This pro- portion can vary, depending on treatment cost, effectiveness, and agency policy. One possible source for obtaining the esti- mate of the number of affected units is the expert opinion of practicing highway safety engineers. These engineers could be organized into a transportation agency partners (TAP) group. The “crash distribution” represents the distribution of aver- age annual crashes per unit for the candidate units. It is developed for each proposed research project. The crashes rep- resented in the distribution are referred to as “target” crashes. The procedure used to estimate the distribution parameters is summarized in the next subsection. The “cost of target crash” is computed for each proposed research project. It must be representative of the range of severi- ties included in the target crash distribution. Once the severity distribution is identified, current estimates of the crash cost for each severity level can be used to determine the target crash cost. The data for several states represented in the Highway Safety Information System (HSIS) database were used to develop rep- resentative crash severity distributions by functional class for segments and for intersections. The development of these dis- tributions is documented in Appendix F. The “cost of implementation” represents an annualized cost on a per-unit basis. The initial cost is annualized over the service life of the project, based on an expected service life and typical discount rate for highway investments. This project has compiled cost information for a series of counter- measures (treatments). This information should be archived by the SAC such that it can be subsequently used to evalu- ate other research projects whose treatments are judged to be similar to treatments previously considered. Information Needs and Sources This section describes the key factors that are incorpo- rated in the VOR method. Table 1 identifies these factors and provides a brief description. The term “unit” in this table is defined herein as either an intersection or a highway segment. These two entities are typically the basis for SPF development because they are represented in this manner in the highway safety databases used for calibration. This need for consistency between the VOR method and the SPF stems from the use of the SPF regression results to define the distribution of crashes. This section also reviews possible sources of information associated with each factor in this table. A source may be an existing database, prior research, or expert opinion. In some cases, the data obtained will require some “post processing” to convert it into the proper form for use in the VOR procedure. The calculations associated with this post processing are also described where appropriate. The role of expert opinion as a source of information is described in a subsequent section. The source of information may vary depending on whether the treatment to be evaluated by the proposed research is applicable to an entire class of roadway in most states, or just to a specific portion of roads (e.g., curved sections, three-leg signalized intersections) in one state or region. The “number of candidate units” is defined as the num- ber of highway segments or intersections that can be treated with the subject treatment. The emphasis on “can be treated” is a reminder that all units may not be treated by the sub- ject treatment. When the treatment can be widely applied, it should be sufficient to consider the total mileage of one or more functional classes of roadway in the states likely to use the treatment. The Highway Statistics database (http://www. fhwa.dot.gov/policy/ohpi/hss/index.cfm) provides miles by functional class and state, as needed for this type of appli- cation. When the treatment is not widely applied, then the Highway Performance Monitoring System (HPMS) database Factors Description Number of candidate units Number of highway segments or intersections that can be treated with the treatment being addressed in the proposed research project. Number of affected units Number of segments or intersections likely to be considered for treatment. Crash distribution Distribution of crashes on treated units. Cost of target crash Average cost of the crash that is likely to be prevented by the treatment. Cost of implementation Average cost (per mile or intersection) to the transportation agency to implement the treatment. Estimated treatment effectiveness Mean effectiveness of the treatment (i.e., CMF) and its standard deviation (before new research is conducted). Table 1. Factors considered in VOR calculation.

14 agency perspective and information to the SAC during the research project prioritization process. Each transportation agency represented in the TAP could designate one person that could be contacted periodically (e.g., annually) by the SAC to obtain some of the information in Table 1. The TAP members would be contacted annually to pro- vide the information identified in Table 2. It is envisioned that the SAC will perform an initial triage on the proposed research projects at a specified time in the annual research cycle. Then, they will identify all of the information they need for the collective set of projects being considered for prioriti- zation. One comprehensive information request will then be sent to the TAP. Pilot Testing the VOR Method The VOR method was pilot tested for two treatments: effectiveness of driveway access control and effectiveness of edgeline rumble strips. Further discussion about the two pilots is provided below. Evaluation of Proposed Research on the Effectiveness of Driveway Access Control This section is a brief summary of the pilot study entitled “Limited Driveway Access at Intersections—Sample Appli- cation of Value-of-Research Evaluation,” which is given in more detail in Appendix H and which documents the con- duct of a VOR analysis for a specific safety treatment. The example treatment is the removal of major-street driveway access for a corner business at an urban or suburban signal- ized intersection. The minor-street access to the business would be preserved or added if not existing. The objective of this treatment is to reduce the frequency of driveway-related crashes by moving driveway access maneuvers to the (lighter- volume) minor street. The following seven questions are answered in the conduct of a VOR analysis: 1. How many units (highway segments or intersections) will be affected by the treatment of interest? 2. What is the frequency of crashes that are targeted by the treatment? 3. What is the cost of an average target crash? 4. What is the annual cost of deciding to implement the treatment on one unit? 5. What is the limiting benefit/cost ratio? 6. What are the mean and standard deviation of the CMF based on existing knowledge? 7. What is likely to be the standard deviation of the CMF after the proposed research is completed? The “estimated treatment effectiveness” is characterized by its mean CMF value and the standard deviation of this value. This information can be obtained from previous projects that evaluated the same (or a similar) treatment. If the proposed project is evaluating a treatment for the first time, then expert opinion may be used to estimate the necessary values. A pro- cedure for estimating the mean and standard deviation of a CMF is described in Appendix D. Estimating the Crash Distribution The crash distribution used in the VOR method should be obtained from a highway safety database. Ideally, a highway safety database would be available for each of the 50 states. The crash distribution would be developed from these state databases, with consideration of the treatment characteristics. Unfortunately, this option is not currently viable because a highway safety database is not readily available for each state. This section summarizes the procedure used to estimate the distribution parameters. Additional information about the procedure is provided in Appendices E and F. Two procedures were developed for estimating the crash distribution parameters. One procedure is applicable for evaluating research projects addressing treatments that can be widely applied. It consists of eight calculation steps. The other procedure is for projects that address treatments that are not widely applied. It consists of four calculation steps. The procedure for widely applicable treatments is based on the use of one SPF (and its associated over-dispersion param- eter) for each of the desired functional classes in one state. This SPF is calibrated to each of the other states being consid- ered and used to estimate the mean annual crashes per unit for each state (as well as its variance). With this approach, the crash distribution parameters for all states combined are computed and one value of VOR is computed for the com- bined crash distribution. The procedure for treatments that are not widely appli- cable is also based on the use of one SPF (and its associated over-dispersion parameter) for one state. This SPF is devel- oped using HSIS data that are specifically screened to reflect the target site characteristic (e.g., curve road sections on rural two-lane highways). The SPF is then calibrated to each of the other states being considered and used to estimate the mean annual crashes per unit for each state (as well as its variance). The crash distribution parameters are combined in the same manner as for widely applicable treatments. Transportation Agency Partner (TAP) One possible source of information is the expert opinion of practicing highway safety engineers, also called as the TAP. The TAP is envisioned to be a group of engineers that provide

15 revealed that the typical urban or suburban signalized inter- section experiences 0.565 injury or fatal driveway-related crashes per year. Question 3 was answered by analyzing the severity distri- bution of crashes at the 180-intersection sample in Texas. The severity distribution was combined with crash cost estimates from the literature to yield an average driveway-related crash cost of $44,732. To answer Question 4, it is necessary to determine the aver- age construction cost of adding a driveway and the average business impact of removing driveway access. Based on cost estimates from several states, the cost of adding a driveway was estimated as $10,000. This cost would apply to affected units that lack minor-street driveway access. To determine the business impact of removing driveway access, literature sources relating to the impacts of access management were reviewed. Literature on the sales and profit trends for con- venience stores was also reviewed. It was estimated that the average business would lose about $22,113 in profit annually as a result of removing major-street driveway access. When this loss is combined with the annualized cost of adding a With respect to Question 1, an affected unit was defined as one corner at an urban or suburban signalized intersection, where the corner has a business with a driveway on the major street and frontage to the minor street. To determine the number of affected units in the nation, the number of urban and suburban traffic signals was first estimated using infor- mation published by the Institute of Transportation Engi- neers (ITE). The distributions of driveway count and land use at those intersections was then estimated through a sampling of 180 intersections in Texas. This sampling involved review- ing aerial photography to count and classify driveways by the land use type that they serve. The observed trend in the Texas data was extrapolated to the rest of the nation to yield an estimated count of 482,044 affected units. To answer Question 2, the target crash was defined as a driveway-related crash at an urban or suburban signalized intersection. Crash data from the sampled 180 Texas inter- sections were used to develop an SPF to estimate the number of target crashes at an intersection. Application of this SPF to an intersection with average traffic volumes (20,000 veh/d on the major street and 10,000 veh/d on the minor street) Factors TAP Assistance Number of candidate units Occasional assistance needed. When a proposed countermeasure is specific to a unique application and HSIS or HPMS are not helpful, then each TAP member is asked to provide an estimate of the count of candidate units (or the proportion of total units) in the jurisdiction they represent. Aggregation of all TAP input should reflect the entire United States. Number of affected units Each TAP member is asked to provide an estimate of the proportion of candidate units to which a countermeasure will likely be applied in the jurisdiction they represent. Aggregation of all TAP input should reflect the entire United States. Crash distribution No assistance needed. Cost of target crash No assistance needed. Cost of implementation Occasional assistance needed. For those countermeasures with no cost information available, each TAP member is asked to provide an estimate of the initial cost, annual maintenance cost, and service life of each proposed countermeasure. The number of these requests will likely be reduced over time as a “library” of this information is cataloged by the SAC. Estimated treatment effectiveness Occasional assistance needed. For those countermeasures with no previous research, an initial attempt will be made to use effectiveness estimates for similar treatments (e.g., estimates for shoulder rumble strip effectiveness as a guide for edgeline rumble strip effectiveness). When this is not possible, each TAP member is asked to provide an estimate of the mean effect of a countermeasure and the range outside of which the expected effectiveness of a proposed countermeasure is very unlikely. The members with some familiarity with the countermeasure will likely provide a reply. It is not necessary for all TAP members to provide this information. It is important to note that the research community could also be contacted to get insight into the safety effect of a countermeasure. Table 2. Information needed from Transportation Agency Partner (TAP).

16 • Determine the treatment cost and the limiting benefit cost ratio. • Determine the distribution of target crashes for the target miles. • Apply the procedure to estimate the value of conducting research on this topic. Following is a discussion of each step. Target Miles for Edgeline Rumble Strips (Rumble Stripes). For this example, it is assumed that the treatment is applicable to rural two-lane paved roads with either narrow paved shoul- ders (two feet or less in width) or unpaved shoulders (it was assumed that on roads with wider paved shoulders, shoulder rumble strips would be used instead of edgeline rumble strips). For the latter, the treatment would be located on the edge- line itself, extending into the travel lane unless the edgeline is moved slightly. The treatment could be applicable to other cat- egories or roadways (e.g., urban two-lane roads or rural undi- vided multilane roads with no/narrow paved shoulders), but the two-lane rural roads would appear to be the primary target. One possible source for estimating the number of target miles is HPMS. However, not all needed variables are included in the HPMS Universe or Sample files for all rural functional classes. Thus, assumptions are necessary. For example, as noted there, based on HSIS data from NC and MN, it appears valid to assume that all paved road mileage in the Rural Minor Collector and the Rural Local Roads categories are two-lane. Surface/Pavement Type is a Sample File variable and data are not available for Rural Minor Collectors or Rural Local Roads. However, based on supplemental data captured by HPMS from the states, the 2008 Highway Statistics contains the data shown in columns 2 – 4 of Table 3. These data were then used to calculate the percent of total mileage that is paved as shown in the final column. The HPMS data were then used to develop estimates of target miles for each functional class where data are available. Table 4 shows how that estimate was made. The second column in Table 4 provides the estimated num- ber of miles of two-lane paved roads with narrow or unpaved shoulders developed using the Universe and (weighted) Sam- ple variables. Note that no estimates exist for Rural Minor Collectors or Rural Local Roads since Sample File data do not exist for those classes. What are needed are estimates of mile- age for those two cells. The third column provides the total number of paved miles in each rural functional class extracted from Table 3. The fourth column provides the percent of total paved miles which are target miles for the three classes where HPMS Sample data were available. Based on the trend shown there, the fifth column shows project team estimated percentages of the percent of paved roads expected to have narrow paved or minor-street driveway (for businesses requiring this mitiga- tion), the annualized cost of the treatment was estimated as $22,785 per year. To address Question 5, the limiting benefit/cost ratio was established at 2.5, a value that is often used by agencies when choosing among alternative projects. Questions 6 and 7 were answered using the SPF that was calibrated using the sample of intersections from Texas. (This SPF was previously used to answer Question 2.) Application of the SPF revealed that if a corner business’ major-street drive- way is removed and replaced with a new driveway on the minor street, the expected injury and fatal crash frequency drops to 0.366 crashes per year. Similarly, if the business previously had minor-street driveway access and its major-street access is removed, the expected injury and fatal crash frequency drops to 0.244 crashes per year. These two cases yield CMFs of 0.648 and 0.431, respectively. Further statistical analysis yields stan- dard deviations of 0.277 and 0.181 for the two CMFs. Using the aforementioned information and the VOR cal- culation procedure summarized earlier and discussed further in Appendices E, F, and G, the VOR for removing major-street driveway access (and replacing it with minor-street driveway access if needed) is $18,950,000. This number represents the estimated benefit that would be realized if the following events occur: • The suggested new research on this treatment is conducted to develop CMFs to quantify the expected crash frequency reduction, • All 482,044 affected units are evaluated for potential treat- ment, and • The units having sufficiently high crash frequencies to jus- tify the treatment (based on the limiting benefit/cost ratio of 2.5) are treated accordingly. It is noted that only about five percent of the affected units are likely to have sufficiently high crash frequencies to need treatment. Evaluation of Proposed Research on the Effectiveness of Edgeline Rumble Strips The VOR approach was used to determine the value of conducting research to develop a CMF for edgeline rumble strips. This approach involved the following steps: • Estimate the number of target miles in the nation for this treatment. • Determine the CMF and the standard deviation of the CMF based on previous studies of this treatment. • Estimate the standard deviation after the new research is conducted.

17 CMF for this Treatment from Previous Studies. A search of FHWA’s CMF Clearinghouse did not find any CMFs for edgeline rumble strips on rural two-lane roads. However, NCHRP Report 641: Guidance for the Design and Application of Shoulder and Centerline Rumble Strips), provides CMFs for shoulder rumble strips on rural two lanes. The recommended CMF from that report for single vehicle run off road crashes (SVROR) was 0.85. The standard deviation of the CMF was estimated to be 0.126. Standard Deviation of the CMF after the New Research Is Conducted. Based on procedures described in Appendix D, the standard deviation of the CMF after the new research is completed was estimated to be 0.089. Treatment Cost and Benefit Cost Ratio. Based on infor- mation from a couple of states, the initial cost for installing edgeline rumble strips was assumed to be $2,000 per mile with an annual maintenance cost of $500 per mile. The life of unpaved shoulders for road classes 8 and 9. (Note that these are simply estimates and could be changed if better informa- tion existed.) The next column then carries over the second- column mileage for the three classes where HPMS data exist and multiplies the estimated percent in column five times the total paved miles in column three for the latter two classes. These are then summed to provide the estimated target miles of 1,216,000 miles nationwide. The last column shows the percentage of total miles that are target miles in each category. The final adjustment would be to decrease these estimates in each functional class if there were known to be substan- tial miles where the treatment has already been implemented. Project staff is aware of some implementation for state-system mileage in Washington, Minnesota, and Pennsylvania. How- ever, for this example, our feeling is that the percent of miles already treated will be very small (or zero) in the final two roadway classes where the bulk of the mileage is. Thus, there were not adjustments for treated miles in this case. This, too, would be a simple change to make. Functional Class Paved Unpaved Total % Paved 1 Rural Interstate 28,846 - 28,846 100.0% 2 Rural Other Prin Arterials 94,845 - 94,845 100.0% 6 Rural Minor Arterials 134,900 - 134,900 100.0% 7 Rural Major Collectors 380,644 37,814 418,458 91.0% 8 Rural Minor Collectors 179,622 83,227 262,849 68.3% 9 Rural Local 881,206 1,157,311 2,038,517 43.2% 11 Urban Interstate 16,442 - 16,442 100.0% 12 Urban Other Freeways/Exp. 11,327 - 11,327 100.0% 14 Urban Other Princ. Arter. 64,745 - 64,745 100.0% 16 Urban Minor Arterials 106,432 510 106,942 99.5% 17 Urban Collector 113,762 1,113 114,875 99.0% 19 Urban Local 719,838 43,775 763,613 94.3% TOTAL 2,732,609 1,323,750 4,056,359 67.4% Source: 2008 Highway Statistics, Table HM-51 of Section 4.4.4 – Arterials and Collectors. Table 3. Paved and unpaved miles by functional class. Table 4. Calculation of estimated number of target miles. Functional Class Miles With Narrow Paved or Unpaved Shoulders Total Paved Miles 2-Rural Principal Arterial Other 8,050 94,845 6-Rural Minor Arterial 33,891 134,900 7-Rural Major Collector 202,197 380,644 % of Total Paved Miles 8% 25% 53% 8-Rural Minor Collector 179,622 9-Rural Local 881,206 Estimated % for Class 8 and 9 75% 95% TOTAL RURAL Estimated Target Miles 8,050 33,891 202,197 134,717 837,146 1,216,000 Total % Miles that are Target 8.5% 25.1% 48.3% 51.3% 41.1% 29.98%

18 TRBNetProjectDisplay.asp?ProjectID=2727. Thus, the team developed another quantitative approach to rate and rank studies that deal with more fundamental (basic) research that do not directly produce CMFs. The following is a discussion of that method that was developed and applied in this study for ranking fundamental and non-CMF applied RNSs. Research Prioritization Methodology Background Research needs were identified through several efforts and from several sources as discussed in Chapter 3. Not all of the identified research needs were applicable and an initial screen- ing was conducted to eliminate projects that did not focus on infrastructure-related topics. Research needs were further divided based on the relation to decision-making. As discussed earlier, there are two types of research con- sidered in this report: applied research and fundamental research. (Also, as discussed earlier, other types of studies including non-research implementation studies were dis- cussed, but not included in the prioritization, because they were considered outside the scope of this effort.) Applied research relates directly to decision-making in the road- way safety management process and includes three primary areas: (1) research to estimate CMFs, (2) research to estimate expected future crashes, and (3) research to estimate the value of target crashes. The intent of applied research is to reduce the uncertainty about any of these three factors that determine the safety benefit of a specific strategy. Research that makes applied research feasible or improves it is termed fundamental research and includes efforts such as those con- ducted to improve statistical methods or to better understand the underlying contributing factors of crashes. This section focuses on a prioritization process for funda- mental research needs as well as non-CMF applied research needs (i.e., research to estimate expected future crashes or the value of target crashes). These research projects cannot be compared on a common basis with CMF-related applied research as the prioritization criteria used for CMF-related applied research are not as easily nor accurately quantified for Fundamental and Non-CMF Applied Research projects. As such, the research projects cannot be compared on the same relative scale and it may be necessary to designate separate funding pools for applied and fundamental research pro- grams. Note that in TRB Special Report 261: The Federal Role in Highway Research and Technology, FHWA’s Research and Technology Coordinating Committee recommended that at least one-quarter of FHWA’s Research and Technology bud- get should be invested in fundamental, long-term research. The objective of this project is not to establish this split, but instead, focus on the prioritization within each research type. the milled rumble strip was assumed to be 5 years. The limit- ing benefit cost ratio was assumed to be 2.5. Distribution of Target Crashes. SVROR were the pri- mary target crashes. Data from Minnesota and North Caro- lina were extracted from the HSIS for rural two-lane roads with unpaved shoulders or narrow paved shoulders (2 feet or less). North Carolina was chosen because this state (compared to most of the other HSIS states) includes a significant num- ber of miles from local or minor collector roads within their HSIS files. Since a significant number of rural two-lane miles within the nation are local or minor collector, including these roadway types was important. Negative binomial regression was used to develop SPFs using Annual Average Daily Traffic (AADT) as the independent variable. The following SPF was estimated using data from North Carolina. Only rural two- lane roads with unpaved shoulder or paved shoulder less than or equal to 2 feet were included in estimating the SPF: ( ){ }= − + ∗SVROR Crashes per mile exp 3.591 0.3662 AADTln This SPF was entered into the tool developed in this effort (see Appendices E, F, and G) to determine the VOR. When SPFs from North Carolina were used for estimating the crash distribution for the nation, the VOR was estimated to be $174,742,763. For this estimation, it was assumed that only 50% of miles identi- fied earlier in Table 4 could be treated (the 50% number is arbi- trary and could be modified if further information is available from the states or other entities such as the TAP). Rating Fundamental and Non-CMF Applied Research Topics The term “fundamental research” refers to various types of basic, non-CMF research. The premise of this work is that safety research has value if it makes for better decisions and if it helps to get more crash reduction per invested dollar. The monetary VOR is the amount by which its results can increase the net benefit (difference between the benefit and cost) of safety-related projects. The proposed method for estimating the monetary value of CMF research was discussed earlier, but it was not clear how to do the same for non-CMF safety research. This is primarily because fundamental research studies generally do not involve quantifying CMFs, and spe- cific countermeasures are often not identified in the research problem statement, so it is difficult or impossible to be able to estimate countermeasure-related safety benefits for study topics where the countermeasures are unknown. A detailed discussion of some of the initial thinking and analysis on quantifying the VOR for fundamental (non- CMF) research topics is given in Appendix I, which is avail- able on the TRB website at: http://apps.trb.org/cmsfeed/

19 target crash population would be all crashes because improved CMFs could be developed for any crash type. b. Data source: General Estimates System (GES) database is used to estimate the number of crashes for various target populations. In the event that GES is not able to provide the desired input, the HSIS may be an alterna- tive source. HSIS provides detailed-level data for select states. Note, however, that HSIS does not provide a nationally representative sample and it would be nec- essary to adjust any numbers obtained from HSIS to estimate national numbers. c. Process: The GES crash numbers were generated using query definitions from the 2010 GES data set. The NASS GES Analytical Users Guide was used to determine all possible factors related to the crash type, area type, and facility type for each research need statement. GES data do not have a field for functional classification or area type. For all research need statements that consider dif- ferent area types (i.e., rural, urban) and facility types (e.g., highways, freeways, intersections, etc.) a calibra- tion factor was used to determine the number of crashes in 2010. The calibration factors were based on 2010 Fatal Accident Reporting System (FARS) data. For example, the pedestrian-related urban crashes calibration factor is the number of pedestrian-related urban fatal crashes divided by the number of total pedestrian-related fatal crashes. In this example, the calibration factor was then multiplied by the number of pedestrian-related crashes generated from the GES query. d. Score: VOR increases as the number of target crashes increases. 2. Severity of Expected Target Crashes a. Definition: At a national level, the current focus is on fatal and severe injury crashes. While the previous fac- tor favors research that targets large crash populations, this factor favors research that focuses on fatal crashes. This factor identifies the target crash population of the proposed research and estimates the number of fatal crashes associated with the given population. b. Data source: FARS is used to estimate the number of fatal crashes for the target population. In the event that FARS is not able to provide the desired input, the HSIS database may again be an alternative source. Recall that HSIS would not provide a nationally representative sample and it would be necessary to adjust any num- bers obtained from HSIS to estimate national numbers. c. Process: The FARS fatal crash and fatality numbers were generated by querying FARS 2010 data. The FARS Analytical Reference Guide was used to determine all possible factors related to the crash type, area type, and facility type for each research need statement. Not all the Objective This section presents a method for prioritizing fundamen- tal research and non-CMF applied research projects that do not rely solely on expert/practitioner experience/judgment. First, several issues are identified that are related to the priori- tization of research. Next, factors are identified and defined for use in the prioritization process. A weighting scheme is then presented to combine the factors and compute a score for each potential research project. The process was pilot tested for a sample of the research needs and compared to the results of a traditional expert panel; these results are presented and discussed. Finally, the process was applied to the complete list of fundamental research and non-CMF applied research proj- ects. A prioritized list is provided for further consideration by future funding agencies. The list is also separated by funda- mental research and non-CMF applied research projects in the case that these projects are to be considered separately. Selection Criteria For fundamental research and non-CMF applied research, the VOR depends on several factors. The following list identifies eight quantifiable factors that should be considered when eval- uating fundamental research and non-CMF applied research programs for potential funding. It is understood that other fac- tors may influence the VOR, but these factors were identified as the most critical. Also, it was determined that these factors can be applied in a practical application of the method (i.e., factors can be quantified through existing data sources or stakeholder input). Limited stakeholder input is included in several factors (i.e., 3, 4, and 5) through the use of an expert panel or SAC. It is proposed that more general stakeholder input be included once the prioritized list of fundamental research has been developed. This is discussed at the end of the section. For each factor, the definition and data source is provided along with the process for obtaining and applying the infor- mation. Note that each criterion is intended to be mutually exclusive to avoid double-counting considerations. Prior to quantifying the factors, it is useful to determine the target crashes and applicable area type and facility type to help pro- vide a frame of reference for each potential project. 1. Number of Expected Target Crashes a. Definition: This factor identifies the target crash pop- ulation of the proposed research and estimates the number of total crashes associated with the given pop- ulation. For example, the target crash population for fundamental research related to pedestrian treatments at intersections would be pedestrian-related crashes at intersections. If the fundamental research is focused on improving statistical methods for developing CMFs, the

20 Note: the expert panel should only consider the poten- tial to impact the science of safety in this step without consideration of other factors such as project cost and probability of success as these are covered by other selec- tion criteria. d. Score: Research that produces many peripheral prod- ucts that support many different types of decisions would be given higher priority than research that has a limited focus. 4. Potential to Improve Existing Information for Target Crashes a. Definition: The ultimate goal of safety research is to help state and local agencies more effectively reduce the fre- quency and severity of crashes through new or improved tools/methods/information. The three primary areas that influence decision-making are CMFs (i.e., estimate of change in crashes given treatment); expected crashes (i.e., estimate of crashes without treatment); and crash costs. This factor focuses on the potential to improve decision-making as a result of the research. b. Data source: An initial estimate should be identified by the individual/agency proposing the research with writ- ten support for their estimate. A literature search could be conducted by the proposing individual/agency to identify existing research related to the target crashes. The final decision is a consideration for an expert panel such as the SAC (see Chapter 4). c. Process: The initial estimate will be collected during future solicitations for research needs. This project identified existing research needs that did not include a specific discussion of the potential to improve existing information for target crashes. As such, the project team based their decision on the following considerations and rated each potential project on a scale of 1 to 5 where 5 represents the highest potential for improvement. i. Limitations of existing knowledge – The HSM was used to identify limitations of existing knowledge, including methods for estimating the expected num- ber of crashes, CMFs, and general safety decision- making procedures. ii. Quality of existing information – The HSM and CMF Clearinghouse were used to help judge the quality of existing information. The main body of the HSM presents the “best available” CMFs for specific design and operational features for each facility type, but also includes a knowledge base. The knowledge base is more inclusive, and indicates the relative quality of each CMF. The Clearinghouse provides a star rating for each CMF on a scale of 1 to 5, where 5 stars is the highest possible rating. If there are several CMFs related to a specific topic and all are of sufficient quality, the research may not variables for the RNSs are captured in the FARS data- base, so assumptions were made to narrow down the FARS variables to resemble the research need factors as close as possible. d. Score: VOR increases as the number of fatal crashes increases. 3. Extent of Impacts on the Science of Safety a. Definition: The goal of most fundamental research is to advance the science of the particular field as opposed to providing directly applicable solutions for immediate implementation. Fundamental research may include a number of topics and some may have wider applica- bility than others. This factor focuses on the potential for a specific research project to advance the science of safety with respect to the number of decisions that could be improved as a result of the research. In this way, fundamental research is like the trunk of a tree and the decision to implement a specific treatment is the leaf at the end of a branch. Trees with larger trunks can support more branches and thus more leaves. b. Data source: An expert panel provides insight on the potential breadth of the research results (i.e., size of the tree trunk). The inclusion of non-researchers (e.g., state DOT representatives) on the expert panel would provide more comprehensive representation of the user community; however, few non-researchers have a good understanding of fundamental research needs and long-term potential. As such, this factor should include input from select “experts” within the trans- portation community, with significant representation from researchers. c. Process: Each member of the panel receives a list of potential research topics and provides feedback, including a score for each topic (i.e., rate the potential impact on a scale of 1 to 5 where 5 represents the high- est level of impact). Using the tree-and-leaf example from above, the expert panel will be responsible for determining the size of the tree trunk relative to other research projects and the number of leaves each tree can support. This step requires a fair amount of judg- ment and insight on behalf of the experts. As such, it is critical that the potential research topics provide detailed information about the intent and applica- bility of the research. One method to help reviewers define the size of the trunk (i.e., extent of the impact) is to list the specific actions, processes, and decisions that would then be possible (or more effective) if the fundamental research is successful (i.e., what applied research could be conducted or would be improved as a result of the fundamental research?). For the problem statements in this study, the project team served as the expert panel and awarded points.

21 relatively expensive research even though it has a high potential for improving safety. Another option is to use the cost as a factor in the overall weighting scheme. This is the option that the research team has employed. b. Data source: The estimated cost of research should be identified by the proposing individual/agency. The cur- rent template for identifying research needs includes a section for estimated cost. c. Process: This information will be collected during future solicitations for research needs. Cost estimates were not available for many of the current research needs considered in this project, so the project team estimated relative costs as discussed in the next section, Weighting Scheme. d. Score: The VOR increases as cost decreases. 7. Potential to Identify More Effective Strategies for Target Crashes a. Definition: The ultimate goal of safety research is to help state and local agencies more effectively reduce the fre- quency and severity of crashes through new or improved tools/methods. This factor focuses on the potential to identify new and more effective countermeasures. If relatively effective countermeasures have already been developed to address specific crashes, then additional research may not be justified in that area. b. Data source: The CMF Clearinghouse is used to iden- tify existing countermeasures related to each research topic and the relative effectiveness of the related coun- termeasures. If a countermeasure is not included in the CMF Clearinghouse, it can be assumed that the safety effect has not yet been determined. c. Process: All proposed research projects are reviewed to identify target crashes and the applicable area type and facility type. The CMF Clearinghouse is then queried to identify existing countermeasures and their relative effectiveness (i.e., CMF). The CMF is an indication of the potential effectiveness of that measure. If all counter- measures related to a specific research topic are relatively ineffective (e.g., CMFs close to 1.0), then research to develop new countermeasures in this area may be more favorable than further research on countermeasures that have already proven to be very effective. The specific steps employed by the project team include the following. i. Query CMF Clearinghouse – search by keywords or countermeasure type and screen the list using crash, roadway, and vehicle characteristics to iden- tify applicable countermeasures. ii. Eliminate CMFunctions – Crash modification func- tions (CMFunctions) were eliminated from the list as their value changes depending on the character- istics of the specific location to which the counter- measure will be applied. be as important as research that has the potential to improve several CMFs of relatively low quality. d. Score: VOR increases as the quantity and quality of existing information decreases (i.e., increased potential for improvement). 5. Probability of Success a. Definition: This factor identifies the likelihood of the research to produce useful results (i.e., the result of the research will produce more value per unit cost in applied research projects). This factor is based on the likelihood of satisfactory completion of the project. A project is “successful” if it finds meaningful informa- tion with a high degree of certainty. The information could be a new method, confirmation of the adequacy of an existing method, or an indication that an exist- ing method is, in fact, not appropriate for specific circumstances. Note: This factor does not include considerations of bud- get or time constraints as these are considered in Factor #6. It also disregards institutional obstacles, inertia, etc., for implementing the results and fundamental research is deemed successful if its results enable us to conduct applied research projects or to conduct them more effectively. b. Data source: An initial estimate should be identified from the individual/agency proposing the research. The final decision is a consideration for an expert panel such as the SAC (see Chapter 4). c. Process: The initial estimate will be collected during the solicitation for future research needs. While it may be difficult to clearly define the probability of success for an individual project, the relative probability of success could be judged by an expert panel reviewing all proposed research projects at the same time. For this project, the team based their decision on the following factors: i. Complexity of research – Is the research so com- plex that it is likely to fail? ii. Existing capabilities – Is it possible to conduct the research with existing skills, methods, data, and resources OR does it first require the development and implementation of sophisticated data collec- tion equipment/techniques? iii. Results – Will the research produce a quanti- fiable improvement in the certainty of safety information? d. Score: The VOR increases as the probability of success increases. 6. Cost of Research a. Definition: This is the estimated cost to complete the proposed research project. One option is to divide the overall VOR for each potential project by the estimated cost of research. This option, however, may overlook

22 a great number of safety decisions than research that is fewer levels removed. However, this factor only looks at the distance between the research and treatment-related decisions. The potential impact is addressed under Fac- tor #3 (Extent of Impacts on the Science of Safety). b. Data Source: An expert panel reviews all proposed research projects to extract the needed project descriptions. c. Process: An expert panel determines the “level” of research using the following categories. i. Non-CMF applied research. ii. Fundamental one level removed. iii. Fundamental two or more levels removed. d. Score: VOR increases as the number of levels decreases. Note that this assumes all else is equal. Weighting Scheme The weighting scheme discussed in this section is based on the premise that all factors identified in the previous sec- tion will contribute to the overall VOR. This is not to say that all factors should carry equal weight. For example, the levels between the research and final decision-making may not be as important as the potential impact on the target crashes and crash severity. This section attempts to identify the potential values for each factor and the relative importance of each fac- tor in the development of an overall weighted utility index. Values for each factor are listed below, but note that the ranges and values are highly flexible and may be revised as the method is employed and evaluated. 1. Number of Expected Target Crashes a. Score 1–5 based on related crash frequency. The num- ber of target crashes is identified for each topic and sorted from greatest to least. i. 1 = topics in the lowest 10 percent. ii. 2 = topics between 10 to 30 percent. iii. 3 = topics between 30 to 70 percent. iv. 4 = topics between 70 to 90 percent. v. 5 = topics in the top 10 percent. 2. Severity of Expected Target Crashes a. Score 1–5 based on severity. The number of fatal and injury target crashes is identified for each topic and sorted from greatest to least. i. 1 = topics in the lowest 10 percent. ii. 2 = topics between 10 to 30 percent. iii. 3 = topics between 30 to 70 percent. iv. 4 = topics between 70 to 90 percent. v. 5 = topics in the top 10 percent. 3. Advancement of Science of Safety a. Score 1–5 based on stakeholder input and ranking. i. 1 = lowest impact. ii. 5 = highest impact. iii. Compute Relative Effectiveness – the result from the query is a list of CMFs for the related counter- measures. The list may include CMFs greater than, less than, or equal to 1.0. To ensure that CMFs greater than and less than 1.0 do not cancel-out, the absolute effect was computed for each counter- measure. For example, a CMF of 0.7 and 1.3 both indicate a 30 percent change in crashes, but the for- mer indicates a decrease while the later indicates an increase. For this project, any numbers greater than 1.0 were converted to the equivalent change indi- cated by a CMF less than 1.0. For example, a CMF of 1.3 would be converted to 0.7. In some cases, the CMF is greater than 2.0, indicating an increase of more than 100 percent. The conversion would result in a negative number and these CMFs were truncated at zero to avoid negative numbers. iv. Compute Average – the average effectiveness is com- puted for the list of countermeasures associated with each research project and the average effectiveness is used to compare the potential to identify more effective strategies. d. Score: This score pertains specifically to research focused on the identification of new countermeasures or research that would allow for the identification/analysis of new countermeasures. VOR increases as the average effective- ness of existing treatments decreases (i.e., average CMF close to 1.0). A value of 1.0 is assumed when existing strategies are not identified for a given research project. 8. Distance Between Expected Research Results and Treatment-Related Decisions a. Definition: Unlike CMF research that can produce an improved CMF that can immediately lead to a better safety decision, fundamental research develops instru- ments which can lead to better decision-making (i.e., better CMFs, crash costs, and methods for estimating expected crashes). Thus, fundamental research is one or more “levels” removed from actual safety decisions. Some fundamental research is one level removed (e.g., research to develop a crash surrogate to allow CMF research on pedestrian crossing treatments; research to develop bet- ter modeling techniques to allow extraction of CMFs from regression coefficients; research to develop models that will be used to estimate future target crashes). Other fundamental research is two or more levels removed (e.g., phased research, research to develop a tool which will compare different regression modeling techniques and determine which is better; research to develop a bet- ter screening tool to choose locations in need of further study). It may be the case that research that is further removed from actual safety decisions (i.e., more lev- els) actually has the potential to enhance and influence

23 ii. 2 = average effectiveness (i.e., CMF) of related countermeasures is 0.51 to 0.74. iii. 3 = average effectiveness (i.e., CMF) of related countermeasures is 0.75 to 0.84. iv. 4 = average effectiveness (i.e., CMF) of related countermeasures is 0.85 to 0.94 v. 5 = average effectiveness (i.e., CMF) of related countermeasures is 0.95+. 8. Distance Between Expected Research Results and Treatment-Related Decisions a. Score 1–3 based on the number of levels from treatment-related decisions. i. 1 = research that is three or more levels from treatment-related decisions. ii. 3 = research that is two levels from treatment- related decisions. iii. 5 = research that is one level from treatment- related decisions. Figure 3 illustrates the relative importance of the factors. The level of relative importance increases on the horizontal axis from left to right. In this case, number of target crashes is identified as the most influential factor. As such, it would receive a greater weight in the overall utility index than other factors. Based on the level of importance identified in Figure 3, the following weighting scheme is proposed to estimate the relative utility index (RUI) for fundamental research. The weighting scheme is based on four tiers of relative importance with two factors per tier. The first tier includes the number and sever- ity of target crashes. This tier provides a top-down approach to the prioritization (i.e., problem-centric). The second tier is an attempt to balance the problem-centric input with practi- cal needs and potential to advance the science of safety. This 4. Potential to Improve Existing Information for Target Crashes a. Score 1, 3, or 5 based on potential as determined by an expert panel such as the SAC. i. 1 = limited potential for improvement. ii. 3 = moderate potential for improvement. iii. 5 = significant potential for improvement. 5. Probability of Success a. Score 1–5 based on relative chance of success as deter- mined by an expert panel such as the SAC. i. 1 = low chance of success. ii. 2 = medium-low chance of success. iii. 3 = average chance of success. iv. 4 = medium-high chance of success. v. 5 = high chance of success. 6. Cost of Research a. Score 1–5 based on overall cost of research. i. 1 = $5M+. ii. 2 = $1M to $5M. iii. 3 = $500,000 to $1M. iv. 4 = $100,000 to $500,000. v. 5 = <$100,000. 7. Potential to Identify More Effective Strategies for Target Crashes a. Score 1–5 based on the average effectiveness of existing countermeasures related to the research topic. This score pertains specifically to research focused on the identifica- tion of new countermeasures or research that would allow for the identification/analysis of new countermeasures. i. 1 = average effectiveness (i.e., CMF) of related countermeasures is less than 0.50 OR research is not related to the identification of new countermeasures OR research does not allow for the identification/ analysis of new countermeasures. Extent of Impacts on Science of Safety Level of Importance in the Overall Utility Index Probability of Success Number of Target Crashes Potenal to Improve Informaon Potenal to Idenfy More Effecve Strategies Distance between Results and Decisions Cost of Research Fa ct or s f or U til ity In de x Severity of Target Crashes Figure 3. Relative importance of factors to be included in utility index.

24 tions to rank them by assigning a number between 1 and 20 to each of the research statements, where 1 is the highest priority. It should be noted that the list was randomized for each panel member to address potential order bias. Figure 4 and Figure 5 present the results of the pilot test, comparing the VOR from the RUI with the results from the panel review. The research team received responses from six panel members; the individual and average panel rankings are presented in Figure 4, while Figure 5 shows the average and standard deviation of the panel rankings. The following points are based on a comparison of the RUI and panel rankings as well as general observations from the application of the RUI. • General correlation: There is a slight increasing trend in the average panel rankings when the research needs are sorted by RUI ranking. This indicates that the RUI is rela- tively consistent with the general responses from the panel. • Variability: There is substantial variability in the panel rankings as shown by the individual points in Figure 4 and the standard deviation bars in Figure 5. While the RUI does include inputs from an expert panel or SAC, it also incor- porates several quantitative factors that help to reduce the variability in the results. • Limited detail: There was limited information provided by the author for several of the RNSs. This made it difficult to clearly define the eight factors for the RUI, particularly those that required input from the SAC. In the future, these RNSs would be returned to the author for additional information (convert them to projects, as per earlier para- graphs) before they can be included in the ranking process. Summary and Conclusions on Recommended Prioritization Methods One of the major objectives of this project was to develop prioritization methods to rank and rate research projects. For this purpose, research studies have been divided into two cate- gories: (1) research studies that develop CMFs, and (2) research studies that do not directly deal with the development of CMFs (also referred to as fundamental research). The prioritization method for research studies that develop CMFs is based on the VOR approach. This approach is more objective compared to other methods that are currently being used by agencies to rank research projects. It relies on data to develop the priorities rather than having priorities based on the subjective opinions of individuals and stakeholder groups. The underlying basis for the proposed approach is that all research projects have one goal – to develop information for making a better safety-related decision (i.e., information is accomplished through stakeholder input (a bottom-up approach) and identification of knowledge gaps. The third tier incorporates the probability of success and cost of a proposed research topic; both are important factors, but should not drive the prioritization of fundamental research. The fourth tier includes the potential to identify more effective strategies and the distance between expected research results and treatment- related decisions. These additional factors will not influence the higher-level prioritization, but can help to refine the final list of prioritized topics when two or more topics are assigned the same score based on the previous factors. = + + + + + + +4 4 3 3 2 2RUI TC SC SS QI PS CR ES DR Where: TC = Number of Expected Target Crashes. SC = Severity of Expected Target Crashes. SS = Extent of Impacts on Science of Safety. QI = Potential to Improve Quality of Information for Tar- get Crashes. PS = Probability of Success. CR = Cost of Research. ES = Potential to Identify More Effective Strategies for Target Crashes. DR = Distance between Expected Research Results and Treatment-Related Decisions. The RUI is based on a total possible score of 100. If each factor received the maximum possible score, ( ) ( ) ( ) ( ) ( ) ( ) = + + + + + + + = + + + + + + + = 4 4 3 3 2 2 4 5 4 5 3 5 3 5 2 5 2 5 5 5 100. RUI TC SC SS QI PS CR ES DR Pilot Test of Prioritization Method The RUI and associated weights were based on a review of the literature and careful consideration of the application of other prioritization methods in the transportation field; how- ever, there was a need to test the proposed method. A formal verification of the RUI is difficult because there is no baseline truth to which the results can be compared. Instead, the research team devised a plan to pilot test the method, using a sample of 20 fundamental research problem statements. The sample included a range of projects, varying by topic area, research costs, and anticipated timeframe. The same set of problem statements was distributed to the NCHRP panel for review and prioritization using a traditional “expert panel” ranking (i.e., preference based on personal experience and understanding of the underlying issues and research needs). Specifically, the 20 RNSs were distributed to each panel member with instruc-

25 Figure 4. Value of fundamental research (VOFR) versus average panel score (Individual Points). Figure 5. Value of fundamental research (VOFR) versus average panel score with standard deviations.

26 studies that need to be considered in this project, it may be necessary to develop a prescreening tool to reduce the num- ber of studies that can be examined using the computerized tool to a manageable number. Regarding the prioritization of non-CMF research, two methods have been proposed. The first approach is more quantitative, but we felt that it was very difficult to implement. Hence, we developed an alternative method that is less quan- titative but easier to apply in practice. The alternative method identified the following eight different factors that could be considered in rating a non-CMF research study: • Number of expected target crashes. • Probability of success of the research study. • Stakeholder input. • Cost of research study. • Potential to Improve Quality of Existing CMFs for Target Crashes. • Potential to Identify More Effective CMFs for Target Crashes. • Number of Existing CMFs for Target Crashes. • Distance between Expected Research Results and Treatment-Related Decisions. Based on the relative importance of each factor, a weighting scheme has been developed to weigh the individual factors and compute the utility of a non-CMF study. This method was pilot tested and was found to be reliable for prioritizing non-CMF studies. It is noted that while the proposed factors and weighting scheme have undergone initial review, both can be modified as more is learned from future usage of the method. (See the Lessons Learned section in Chapter 3 for further discussion.) that will reduce the chances of making an inferior decision). Safety-related decisions involving CMFs concern whether or not to implement a given treatment on a specific set of road- way locations. The standard deviation of the CMF is a measure of the uncertainty of the CMF. The smaller the standard deviation of the CMF, the lower the chance of making an inferior deci- sion based on this CMF. If by conducting a research project it is possible to substantially reduce the standard deviation of the CMF, then one could argue that such a research proj- ect has good value. This approach develops a monetary value based on the extent to which a research project can reduce the standard deviation of a CMF and thus increase the chances of making the correct decision. Appendices C and D provide a detailed discussion of the theoretical underpinnings of this method. Appendices E, F, and G provide a discussion of a computerized tool that has been developed in order to imple- ment this method. These appendices are not published herein, but can be accessed on the TRB website at: http://apps.trb. org/cmsfeed/TRBNetProjectDisplay.asp?ProjectID=2727. Two pilot studies were conducted to test the implementa- tion of this method. One pilot involved the estimation of the value of conducting research to determine the CMF for clos- ing driveways at intersections. The second pilot involved the estimation of the value of conducting research to determine the CMF for implementing edgeline rumble strips on rural two-lane roads. Based on these pilot studies it is clear that this approach can be implemented for determining the value of CMF-based research. The intent is to use about 3 to 4 hours to compile the data for each CMF-based research study and apply the tool discussed in Appendices E, F, and G to estimate the VOR. Depending on the number of CMF-based research