Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
15 Chapter 2. Survey of Practitioners Initially, the project team envisioned gathering the NCHRP Project 17-78 panelâs opinion on the model and application-related factors. Based on the kickoff call, the project team agreed to expand and conduct a survey of a broader group of practitioners. In the kickoff call, it was proposed that the project team focus more on application-related factors. The project team identified a list of practitioners for the survey by contacting FHWA Office of Safety and AASHTOâs Senior Engineering Program Manager for Safety. These practitioners included members of the AASHTO Safety Management subcommittee, AASHTO HSM2 Steering Committee, and FHWA HSM Pooled Fund Group. Following are the details of the survey, including the results. Twenty practitioners responded to the survey. The first page of the survey provided the following background about the survey, and this shown below: Crash prediction models (CPMs), also called safety performance functions, are models used to predict crash frequency based on site characteristics. They may come with or without crash modification factors (CMFs) that adjust the prediction to account for site conditions that are different from the base condition of the model. Many CPMs are presented in Part C of the Highway Safety Manual. You are invited to share your experience as a user of crash prediction models (CPMs) by responding this short questionnaire, needing approximately 10 minutes. Your response will help direct the NCHRP 17â78 project team to address the priority needs of safety practitioners. Brief Background The NCHRP 17â78 project, recently launched, has three main objectives:  1. Develop guidance for the quantification of the reliability of crash prediction models (including crash modification factors (CMFs), crash modification functions and safety performance functions (SPFs)) for practitioner use. 2. Develop guidance for user interpretation of model reliability. 3. Develop guidance for the application of CPMs accounting for, but not limited to assumptions, data ranges, and intended and unintended uses. The project team will greatly benefit from the feedback from users of crash prediction models, like yourself. Your responses will help us in identifying the dayâtoâday needs and high priority gaps in understanding and communicating reliabilityârelated factors or aspects, and will be instrumental in developing pragmatic and needed guidance to practitioners, like yourself.  The second page asked the respondents to indicate whether a particular issue regarding CPMs were âVery concerningâ, âSomewhat concerningâ, âNeutralâ, âNot very concerningâ, or âNot a concernâ. The respondents were asked to respond to seven issues. Following is the list of questions along with the summary of the results.
16 Potential Concerns in Applying CPMs When you use a CPM to what level does each of these potential issues concern you?  "Concern" in this usage refers to your level of trust in the results of the CPM predictions for your own use as well as your concerns about the ease of communicating the reliability and results of the CPM to your colleagues, managers, and the public. Some of these potential concerns may not be applicable to you because your data does not present that type of concern or you have other ways of handling it. In those cases, mark your answer as "Not a concern".Â
17 Table 3. Results of Practitioner Survey. Issue Number of Responses Very concerning Somewhat concerning Neutral Not very concerning Not a concern Did not Answer Using CPMs that were developed in another location but calibrated for your local jurisdiction 0 7 2 4 7 0 Using crash modification factors that were not included in the original CPM but are consistent with the base conditions of the CPM 1 4 4 9 1 1 Using crash modification factors that were not included in the original CPM and are inconsistent with the base conditions of the CPM 7 7 4 1 1 0 Using a CPM that does not represent the characteristics of your project (e.g., using a 4-lane road CPM to predict crashes for a 6-lane road project) 13 6 0 0 1 0 Using input values that are uncertain (e.g., estimated AADTs based on old counts, crashes with unverified locations, etc.) 4 9 6 1 0 0 Using a CPM for a project whose characteristics lie outside the range of values of the CPM development (e.g., AADT volumes higher than the maximum value documented for the CPM) 7 8 3 2 0 0 Using a CPM to estimate rare crash types, such as fatal crashes or pedestrian crashes 6 5 4 5 0 0 Finally, the respondents were asked to answer the following question in their own words (the answers are provided below the question): Question: Are there any other concerns you have when communicating the reliability or application of CPMs (data, values used for variables, estimates calculated, and to your decision to adopt or not to adopt a given CPM and CMFs) to your colleagues, managers, or the public? Please describe.Â
18 Answers: ï· The biggest difficulty is explaining what a CPM is and why one would use a CPM instead of pure naive prediction. If they do accept it, they tend to overrate the reliability of the CPM. Some prefer whatever predictive method gives the results they want, regardless of which is most accurate or applicable. ï· Communicating the value of CPM (or SPF) that predict a very low number of crashes, such as those for very specific conditions, e.g., pedestrians, bicycles, rail road crossings, specific geometric conditions, etc. ï· Any CMF that has a broad range of standard deviation, especially those that are not statistically significant. I don't put much trust in those CMFs. ï· Too many input parameters. ï· It is important to interpret the data and describe what the data represents. Typically, we describe these predictions as planning level estimates for what we expect safety to look like in some future scenario based on the specific variables that were captured in a specific CPM. This can be difficult to explain, because there are so many variables that can affect future safety performance, and these can be difficult to estimate. ï· Very few people (including myself) use CPM and CMFs on a daily basis. It is hard to develop expertise. When getting outside of this group, explaining and justifying numbers can be difficult. It is more difficult when you disagree with how someone else's numbers were computed and why you disagree. ï· Results that show a crash reduction of less than one crash. Hard to make the case for investing in a project. We have low numbers of crashes in VT. Another issue is with calibration. We have not calibrated the HSM type models. Results are thus potentially unreliable. ï· Still some concern about how sensitive the development of the SPF is to the available data components. What attributes (and to what accuracy) are most needed versus what attributes are less "needed" to achieve usable functions/models. ï· Yes. We want to use the CPMs but there are many uncertainties as mentioned in the questions above. With limited experience, it's difficult to confidently assess the reliability of the CPMs and the results they produce. Summary of Survey Responses The survey provided valuable information about the importance of the different application-related factors. Those issues that were rated as âVery Concerningâ by multiple respondents were considered important enough for further examination in this research project. These issues included the following: ï· Using crash modification factors that were not included in the original CPM and are inconsistent with the base conditions of the CPM. ï· Using a CPM that does not represent the characteristics of your project (e.g., using a 4-lane road CPM to predict crashes for a 6-lane road project). ï· Using input values that are uncertain (e.g., estimated AADTs based on old counts, crashes with unverified locations, etc.) ï· Using a CPM for a project whose characteristics lie outside the range of values of the CPM development (e.g., AADT volumes higher than the maximum value documented for the CPM). ï· Using a CPM to estimate rare crash types, such as fatal crashes or pedestrian crashes.