National Academies Press: OpenBook

Understanding and Communicating Reliability of Crash Prediction Models (2021)

Chapter:Chapter 2. Survey of Practitioners

« Previous: Chapter 1. Introduction and Background
Page 17
Suggested Citation:"Chapter 2. Survey of Practitioners." National Academies of Sciences, Engineering, and Medicine. 2021. Understanding and Communicating Reliability of Crash Prediction Models. Washington, DC: The National Academies Press. doi: 10.17226/26440.
×
Page17
Page 18
Suggested Citation:"Chapter 2. Survey of Practitioners." National Academies of Sciences, Engineering, and Medicine. 2021. Understanding and Communicating Reliability of Crash Prediction Models. Washington, DC: The National Academies Press. doi: 10.17226/26440.
×
Page18
Page 19
Suggested Citation:"Chapter 2. Survey of Practitioners." National Academies of Sciences, Engineering, and Medicine. 2021. Understanding and Communicating Reliability of Crash Prediction Models. Washington, DC: The National Academies Press. doi: 10.17226/26440.
×
Page19
Page 20
Suggested Citation:"Chapter 2. Survey of Practitioners." National Academies of Sciences, Engineering, and Medicine. 2021. Understanding and Communicating Reliability of Crash Prediction Models. Washington, DC: The National Academies Press. doi: 10.17226/26440.
×
Page20

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

15 Chapter 2. Survey of Practitioners Initially, the project team envisioned gathering the NCHRP Project 17-78 panel’s opinion on the model and application-related factors. Based on the kickoff call, the project team agreed to expand and conduct a survey of a broader group of practitioners. In the kickoff call, it was proposed that the project team focus more on application-related factors. The project team identified a list of practitioners for the survey by contacting FHWA Office of Safety and AASHTO’s Senior Engineering Program Manager for Safety. These practitioners included members of the AASHTO Safety Management subcommittee, AASHTO HSM2 Steering Committee, and FHWA HSM Pooled Fund Group. Following are the details of the survey, including the results. Twenty practitioners responded to the survey. The first page of the survey provided the following background about the survey, and this shown below: Crash prediction models (CPMs), also called safety performance functions, are models used to predict  crash frequency based on site characteristics. They may come with or without crash modification  factors (CMFs) that adjust the prediction to account for site conditions that are different from the  base condition of the model. Many CPMs are presented in Part C of the Highway Safety Manual.  You are invited to share your experience as a user of crash prediction models (CPMs) by responding  this short questionnaire, needing approximately 10 minutes. Your response will help direct the NCHRP  17‐78 project team to address the priority needs of safety practitioners.  Brief Background  The NCHRP 17‐78 project, recently launched, has three main objectives:   1. Develop guidance for the quantification of the reliability of crash prediction models (including crash modification factors (CMFs), crash modification functions and safety performance functions (SPFs)) for practitioner use. 2. Develop guidance for user interpretation of model reliability. 3. Develop guidance for the application of CPMs accounting for, but not limited to assumptions, data ranges, and intended and unintended uses. The project team will greatly benefit from the feedback from users of crash prediction models, like  yourself. Your responses will help us in identifying the day‐to‐day needs and high priority gaps in  understanding and communicating reliability‐related factors or aspects, and will be instrumental in  developing pragmatic and needed guidance to practitioners, like yourself.   The second page asked the respondents to indicate whether a particular issue regarding CPMs were “Very concerning”, “Somewhat concerning”, “Neutral”, “Not very concerning”, or “Not a concern”. The respondents were asked to respond to seven issues. Following is the list of questions along with the summary of the results.

16 Potential Concerns in Applying CPMs  When you use a CPM to what level  does each of these potential issues  concern you?   "Concern" in this usage refers to your level of trust in the results of the CPM predictions for your own  use as well as your concerns about the ease of communicating the reliability and results of the CPM to  your colleagues, managers, and the public. Some of these potential concerns may not be applicable to  you because your data does not present that type of concern or you have other ways of handling it. In  those cases, mark your answer as "Not a concern". 

17 Table 3. Results of Practitioner Survey. Issue Number of Responses Very concerning Somewhat concerning Neutral Not very concerning Not a concern Did not Answer Using CPMs that were developed in another location but calibrated for your local jurisdiction 0 7 2 4 7 0 Using crash modification factors that were not included in the original CPM but are consistent with the base conditions of the CPM 1 4 4 9 1 1 Using crash modification factors that were not included in the original CPM and are inconsistent with the base conditions of the CPM 7 7 4 1 1 0 Using a CPM that does not represent the characteristics of your project (e.g., using a 4-lane road CPM to predict crashes for a 6-lane road project) 13 6 0 0 1 0 Using input values that are uncertain (e.g., estimated AADTs based on old counts, crashes with unverified locations, etc.) 4 9 6 1 0 0 Using a CPM for a project whose characteristics lie outside the range of values of the CPM development (e.g., AADT volumes higher than the maximum value documented for the CPM) 7 8 3 2 0 0 Using a CPM to estimate rare crash types, such as fatal crashes or pedestrian crashes 6 5 4 5 0 0 Finally, the respondents were asked to answer the following question in their own words (the answers are provided below the question): Question: Are there any other concerns you have when communicating the reliability or application of  CPMs (data, values used for variables, estimates calculated, and to your decision to adopt or not to  adopt a given CPM and CMFs) to your colleagues, managers, or the public? Please describe. 

18 Answers:  The biggest difficulty is explaining what a CPM is and why one would use a CPM instead of pure naive prediction. If they do accept it, they tend to overrate the reliability of the CPM. Some prefer whatever predictive method gives the results they want, regardless of which is most accurate or applicable.  Communicating the value of CPM (or SPF) that predict a very low number of crashes, such as those for very specific conditions, e.g., pedestrians, bicycles, rail road crossings, specific geometric conditions, etc.  Any CMF that has a broad range of standard deviation, especially those that are not statistically significant. I don't put much trust in those CMFs.  Too many input parameters.  It is important to interpret the data and describe what the data represents. Typically, we describe these predictions as planning level estimates for what we expect safety to look like in some future scenario based on the specific variables that were captured in a specific CPM. This can be difficult to explain, because there are so many variables that can affect future safety performance, and these can be difficult to estimate.  Very few people (including myself) use CPM and CMFs on a daily basis. It is hard to develop expertise. When getting outside of this group, explaining and justifying numbers can be difficult. It is more difficult when you disagree with how someone else's numbers were computed and why you disagree.  Results that show a crash reduction of less than one crash. Hard to make the case for investing in a project. We have low numbers of crashes in VT. Another issue is with calibration. We have not calibrated the HSM type models. Results are thus potentially unreliable.  Still some concern about how sensitive the development of the SPF is to the available data components. What attributes (and to what accuracy) are most needed versus what attributes are less "needed" to achieve usable functions/models.  Yes. We want to use the CPMs but there are many uncertainties as mentioned in the questions above. With limited experience, it's difficult to confidently assess the reliability of the CPMs and the results they produce. Summary of Survey Responses The survey provided valuable information about the importance of the different application-related factors. Those issues that were rated as “Very Concerning” by multiple respondents were considered important enough for further examination in this research project. These issues included the following:  Using crash modification factors that were not included in the original CPM and are inconsistent with the base conditions of the CPM.  Using a CPM that does not represent the characteristics of your project (e.g., using a 4-lane road CPM to predict crashes for a 6-lane road project).  Using input values that are uncertain (e.g., estimated AADTs based on old counts, crashes with unverified locations, etc.)  Using a CPM for a project whose characteristics lie outside the range of values of the CPM development (e.g., AADT volumes higher than the maximum value documented for the CPM).  Using a CPM to estimate rare crash types, such as fatal crashes or pedestrian crashes.

Next: Chapter 3. Procedures for Quantifying the Reliability of Crash Prediction Model Estimates with a Focus on Mismatch Between CMFs and SPF Base Conditions »
Understanding and Communicating Reliability of Crash Prediction Models Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Understanding and communicating consistently reliable crash prediction results are critical to credible analysis and to overcome barriers for some transportation agencies or professionals utilizing these models.

The TRB National Cooperative Highway Research Program's NCHRP Web-Only Document 303: Understanding and Communicating Reliability of Crash Prediction Models provides guidance on being able to assess and understand the reliability of Crash Prediction Models.

This document is supplemental to NCHRP Research Report 983: Reliability of Crash Prediction Models: A Guide for Quantifying and Improving the Reliability of Model Results.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!