Improving Motor Carrier Safety Measurement (2017) / Chapter Skim
Currently Skimming:
4 Applying Item Response Theory Models to Highway Safety
4 Applying Item Response Theory Models to Highway Safety
Pages 81-96
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 81... ...
. The main difference is that the BASICs methodology uses severity weights that are dependent on expert opinion and empirical observations in a less empirical and static manner, whereas the item difficulty and discrimination parameters are estimated based on a formal combination of the observed data and expert opinion through the use of priors that are updated dynamically as more data are collected.
|
From page 82... ...
; • provide a basis with which to evaluate how data insufficiency could impact safety ratings of carriers (e.g., impact of inaccuracy of vehicle miles traveled on safety ratings) since it yields mea sures of uncertainty of the latent safety score for each carrier, unlike the current deterministic algorithm; • provide a basis to more rigorously and empirically evaluate the utility of individual violations since it yields measures of rel evance of individual violations -- in other words, the extent to which each violation is related to the latent safety construct; • allow severity weights to change over time (e.g., as violations become more or less prevalent)
|
From page 83... ...
. The rate, λij , is allowed to depend on carrier-level characteristics, Xi , that may influence the rate at which a carrier receives inspections.
|
From page 84... ...
safety measure for carrier i; ak , parameters that capture how strongly item k is related to the latent safety measure qi for carrier i, which take the place of severity weights, and which determine how well that violation discriminates safe versus less safe carriers, and exp ( β k ) bk , which when transformed as represents the prevalence of 1+ exp ( β k )
|
From page 85... ...
First, the Bayesian approach is the most natural way to incorporate expert opinion into the model. One area where this could be particularly beneficial is to enable the use of information that led to the development of severity weights in the current Safety Measurement System (SMS)
|
From page 86... ...
For instance, if some hazmat violation rates were found not to be strongly associated with safe operations, but FMCSA thought important to retain, instead of downweighting those violation rates through updating of a prior, FMCSA could specify a prior distribution that assigns high a priori probability to the severity weights of those violations. ITEM RESPONSE THEORY MODEL MODIFICATIONS In this section we propose a number of extensions and modifications to the model described above.
|
From page 87... ...
A negative value of h suggests that safe carriers have a lower rate of inspections than unsafe carriers, while a positive value suggests that safe carriers have a higher rate of inspections. When h is equal to zero, then there is no benefit to this joint modeling approach because the number of inspections Nij provides no information about the safety of the carrier ( atfield, Hodges, and Carlin, 2014)
|
From page 88... ...
Models That Downweight Violations Further Back in the Past An additional extension of the model can downweight violations that occurred further in the past, similar to time weights used in computing BASICs. However, we advocate letting the weights change smoothly over time, rather than using time severity weights that change at discrete time windows, so that changes in the safety scores occur smoothly.
|
From page 89... ...
As above, qi represents an "overall" safety measure over the current time window. However, as time evolves, qi will change smoothly since safety measures are updated monthly, and there is a 23-month overlap for safety measures computed in consecutive months.
|
From page 90... ...
In this way, the overall latent trait reflects the general safety measured by an inspection, and each secondary latent trait indicates the unique contribution of, for example, unsafe driving over and above the general safety latent trait. Relative to a fully multidimensional IRT model, the secondary factors in the bifactor model represent variation attributable to the violations that are beyond the overall primary latent
|
From page 91... ...
These multiple latent traits would be analogous to the multiple BASICs, in that they represent different groupings of the violations. However, the way in which the items of a particular latent trait "hang together" could be determined empirically.
|
From page 92... ...
But, for example, the Unsafe Driving factor does not impact violation 21 since the Unsafe Driving factor is uncorrelated with the Uncontrolled Substance factor. HM, hazardous materials; HOS, hours of service.
|
From page 93... ...
Thus, crashes represent a safety measure that is conceptually different from, for instance, maintenance violations. Further, crashes are relatively rare and, thus, would be more difficult to incorporate in the IRT modeling approach.
|
From page 94... ...
For example, unsafe driving (secondary factor) impacts violations 11-13 over and above the impact of safety score (primary factor)
|
From page 95... ...
SUMMARY ABOUT THE IRT MODEL FOR CARRIER SAFETY In this chapter, we outlined a probabilistic approach with which to obtain estimates of a carrier's safety. A key advantage of using a probabilistic approach over a deterministic model such as the current BASIC methodology is that probabilistic approaches yield measures of uncertainty in addition to estimated safety measures or rankings.
|
From page 96... ...
• They can allow for the addition of new safety measures as they become available, without having to start from scratch. • They can produce ranking ranges (by sampling from the pos terior distribution of theta)
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.