Improving Motor Carrier Safety Measurement (2017) / Chapter Skim
Currently Skimming:
3 Applications of Item Response Theory Models in Other Contexts and Public Release and Transparency
3 Applications of Item Response Theory Models in Other Contexts and Public Release and Transparency
Pages 63-80
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 63... ...
In commercial air, rail, and waterway transportation, as with the Federal Motor Carrier Safety Administration (FMCSA) , safety reporting is aggregated for segments of the respective industries and specific to individual companies.
|
From page 64... ...
. These two sectors have a history of measurement and public reporting, with many similar measurement, modeling, and public reporting challenges as those involved in assessing motor carrier safety.
|
From page 65... ...
Structural measures refer to characteristics of the provider that enable the capacity to deliver high quality of care and include physical attributes such as the existence of computer order entry systems. Process measures refer to what providers do to and for patients, often linked to adherence to established best practices and medical guidelines, for example, if a patient admitted for a heart attack who is eligible for beta-blocker therapy receives a prescription for beta-blocker therapy upon discharge.
|
From page 66... ...
Hierarchical generalized linear models are estimated for hospital outcome measures for specific diseases in which patients are clustered within hospitals. Because patients are not randomized to hospitals, admission characteristics of the patients are used to adjust for differences in patients across hospitals.
|
From page 67... ...
Value-added models are statistical models for students' current standardized achievement test scores as functions of prior years' test scores, student background variables, and educational inputs such as schools or teachers. They are closely related to item response theory (IRT)
|
From page 68... ...
. In the late 1990s, multiple researchers used value-added models to argue that teachers were the most important schooling input to student achievement and that large differences existed in the effectiveness of teachers to support student learning as measured by standardized achievement tests (Sanders and Rivers, 1996; Wright, Horn, and Sanders, 1997)
|
From page 69... ...
ASSESSING MOTOR CARRIER SAFETY As highlighted throughout this report, the mission of the FMCSA is to reduce crashes, injuries and fatalities involving large trucks and buses operationalized through safety measurement. Simply stated, safer commercial motor vehicles should have fewer crashes, all else being equal.
|
From page 70... ...
However, the subdimensions and the overall score should be more thoroughly empirically tested to assess the reliability and other operating characteristics, such as construct validity -- the degree to which each BASIC measures the types of unsafe operation it is intended to, and discriminant validity -- the degree to which the BASICs TABLE 3-1 Steps Required to Empirically Derive a Composite Safety Measure Task Example Specify Safety Subdomains Unsafe driving, fatigued driving Derive Content for Each Subdomain Violations assigned to specific latent subdomains Assess Relationship between Violations Item response theory (IRT) model and Subdomain Score Derive Subdomain Scores Estimated from IRT model Combine Subdomain Scores Composite score via aggregation of estimates
|
From page 71... ...
There are additional considerations, however, that complicate the creation of a safety construct using violation data. First, violation information is collected only when carriers are inspected.
|
From page 72... ...
Figure 3-2 (explained more fully in Appendix C) is an illustrative example of the relationship between each of the violations comprising the Unsafe Driving BASIC obtained from estimating a two-parameter logistic item response heory model.
|
From page 73... ...
As a provision of the Fixing America's Surface Transportation (FAST) Act of 2015, FMCSA was precluded from publishing the SMS percentile ranks for any BASIC, though it was still permitted
|
From page 74... ...
The hope is that the initial interventions, typically notifications, motivate carriers to make changes so that their BASIC percentile ranks become more representative of typical carriers. Making the percentile ranks public creates pressure on motor carriers to make these changes because they then face the prospect of losing business to their rivals or having their insurance rates increased.
|
From page 75... ...
Therefore the Crash Indicator BASIC could never be a perfect discriminator of small carriers. For example, the National Highway Traffic Safety Administration estimated that in 2012, the fatal crash rate for large trucks was 1.42 per 100 million miles.
|
From page 76... ...
He worried that not disclosing the measures and percentiles will deprive the public, shippers, and insurance brokers from learning about the comparative safety of motor carriers. According to Shuie Yankelewicz and Jean Gardner of the Central Analysis Bureau, SMS percentile ranks create a common language among CMV industry partners who are focused on monitoring and risk assessment of CMV operations, such as insurance companies, insurance agents and brokers, shippers, safety and risk management companies, and loss control companies.
|
From page 77... ...
The panel would also need to better understand the statistical operating characteristics of the measures and percentile ranks to judge decisions regarding their usability. RECOMMENDATION: The Federal Motor Carrier Safety Adminis tration should undertake a study to better understand the statistical operating characteristics of the percentile ranks to support deci sions regarding the usability of public scores.
|
From page 78... ...
Motor carriers could actively manage their business in ways that they know would improve their BASIC measures. A complication is that to see the impact of a carrier's recent performance or to produce indications of alert status for the carrier, it is necessary to view not only the carrier's data, but also the input data for all carriers in the same safety event group (which is almost certainly different for each BASIC)
|
From page 79... ...
should structure a user-friendly version of the Motor Carrier Management Information System data file used as input to the Safety Measurement System (SMS) without any per sonally identifiable information to facilitate its use by external parties, such as researchers, and by carriers.
|
From page 80... ...
The field test includes multiple vehicles from multiple motor carriers in multistate corridors. A wireless inspection will require modifying existing telematics systems in CMVs.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.