Evaluation and Comparison of Roadside Crash Injury Metrics (2023)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

114 7 Analyze IAD for Oblique Crashes Introduction The purpose of this chapter is to present the injury models and risk curves associated with each of the candidate injury metrics in oblique crashes. The oblique IAD subset was used to train MAIS2+F whole body logistic models based on five candidate injury metrics: MDV, OIV, OLC, ASI, and VPI. The injury models were then evaluated with an equivalent test dataset composed of oblique crashes extracted from CISS. The ability of each candidate injury metric to predict real- world occupant crash injury was then ranked according to various statistical methods. Much of this chapter is provided in Dean, M.E., Gabauer, D.J., Riexinger, L.E., and Gabler, H.C., "Comparison of Vehicle-Based Crash Severity Metrics for Predicting Occupant Injury in Real-World Oblique Crashes," Transportation Research Record: Journal of the Transportation Research Board, Volume 2677, Issue 2, pp 505-518, 2022, which is listed as the first cited work at the end of this document. Text and figures are reproduced largely verbatim and are © 2022, SAGE Publishing. Methods The IAD contained 176 sampled occupants representing 67,107 occupants involved in real- world oblique crashes from NASS/CDS and 179 sampled occupants representing 103,839 occupants involved in real-world oblique crashes from CISS. The training dataset contained occupants in real-world oblique impacts sampled in NASS/CDS with associated EDR data from the VT EDR database. Similarly, the test dataset contained occupants in real-world oblique impacts sampled in CISS with associated EDR data from the VT EDR database. 7.2.1 Metric Computation and Validation The methods used to validate the computation of each metric are described in the frontal crash IAD analysis chapter (Section 5.2.1). For the oblique crash model, both the longitudinal and lateral delta-v crash pulses were used to calculate every metric: MDV, OIV, OLC, ASI, and VPI. For MDV, OLC, and VPI, the metrics were calculated once with the longitudinal crash pulse and once with the lateral crash pulse for each case. For each pair of values, the resultant metric value was found and used in the oblique model. The same OIV values computed for the frontal and side crash models were used to compute a resultant OIV value to use in the oblique models. When calculating ASI, the raw values are normalized by dividing by a threshold. For the longitudinal and lateral directions, these thresholds are 12 G and 9 G, respectively. For the oblique model, ASI was calculated once with the longitudinal crash pulse and once with the lateral crash pulse, each time dividing by the appropriate threshold for that direction. Then, the normalized values were used to calculate a resultant value for each case. 7.2.2 Injury Risk Modeling The injury models were developed using logistic regression. The 1998 AIS was used to determine injury severity, since the 1998 injury codes were available for all the case years in the dataset (AAAM 1998). Any occupant with an MAIS2+F (occupants with an MAIS of 2 or greater, including occupants who were fatally injured) rating was considered injured. Occupants with an

115 unknown injury severity were excluded unless their injuries resulted in fatality. Fatally injured occupants were included in the MAIS2+F category, regardless of MAIS level. Several predictor variables were used within the models to predict injury: • Crash Severity Metrics. Five injury models were built for oblique crashes. Each model used one of the five crash severity metrics as an independent predictor variable. The crash severity metrics are each a function of both the longitudinal and longitudinal delta-v. The delta-v versus time series data were obtained from the EDR. As described above, the metric value was generally computed based on the longitudinal and lateral metric values to provide a single value for each case. • Belt Status. Belt status was a binary variable, where a value of 1 indicates an occupant using a three-point belt restraint, and 0 indicates the occupant was unbelted. Belt status was determined using the EDR belt status variable. • Age. Age was a binary variable, where 1 indicates ≥ 65 years old and 0 indicates ≥ 13 years old and < 65 years old. • Sex. Sex was a binary variable, where 1 indicates male and 0 indicates female. • BMI. Body mass index (kg/m2) was a binary variable, where 1 indicates obese (BMI ≥ 30 kg/m2) and 0 indicates not obese (BMI < 30 kg/m2). • Occupant Seating Location. The occupant’s seating location was a binary variable, where a value of 1 indicates the occupant was the driver, and 0 indicates the occupant was in the right front passenger seat. • General Area of Damage (GAD). GAD describes where the greatest area of deformation occurred. It was a binary variable, where 0 indicates side damage and 1 indicates frontal damage. • Vehicle Type. This variable was defined using NHTSA’s Vehicle Body Type Classification (NHTSA 2018). All the vehicles in this dataset fall into the category of either Passenger Car (PC; body types: 1-11, 17) or Light Trucks and Vans (LTV; body types: 14-16, 19-22, 24, 25, 28-41, 45-49). All of the vehicles in the dataset fell into one of these categories. Vehicle type was a binary variable, where a value of 1 indicates a PC and 0 indicates an LTV. First, logistic regression models were developed using the full set of covariates. These initial regression models were examined to determine which of the covariates were statistically significant. Based on those results, a second set of regression models was developed in which only the statistically significant covariates were included. The logistic regression models for each set were developed using the survey package and SVYDESIGN function in R (Lumley 2020). The R survey package was designed based on SAS and is capable of computing confidence limits that account for the complex NASS-CDS stratified sampling scheme.

116 7.2.3 Body Region-Specific Models The body region injury data, from the NASS/CDS occupant injury table, list every injury for each body region for each occupant. Any occupant with unknown body region data was removed from the dataset. The NASS/CDS regions were combined into the HF, N, and TALT regions (Table 7-1). There were too few MAIS2+F injuries in the HF, N, and TALT regions to construct oblique body region injury models. Table 7-1. NASS/CDS body regions used to form the oblique body region models. NASS/CDS Region NASS/CDS Specific Anatomic Structure Model Region Head All HF – not modeled Face All Neck All N – not modeled Thorax All TALT – not modeled Abdomen All Spine C-Spine N – not modeled T-Spine TALT – not modeled L-Spine Upper extremity All Injury risk not modeled Lower extremity All 7.2.4 Predictive Capability After building the models using the NASS/CDS training dataset, the developed models were run on the test dataset from CISS. Precision, recall, accuracy, F2 score, and AUC were computed for the training and test datasets. Precision, also known as the predictive value of the model, is the probability that the occupants predicted to be injured were actually injured. Recall, also known as sensitivity or the true positive rate, is the ability of the model to correctly predict injury for occupants who actually did suffer an injury. The F2 score prioritizes recall and is indicative of the model’s ability to accurately predict outcomes and is a reliable metric for unbalanced datasets. A higher F2 score indicates better predictive capability of the model and was the primary comparison metric. When calculating precision and recall, a decision threshold in the form of a percent injury risk must be chosen. This threshold was found for each model individually by selecting the percent injury risk value that optimized the F2 score. These thresholds were found using the training dataset and were used throughout the comparison process. When more than one injury risk value was associated with the same precision and recall summation, the lowest injury risk value was chosen. The NASS/CDS and CISS weighted values were used to calculate each of these comparison metrics. Overall Injury Model Results 7.3.1 Initial Injury Risk Models The initial models used seat belt usage, sex, age, BMI, seating location, GAD, and vehicle type as covariates. Each model additionally used one of the five severity metrics as a covariate. Table E-1 through Table E-5 show the regression coefficients for each of the injury risk models. A p-

117 value < 0.05 was considered significant and is denoted by ** in the parameter tables. A negative coefficient indicates that, with all other predictors held constant, a decrease in a continuous variable will reduce the injury risk. For a binary covariate, the baseline condition (listed in the model tables) reduces the injury risk. A positive coefficient indicates that, with all other predictors held constant, an increase in a continuous variable will increase the injury risk. For a binary covariate, the non-baseline condition reduces the injury risk. For example, the metrics all have positive coefficients because an increase in any of the metrics will heighten the occupant’s risk of injury. Additionally, age (≥ 65) always has a positive coefficient because older occupants are more likely to suffer an MAIS2+F injury. Belt status and GAD were significant in every initial model except the ASI model. Occupant sex, seating location, and BMI were not significant in any of the initial models. The only significant covariate in the initial ASI model was ASI. OLC was not significant in the initial model but was included in the final model where it became significant. Equation 8 is the final form of the model and Equation 12 is the logit expanded. 𝑃𝑃[𝑀𝑀𝐴𝐴𝑀𝑀𝑀𝑀2+ F] = 1 1 + 𝑃𝑃−𝑅𝑅𝑃𝑃𝑙𝑙𝑃𝑃𝑙𝑙 (8) 𝑅𝑅𝑃𝑃𝑙𝑙𝑃𝑃𝑙𝑙 = 𝛽𝛽0 + 𝛽𝛽1 ⋅ (𝑃𝑃𝑃𝑃𝑖𝑖𝐴𝐴𝑃𝑃𝐴𝐴 𝑚𝑚𝑃𝑃𝑙𝑙𝑃𝑃𝑃𝑃𝑃𝑃) + 𝛽𝛽2 ⋅ 𝑏𝑏𝑃𝑃𝑅𝑅𝑙𝑙_𝑃𝑃𝑙𝑙𝑅𝑅𝑙𝑙𝐴𝐴𝑃𝑃 + 𝛽𝛽3 ⋅ 𝑃𝑃𝑃𝑃𝑠𝑠 + 𝛽𝛽4 ⋅ 𝑅𝑅𝑙𝑙𝑃𝑃 + 𝛽𝛽5 ⋅ 𝑃𝑃𝑏𝑏𝑃𝑃𝑃𝑃𝑃𝑃 + 𝛽𝛽6 ⋅ 𝑃𝑃𝑃𝑃𝑅𝑅𝑙𝑙𝑃𝑃𝑃𝑃𝑙𝑙𝑅𝑅𝑃𝑃𝑃𝑃𝑅𝑅𝑙𝑙𝑃𝑃𝑃𝑃𝑃𝑃 + 𝛽𝛽7 ⋅ 𝑉𝑉𝑃𝑃ℎ𝑃𝑃𝑃𝑃𝑅𝑅𝑃𝑃𝑇𝑇𝐴𝐴𝑡𝑡𝑃𝑃 + 𝛽𝛽8 ⋅ 𝐺𝐺𝐴𝐴𝑃𝑃 (11) 7.3.2 Final Injury Risk Models Based on the initial regression models, a second set of injury models was developed. These models use only the covariates that were statistically significant in the initial model: one of the five candidate injury metrics, belt status, and GAD for the MDV, OIV, OLC, and VPI models and ASI for the ASI model. Vehicle type was significant in the initial OIV model but became insignificant in the second pass of the model, so it was removed from the model. Table 7-2 through Table 7-6 show the regression coefficients for each of the models. All covariates and the intercept were significant in every final model.

118 Table 7-2. Parameters for the MDV final oblique logistic regression oblique model used to predict occupant MAIS2+F injuries. ** indicates statistical significance (p-value < 0.05). Predictor Variable Parameter Coefficient Std. Error p-Value --- β0, Intercept -2.073 0.947 -- MDV β1, MDV (m/s) 0.299 0.085 0.001** Belt Status β2, Belted -1.941 0.698 0.007** GAD β8, Frontal Damage -1.921 0.613 0.003** Table 7-3. Parameters for the OIV final oblique logistic regression oblique model used to predict occupant MAIS2+F injuries. ** indicates statistical significance (p-value < 0.05). Predictor Variable Parameter Coefficient Std. Error p-Value --- β0, Intercept -2.588 1.057 -- OIV β1, OIV (m/s) 0.372 0.108 0.001** Belt Status β2, Belted -1.773 0.728 0.017** GAD β8, Frontal Damage -1.966 0.619 0.002** Table 7-4. Parameters for the OLC final oblique logistic regression oblique model used to predict occupant MAIS2+F injuries. ** indicates statistical significance (p-value < 0.05). Predictor Variable Parameter Coefficient Std. Error p-Value --- β0, Intercept -0.874 1.053 -- Resultant OLC β1, OLC (g) 0.127 0.048 0.011** Belt Status β2, Belted -1.761 0.700 0.014** GAD β8, Frontal Damage -1.641 0.662 0.016** Table 7-5. Parameters for the ASI final oblique logistic regression oblique model used to predict occupant MAIS2+F injuries. ** indicates statistical significance (p-value < 0.05). Predictor Variable Parameter Coefficient Std. Error p-Value --- β0, Intercept -4.715 0.731 -- Resultant ASI β1, ASI 2.371 0.449 < 0.001** Table 7-6. Parameters for the VPI final oblique logistic regression oblique model used to predict occupant MAIS2+F injuries. ** indicates statistical significance (p-value < 0.05). Predictor Variable Parameter Coefficient Std. Error p-Value --- β0, Intercept -- 1.053 -- Resultant VPI β1, VPI (m/s2) 0.009 0.002 < 0.001** Belt Status β2, Belted -1.648 0.705 0.022** GAD β8, Frontal Damage -1.322 0.649 0.045** 7.3.3 Injury Risk Curves The predict function in R was used to obtain injury probability values for each of the five final models. To generate the risk curves, the probability of injury was evaluated for belted and unbelted occupants as the crash severity metric increased by 0.01 from zero to a resulting probability greater than 99%. The vertical black lines on each plot represent the minimum and maximum metric values in the original dataset. Portions of the curves outside these bounds have been extrapolated by the model. This process was performed once for a belted occupant and again for an unbelted occupant when belt status was significant. Figure 7-1 through Figure 7-5 plot the risk curves for all five models. The shaded regions represent 95% confidence intervals.

119 Figure 7-1. Oblique impact MDV injury risk curve for front row occupants in oblique crashes. These curves come from the final MDV model. Figure 7-2. Oblique impact OIV injury risk curves for front row occupants in oblique crashes. These curves come from the final OIV model.

120 Figure 7-3. Oblique impact OLC injury risk curve for front row occupants in oblique crashes. These curves come from the final OLC model. Figure 7-4. Oblique impact ASI injury risk curve for front row occupants in oblique crashes. These curves come from the final ASI model.

121 Figure 7-5. Oblique impact VPI injury risk curve for front row occupants in oblique crashes. These curves come from the final VPI model. Model Validation and Comparison The training data were used to determine a decision threshold for each model and to compare the predictive capability of the models on training data by calculating the F2 scores. The same decision thresholds were used to calculate precision, recall, and the resulting F2 score after the models were applied to the CISS test dataset. The accuracy of each model on the test dataset was also calculated. All the final models yield similar decision thresholds, except the OLC model, which has a slightly higher threshold of 20%. Additionally, the F2 scores for each model are very similar. OLC performed better on the test dataset than the rest of the models with an F2 score of 0.61. MDV, however, had an F2 value only slightly lower than OLC at 0.56. All of the models yielded the high accuracy values; however, it is important to note why accuracy is not a reliable metric for an unbalanced dataset. Accuracy is a measure of how many predictions were correct, and because the majority of the predictions are true negative injury cases, it is easy for the models to have a high accuracy but poor precision, and therefore a low F2 score. Additionally, every model yields an AUC value of at least 0.88, indicating the models perform fairly well. However, similarly to accuracy, the AUC for ROCs are not a great diagnostic tool for determining how well a model performs when it is exposed to highly imbalanced data. This is because ROCs consider the false positive rate, which is calculated using the total number of true negatives. When the number of true negatives is very large, the false positive rate does not drop drastically. On the contrary, precision is a metric very sensitive to the number of false positives.

122 Table 7-7. F2 scores for the five metrics’ models. These values come from the final model. Metric Model Decision Threshold Training Test F2 Scores ROC AUC Accuracy Precision Recall F2 Score MDV 14% 0.69 0.90 84% 0.23 0.87 0.56 OIV 14% 0.69 0.86 85% 0.22 0.73 0.51 OLC 20% 0.64 0.88 92% 0.37 0.73 0.61 ASI 6% 0.60 0.88 77% 0.17 0.84 0.47 VPI 7% 0.66 0.88 75% 0.16 0.88 0.47 Discussion The crash severity metric, belt status, and GAD were significant predictors in all the final regression models besides ASI. For MDV, OIV, OLC, and VPI, holding all covariates constant, the risk of injury was substantially lower for a belted occupant than for an unbelted occupant. Using the F2 scores from the test dataset as a metric of predictive capability, the OLC model is most equipped to accurately predict occupant injury outcomes in frontal crashes. For each pair of injury risk curves, the confidence intervals grow wider as the risk of injury approaches 50%. This is due to the variability in injury outcomes for medium-severity crashes. It is important to note some of the differences between NASS/CDS, the database used for training, and CISS, the database used for validation. While largely similar, CISS does use AIS 2015, while the NASS/CDS training data implements AIS 1998. Additionally, the tow-away criterion between the two databases varies. NASS/CDS requires at least one vehicle be towed away from the crash scene due to damage. CISS requires at least one vehicle be towed away from the crash scene due to any reason. Finally, CISS implements more primary sampling units than NASS/CDS. Currently, the U.S. and other roadside safety hardware crash test procedures prescribe preferred and maximum thresholds for the OIV and ASI metrics (Table 7-8). The models presented in this chapter can provide injury risk values associated with these thresholds for the best- and worst-case scenario occupant (Table 7-8). For the OIV model, a best-case scenario occupant would be a belted occupant in a vehicle with the GAD at the front of the vehicle. An unbelted occupant in a vehicle with the GAD at the side of the vehicle would be the worst-case scenario within the OIV model. Since the ASI model uses only ASI as a covariate, the best- and worst-case scenario occupants are the same. While other populations could be explored as well, this chapter examines only the two extremes. Calculating the MAIS2+F injury risk for the OIV and ASI thresholds makes it possible to compare the acceptable level of injury risk across the two metrics. For example, for the best-case scenario occupant for OIV, the injury risk percentages fall far below the maximum allowable injury risk value within the ASI model. For the worst-case scenario occupant for the OIV model, the injury risk percentages are far above the maximum allowable injury risk in terms of the ASI model. It should be noted that the computation of the OIV for the oblique models is the resultant value while the prescribed MASH OIV thresholds apply independently to the lateral and longitudinal directions.

123 Table 7-8. Injury risk associated with the current OIV and ASI thresholds for the best- and worst-case oblique impact scenarios. Crash and Occupant Conditions Injury Risk Scenario Belt Status GAD OIV Threshold (m/s) ASI Threshold (--) 9.1 12.2 1.0 1.4 1.9 Best Case Belted Frontal damage 5.0% 14.3% 8.7% 19.9% 44.8% Worst Case Unbelted Side damage 69.0% 87.5% 8.7% 19.9% 44.8% Limitations One of the limitations of this work is how few occupants suffered MAIS2+F injuries in both the training and test datasets. The value of the models lies in their ability to be able to predict occupants who are injured. The fewer injured occupants the models are exposed to in the training data, the weaker they will be when it comes to accurately predicting these cases in the test data. Additionally, because the F2 scores do not consider true negative cases, their calculations are fully dependent on the presence of injury data. The fewer injury cases there are in the test data, the less reliable the F2 score. Similar to the frontal and side impact crash modes, there were too few MAIS3+F injuries in the dataset to build MAIS3+F injury risk curves. Another limitation to this study is that the inclusion criteria do not consider vehicle incompatibility. The size and weight differences between two colliding vehicles were not restricted. Despite this limitation, some of this incompatibility should be accounted for in the MDV values. The other metrics, however, may not compensate for this incompatibility as well. Conclusions For each of the crash severity metrics, whole body MAIS2+F injury risk curves for oblique crashes were constructed with NASS/CDS cases in the IAD and tested with CISS cases in the IAD. Based on the findings of the literature review conducted as part of this research effort, this is the first study to use real-world crash data with EDR data to investigate the performance of vehicle- based metrics in oblique crashes. In general, the higher the crash severity metric, the higher the risk of an occupant sustaining an MAIS2+F injury. Based on the F2 score, the OLC metric was the best at predicting MAIS2+F in the test dataset. The MDV and OIV metrics, however, performed nearly as well. Factors other than the crash severity metric are important predictors of occupant injury. Belted occupants and crashes with significant frontal damage were at a lower risk of MAIS2+F injury. These covariates can drastically affect the probability of injury even when the crash severity metric is held constant. At the maximum OIV, the best-case scenario had a 14.3% probability of MAIS2+F injury, but the worst-case scenario had an 87.5% probability of MAIS2+F injury.