Naturalistic Driving Study: Alcohol Sensor Performance (2015)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

5Research Approach The present research study built on the pilot investigation and Phase 1. Those earlier exploratory efforts established a framework for understanding the nature of the alcohol sen- sor, provided significant insight into the necessary direction for Phase 2, and demonstrated several research needs. The primary need was a gold standard data set to better under- stand alcohol sensor readings and more precisely evaluate the accuracy of any developed alcohol-detection algorithm using this sensor. Additionally, an accurate assessment of intoxica- tion based on video evidence was necessary to evaluate driver and passenger impairment in existing SHRP 2 trips. To meet these diverse objectives, two research approaches were undertaken. Each of these approaches met different objectives as follows: 1. Mechanical Breather Approach: A gold standard data set was critical to understanding sensor readings and accu- rately evaluating an alcohol-detection algorithm using this signal. A carefully controlled experimental data set was created by instrumenting a sedan with the standard SHRP 2 instrumentation—including the alcohol sensor— and controlling the amount of alcohol present within the vehicle cabin. This was accomplished through a variety of methods that are more fully described in the Mechanical Breather Approach section. 2. Naturalistic Approach: The ultimate objective of the research was to validate the efficacy of the alcohol sensor at iden- tifying imbibed alcohol trips within the SHRP 2 database. Specifically, the naturalistic data approach was designed to determine whether the sensor could a. Differentiate between imbibed and unimbibed alco- hol; and b. Differentiate between cases of suspected moderate alco- hol use and cases with no suspected alcohol use. The naturalistic approach met this objective by coding trips for alcohol involvement from the SHRP 2 database by manually performing visual data inspection to identify both alcohol impairment of the driver or passenger and potential unimbibed alcohol sources. A data set was created from the SHRP 2 database for the nat- uralistic approach. This naturalistic test data set was designed to investigate false positives and the ability of the alcohol sen- sor to differentiate between imbibed and unimbibed alcohol. Given the purpose of this data set, the vast majority of included trips were selected on the basis of positive alcohol sensor values. This overrepresentation of positive alcohol sensor readings made this data set a poor source for evaluating the sensor’s performance at discriminating trips involving alcohol from those trips not involving alcohol. This sampling approach inflated the number of false positives compared with the SHRP 2 database. Thus, a subset of this data set, called the “impaired data set,” was created; the subset was limited to con- trol trips and trips in which a passenger or driver was deemed at least “moderately” impaired by the data reductionists. Spe- cifically, the impaired data set included only trips in which data reductionists suspected at least a moderate impairment of a driver or passenger and control trips that were not initially selected because of alcohol sensor flags or time of day. The sampling, trip selection, and manual coding are described in more detail in the Naturalistic Approach section of this report. Finally, known impaired trips from the SHRP 2 data set were examined to develop the alcohol-detection algorithm. These trips were chosen independently from the trip files used in the naturalistic test data set. These were primarily identified by drivers admitting they were driving while impaired or from trips in which alcohol impairment was identified when the trip was being evaluated for another purpose. Mechanical Breather Approach One important objective of this research was to identify whether the alcohol sensor could reasonably detect alcohol and how it responded to varying levels. This was necessary C h A p t e r 2

6to determine if the sensor was accurately capturing data and to aid in developing an algorithm. Since the purpose of the sensor was to detect imbibed alcohol, ideally a study could have been conducted in which drivers with different levels of blood alcohol content (BAC) would drive a vehicle (or at least be in the driver seat of a vehicle) instrumented with the alcohol sensor. With data from such a study, the sensor’s response to varying BACs could have been assessed. Unfortunately, because of both cost reasons and Institutional Review Board (IRB) challenges, this approach was not practical within the allotted time frame. Given these constraints, the best solution was to develop a mechanical alcohol breather based on a human’s natural breath at varying BACs. This mechanical breather was then used to produce different levels of alcohol in the vehicle while it was being driven. Development of the “Boozooka” Mechanical Breather Collecting data from intoxicated individuals is time con- suming and complex given the extra steps needed to protect human participants and control BrAC. To circumvent these problems, a mechanical breather, fondly called the Boozooka, was developed. The Boozooka—consisting of an air compres- sor, regulator, alcohol chamber, mixing valve, and breath alco- hol tester that released controlled amounts of alcohol vapors into the cabin—diffused alcohol into the air in a manner simi- lar to human breathing (Figure 2.1). The compressor pumped air through a regulator that was tuned to represent a typical human breathing rate of 10 liters of air per minute. The air was then split into two streams, one of which passed air through an alcohol chamber filled with cotton balls saturated with 80 proof vodka (cotton balls helped control splashing in the chamber while the vehicle was in motion). A needle valve allowed the alcohol concentration to be tuned by controlling the amount of air passing through the chamber relative to the bypass. During trips, the compressor was powered using a cigarette lighter plug-in. The exact BrAC was measured using a Lifeloc FC20 breath alcohol tester, which is accurate to ±0.005 g/dL. Three Lifeloc FC20 breath alcohol testers were rotated between trips to ensure none of the units became saturated. All units were fac- tory recertified by Lifeloc Technologies immediately before data collection. During each experiment, the Boozooka released alcohol vapors of varying BrAC concentrations according to the test plan. Even though the Boozooka did not emulate certain charac- teristics of human breath (e.g., humidity, temperature, carbon monoxide and other gases) it was still functionally equivalent for the alcohol sensor. The Boozooka “breathed” the alcohol at a controlled BrAC and volume. Furthermore, the Boozooka was designed to emulate how a human breathes in the vehicle. Since the output from the Boozooka did not blow directly onto the alcohol sensor, it was expected that the various gases of humans in the vehicle and the Boozooka were mixed by the time they reached the alcohol sensor. Finally, the Boozooka was also tested against an alcohol-impaired individual to ensure that the system responded qualitatively in a similar manner. Experimental Vehicle For the in-vehicle testing of the mechanical breather, an M35 Infinity sedan was specially instrumented for the project. It was imperative that this instrumentation mirror the SHRP 2 instrumentation setup as closely as possible. The M35 had two SHRP 2 head units, encompassing the alcohol sensor, mounted just under the rearview mirror. This positioning not only matched that of the single head unit of the SHRP 2 instrumen- tation but also allowed for the collection of readings from two alcohol sensors simultaneously, doubling the amount of data collected and providing a reliability estimate of alcohol sensor values. Specifically, this allowed for the direct comparison of two alcohol sensor readings under the exact same conditions. The vehicle was also equipped with a small data acquisition system (MiniDAS) to collect the alcohol sensor data. Images of the experimental equipment are shown in Figure 2.2. Data-Recording Interface The vehicle data acquisition system (DAS) used a data- recording program that ran on a custom embedded Linux operating system. In addition to choreographing the col- lection of synchronized data, this program also provided an interface for researchers to enter experimental conditions and variables into the in-vehicle data stream (a screenshot of the user interface is provided in Appendix A). In particular, study variables could be directly associated with correspond- ing alcohol sensor readings from the two head units. For example, the researchers kept a record of the in-vehicle BrAC, experimental conditions, air currents, and trip number. Data Figure 2.1. Boozooka mechanical breather used for in-vehicle alcohol testing.

7 were collected and stored at a rate of 10 Hz, the same rate used by the alcohol sensor in the SHRP 2 study. Experimental Procedure Experimental tests were performed on a predetermined study route that included several types of roadways and speeds rang- ing from 25 mph to 65 mph and that lasted approximately 15 minutes. Fifty trials were conducted under three experi- mental conditions. Each trial included only one condition. The conditions and number of trials are as follows: 1. Control testing (no BrAC), 6 trials; 2. Human testing, 11 trials; and 3. Boozooka testing, 33 trials. The heaviest emphasis was on Boozooka testing; the human testing was done primarily for validation. Across all experimental conditions within the vehicle, one researcher drove and another researcher entered data. During Boozooka testing, a third researcher operated the Boozooka and confirmed its BrAC output using the breath alcohol testers. In the human testing condition, an additional researcher who had consumed alcohol sat in the passenger seat. The con- trol trials were performed primarily with the driver and the researcher who entered data, occasionally with a third researcher observing from the back seat. Ten trials were conducted using recirculating air, and 40 trials used fresh air settings. The pilot study research indi- cated fresh air would provide a more challenging environ- ment for detecting alcohol presence, so more fresh air trials were chosen. Windows were rolled up for all trips since previous research indicated that the alcohol sensor could not detect alcohol presence with the windows rolled down. Accordingly, trips with windows rolled down were not conducted. Between each experimental trial for all test conditions, the car was parked and the windows or doors were opened for approximately 10 to 15 minutes to allow the alcohol sensor to return to baseline. Before each Boozooka trial, the level of alcohol within the alcohol chamber was examined. If the level was too low, then alcohol was added. The Boozooka operated for at least 10 minutes before beginning a trip to ensure stable BrAC Figure 2.2. M35 (upper left), outside view of the two head units and MiniDAS (upper right), MiniDAS (lower left), and inside view of two head units and MiniDAS (lower right).

8output, and a trip did not start until multiple consecutive readings were within approximately 0.04 g/dL. Readings on Boozooka BrAC were taken at least every 5 minutes of the trip using the Lifeloc FC20 breath alcohol testers. For the human testing, a BrAC reading was taken at the beginning and end of each trip since slight changes in BrAC were pos- sible during the course of a trip. Mechanical Breather Data Set The mechanical breather data set included a total of 50 15-minute experimental trials. Two alcohol sensors in the instrumentation provided the independent readings repre- sented in Table 2.1, doubling the number of samples for analysis. These trips constituted the gold standard data set because of the experimentally controlled presence of alco- hol within the vehicle. Naturalistic Approach The ultimate goal of a successful alcohol-impairment algo- rithm was to detect impaired trips within the SHRP 2 data set while minimizing false detections due to the presence of unimbibed alcohol. It was important to make sure the sample had an adequate number of true positives in which imbibed alcohol was likely (most important), false positives in which other sources of alcohol were likely, and a control group in which alcohol presence, though possible, was unlikely. To enable testing with SHRP 2 data, a reference data set was developed in which impairment was independently determined. Significant effort went into the development of a behavioral checklist for impairment using visual cues that could be identified using the SHRP 2 cameras. Similarly, the SHRP 2 trips included in the sample and the data reduction team in charge of reducing the trip files were carefully selected. Note that trip files, or trips, for the purpose of the natural- istic driving data are defined as the time from the vehicle being started to it being turned off. Up to a 60-second delay occurred between vehicle ignition and data collection due to the startup of the DAS. All analyses made use of the entire trip to identify the possible presence of unimbibed and imbibed alcohol. Selection of SHRP 2 Trip Files At the beginning of the project, it was impossible to deter- mine the number of total impaired trips in the SHRP 2 data- base; the number of substances or events that would trigger the alcohol sensor; and the success of the algorithm in detect- ing and differentiating types of alcohol. This necessitated a trip selection approach that would maximize the probability of finding impaired trips and populate the data set with trips of interest for evaluation. Thus, the first and largest (n = 562) batch of trips was selected to maximize the likelihood of positive (i.e., sensor-flagged) trips. All trips in this batch were selected from trips occurring between 12:00 a.m. and 4:00 a.m. in the driver’s local time, a time range expected to have a high likelihood of trips containing imbibed alcohol. Additionally, trips were selected on the basis of alcohol sen- sor values to maximize the probability of finding impaired trips and trips with unimbibed alcohol. Trips that crossed a certain threshold for spikes, standard deviations, or aver- ages for sensor readings were flagged and set aside for the reduction team. This process created an ideal data set for evaluating false positives and testing the algorithm’s ability to discriminate between imbibed and unimbibed alcohol within a vehicle. Once a large number of impaired and unimbibed alcohol trips were identified through manual video review, a sec- ond batch of trips was chosen. These trip files were chosen at random, irrespective of time of day or sensor readings, to select a group of trips during which imbibed alcohol presence was unlikely. While the first batch of trips achieved the objective of exploring the sensor’s ability to differenti- ate imbibed from unimbibed alcohol presence, the second batch of trips (n = 97) successfully evaluated the alcohol sensor’s ability to differentiate alcohol-involved trips from trips without alcohol presence (i.e., false alarms). This was primarily accomplished through the inclusion of the con- trol trips. The final data set included a total of 659 trips. Originally, 692 trips were sent to the data reduction team across the two batches. However, 33 trips were excluded because (1) they did not go through the initial data ingestion quality assurance process required for any trip used in the SHRP 2 data set, (2) the consented driver was not present for the trip, or (3) the videos or alcohol sensor data did not properly load. The final data set of 659 trips served as the naturalistic test data set. Alcohol-Impairment Behavioral Checklist and Data Reduction Checklist Other than the alcohol sensor and the kinematic sensors, the camera views of the standard SHRP 2 instrumentation package provided the most reliable method for determining Table 2.1. Mechanical Breather (Gold Standard) Data Set Conditions Experimental Trials Total Samples Control (no BrAC) 6 12 Human Impaired 11 22 Boozooka 33 66 Total 50 100

9 intoxication. Camera views captured the forward roadway, rear view, driver’s face, and driver’s hands. Reductionists looked at all camera views to determine impairment. Data reductionists checked for and coded signs of impair- ment across several broad categories that included visually seeing alcohol sources within a vehicle (both imbibed and unimbibed), behavioral cues from the face-view camera, and driving performance. The data reductionists observed and coded these categories of data and were then asked to make subjective evaluations of driver impairment and degree of impairment. The behavioral checklist was made after carefully review- ing and synthesizing information from literature related to visual signs of impairment (5–10). From this literature more than 90 behavioral and visual cues were identified for assessing alcohol intoxication. Out of this broader list of iden- tified cues, 64 cues were deemed observable using SHRP 2 video. These cues and operational definitions are provided in Appendix B. Reductionists also coded signs or presence of unimbibed alcohol. They were given a protocol with a list of known alco- hol sensor triggers: fast food, hand sanitizer, perfume, cologne, cigarettes, marijuana, other drugs, chewing gum, and wind- shield wiper fluid. Reductionists had the option of noting in the data file any other substances that they deemed suspicious or that might contain any form of alcohol. Data Reduction Procedure Six specially trained data reductionists from the Virginia Tech Transportation Institute (VTTI), all of whom had experience working with intoxicated individuals in a field setting for other research efforts, reduced the trip files. In previous endeavors they collected BrAC data from pedestrians in a bar-centric downtown district; on average, each individual had spent over 100 hours conducting research in this field setting with intox- icated individuals (11–13). Furthermore, each of the data reductionists was trained specifically for this project by a data reduction supervisor. Four of the data reductionists served as the primary data ana- lysts for trip files. These reductionists filled out a check sheet that coded behavioral cues for impairment (see Appendix B), noted any observed types of unimbibed alcohol, noted indica- tors of poor driving performance, and made determinations about the driver’s and passenger’s level of intoxication. These data reductionists were allowed to examine all video views but were instructed not to look at alcohol sensor readings to remove potential bias. Reduction of each video took approximately 10 to 15 minutes with the data reductionists watching the first 2 minutes, last 2 minutes, and 3 minutes at random points in between. Additionally, the data reductionists watched most of the video at 10× speed to find possible sources of unimbibed alcohol and identify critical events that could be particularly informative in determining impairment. One data reductionist served as the alcohol sensor valida- tor and, to avoid bias, was the only individual allowed to look at the alcohol sensor readings. This individual was instructed to consider the alcohol sensor and all video views to further examine points in the trip when the alcohol sensor drastically changed. This reductionist would then code events happening around that time period in an Excel file separate from the one used by the four primary reductionists. This procedure ensured that sources of unimbibed alcohol were not missed since it was difficult for the primary reductionists to catch every substance used, particularly during longer trips. Unimbibed alcohol was caught most often because it usually produced sharp spikes in sensor readings. However, this procedure was also able to iden- tify intoxicated individuals entering the vehicle in designated driver (DD) scenarios, alcoholic beverages being poured within a vehicle, and other similar types of impaired trips. The alcohol sensor validator was not allowed to edit the Excel log of the primary data reductionists and did not determine impairment. At the conclusion of the data reduction effort, a senior member of the research team updated the primary data file to include substances identified by the alcohol sensor validator. Again, only the timestamps and identification of substances were updated, not decisions regarding intoxication. The final data reductionist served as the quality assurance reviewer who went through and validated the work of the four primary reductionists. This reductionist added comments to the data file when discrepancies of judgment were identi- fied, which caused the original reductionists to review their comments. When the data reductionists agreed with the changes, they were entered into the data file. In the event of disagreement, the data reduction supervisor or senior mem- ber of the research team would view the trip and make the final determination. Naturalistic Test Data Set As shown in Figure 2.3, the naturalistic test data set includes 659 trips. Of these trips, 562 were initially chosen because of a preliminary positive alcohol sensor reading and their late- night occurrence when alcohol use was more likely (Batch 1). The remaining 97 trips served as a control group and were selected randomly, irrespective of the time of day (Batch 2). All 659 trips were reduced via manual video coding to iden- tify possible impairment and possible sources of alcohol, including potential false positives. Although the test data set did contain control trips, it con- tained over five times more trips with likely alcohol presence. This ratio is much different from what would be expected in the SHRP 2 database. The SHRP 2 database would be expected to have far fewer trips with alcohol present than trips without

10 Batch 1: Late Night and Alcohol Sensor Positive Trips n = 562 Batch 2: Randomly Selected Control Trips n = 97 Complete Data Set n = 659 Moderately Impaired Trips n = 91 Control Trips n = 97 Trip Batches Impaired Data Set Naturalistic Test Data Set Figure 2.3. Distribution of trip files into relevant analysis databases. alcohol present. Therefore, this data set was not designed to determine the sensitivity and specificity that an algorithm would have when compared with the SHRP 2 database—it was too biased toward positive alcohol readings. Instead, this data set was designed to investigate the accuracy of the alcohol-detection algorithm in differentiating imbibed from unimbibed alcohol because reliably separating false positives from true positives was a primary objective of the study. A second objective of the naturalistic test data set was to determine how well impaired driving trips could be differen- tiated from unimpaired driving trips. As shown in Figure 2.3, a subset of the naturalistic test data set, the impaired data set, was used to test this objective. The impaired data set included all the randomly selected control trips (Batch 2) and trips from the naturalistic test data set identified by reductionists as having at least one vehicle occupant who was moderately impaired. Thus the impaired data set comprised 188 trip files, including 91 trips in which video reductionists rated at least one vehicle occupant as being “moderately” impaired and 97 randomly selected trips serving as a controlled baseline. The impaired data set was designed specifically to show how well the algorithm could differentiate moderately impaired driving from normal driving. Analyses were conducted sep- arately on both the complete naturalistic test data set and the impaired data set derived from this sample. Neither the naturalistic test data set nor its subset, the impaired data set, provided a representative estimate of the algorithm’s effectiveness across the entire SHRP 2 database, as developing a data set for this purpose would have been cost prohibitive. For example, out of the 97 control trips, only one indicated potential driver impairment. Thus, a large number of randomly selected trips would need to be evaluated to find a sufficient number of potentially impaired trips to assess this algorithm, an effort that was outside the budgetary constraints of the project.