Application of Big Data Approaches for Traffic Incident Management (2023) / Chapter Skim
Currently Skimming:
Pages 37-70
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 37... ...
37 This chapter presents and discusses the data pipelines that were developed to demonstrate the feasibility and value of selected TIM big data use cases: • Use Case 1: Improving Incident Detection and Verification to Expedite Response. • Use Case 2: Real-Time TIM Timeline and Performance Measures.
|
From page 38... ...
38 Application of Big Data Approaches for Traffic Incident Management 2 Figure 19. Example N2 diagram.
|
From page 39... ...
TIM Big Data Use Cases 39 • Document database. Also known as a document-oriented database or NoSQL (not only Structured Query Language)
|
From page 40... ...
40 Application of Big Data Approaches for Traffic Incident Management Figure 20. N2 diagram for Use Case 1.
|
From page 41... ...
TIM Big Data Use Cases 41 in kilometers, is the least precise, and level 12, measured in millimeters, is the most precise.) (Khadka & Singh, 2020)
|
From page 42... ...
42 Application of Big Data Approaches for Traffic Incident Management 3. Deduplicate data.
|
From page 43... ...
TIM Big Data Use Cases 43 as they are reported and made available through the system. This dashboard is hosted and made available through the GIS platform to take advantage of the hosted data and to allow for the most efficient method of continually displaying real-time data as it is delivered to the GIS platform environment.
|
From page 44... ...
44 Application of Big Data Approaches for Traffic Incident Management Minnesota Dashboard Active Alerts Figure 23. Screenshot dashboard displaying Minnesota.
|
From page 45... ...
TIM Big Data Use Cases 45 Standardization of the data in this use case could be expanded to provide a single dashboard of the national feed of navigation app traffic alerts. This could allow partnering agencies to readily observe neighboring situations that could impact the system they monitor, particularly in the northeastern states in which interstate travel frequently occurs.
|
From page 46... ...
46 Application of Big Data Approaches for Traf c Incident Management better handle and analyze data and make data available for query. ese innovations in speed, and the level of precision available from mobile phones or CVs, will continue to improve the precision of location coordinates, which could allow this and similar data pipelines to be re ned.
|
From page 47... ...
TIM Big Data Use Cases 47 TIM programs encounter data-related challenges when attempting to populate many of the key points on the TIM timeline, including those associated with the national TIM performance measures recommended by FHWA: RCT and ICT. The data-related challenges include a lack of internal DOT data for all incidents; lack of access to data that resides with other responder agencies (e.g., law enforcement)
|
From page 48... ...
48 Application of Big Data Approaches for Traffic Incident Management Figure 27. N2 diagram for Use Case 2.
|
From page 49... ...
TIM Big Data Use Cases 49 7. Standardize.
|
From page 50... ...
50 Application of Big Data Approaches for Traffic Incident Management 7. In parallel with Step 6, the received navigation app jam alerts, CAD events, and navigation app crash alerts are sent to a roaming geofence streaming database.
|
From page 51... ...
TIM Big Data Use Cases 51 9. As crash data are updated by the processes in Steps 7 and 8, that information is continuously pushed to a cloud function, which sends updates to the cloud database for long-term flat storage for historical analysis and makes the data ready for further display and analysis.
|
From page 52... ...
52 Application of Big Data Approaches for Traffic Incident Management Figure 29. Crash document created at the end of the data pipeline (Minnesota)
|
From page 53... ...
TIM Big Data Use Cases 53 successfully matched to all three data sources -- navigation app crash alert (T0) , navigation app jam alert (T7)
|
From page 54... ...
54 Application of Big Data Approaches for Traffic Incident Management 4.2.5 Lessons Learned During the development of Use Case 2, there were several lessons learned associated with the data used, interacting with the cloud environment, and the general data pipeline. These lessons learned, and any associated recommendations, are detailed in the following list.
|
From page 55... ...
TIM Big Data Use Cases 55 The addition of a data element to capture secondary crashes provides an opportunity for analysis that previously could not be conducted. As such, the approach for this use case was to combine secondary-crash data from multiple states; enrich the data with traffic, roadway, and weather data; and apply big data techniques to uncover relationships and trends that had not been systematically identified with previous approaches.
|
From page 56... ...
56 Application of Big Data Approaches for Traffic Incident Management Figure 30. Crashes flagged as secondary for 10 states.
|
From page 57... ...
TIM Big Data Use Cases 57 6. The Standardize function receives the matched crashes to provide consistency, then passes the standardized data to the Merge function.
|
From page 58... ...
58 Application of Big Data Approaches for Traffic Incident Management from the secondary crashes, the team identified crashes that occurred within two hours (prior to) and within 2 kilometers (in either direction)
|
From page 59... ...
TIM Big Data Use Cases 59 4.3.4 Data Products Unlike the other use cases in this report, Use Case 3 tested methodological approaches for data analyses, rather than producing a singular dashboard product or actionable outcome. This use case includes the steps taken to enrich secondary-crash data, to verify secondary crashes by identifying associated primary crashes, and to analyze the verified secondary crashes.
|
From page 60... ...
60 Application of Big Data Approaches for Traffic Incident Management Clear Day Clear Night Partly Cloudy Day Partly Cloudy Night Wind 2,800 2,600 2,400 2,200 2,000 1,800 1,600 1,400 1,200 1,000 800 600 400 200 0 N um be r of S ec on da ry C ra sh es Cloudy Rain Snow Fog Null Cluster 1 2 3 Figure 34. Cluster results for weather conditions.
|
From page 61... ...
TIM Big Data Use Cases 61 contains a higher proportion of secondary crashes that occurred on lower-classification roadways (i.e., other principal arterials, minor arterials, major collectors)
|
From page 62... ...
62 Application of Big Data Approaches for Traffic Incident Management 0% 20% 40% 60% 80% 100% Verified Original Verified Original W ith U nk no w n W ith ou t U nk no w n Angle Front to Front Front to Rear Rear To Side Sideswipe, Opposite Direction Sideswipe, Same Direction Other/Unknown Figure 36. Comparison of manner of secondary crashes between original and verified datasets, with and without unknown data attributes.
|
From page 63... ...
TIM Big Data Use Cases 63 the secondary crashes where "driver skills" was a circumstance, it is less of a factor in the verified dataset (43.5 percent) than the original dataset (61.5 percent)
|
From page 64... ...
64 Application of Big Data Approaches for Traffic Incident Management 4.4.3 Description of Data Pipeline The N2 diagram for the Use Case 4 data pipeline is shown in Figure 38, which illustrates the various components of the system configuration. The data pipeline contains the following functions and steps.
|
From page 65... ...
TIM Big Data Use Cases 65 (in the crash data) , the Distance-Calculate function uses the latitude and longitude from the datasets to calculate the distance between the crash in the crash data and CV data that are within 10 minutes before and after the crash occurred.
|
From page 66... ...
66 Application of Big Data Approaches for Traffic Incident Management requirements, which may include encryption and authentication procedures so that only those with adequate licensing and rights can access the data. For this reason, this step may require coordination with IT staff to perform the secure data retrieval.
|
From page 67... ...
TIM Big Data Use Cases 67 of the crash, or after the crash)
|
From page 68... ...
68 Application of Big Data Approaches for Traffic Incident Management relies on known crash locations so that associated changes in system performance can be used to develop performance thresholds to identify crashes in real time. 4.4.4.2 CV Driver Event Data on Crash Detection Dashboard The CV Driver Event Data on Crash Detection Dashboard is intended to allow users to explore the information associated with the matched CV driver event data and crashes.
|
From page 69... ...
TIM Big Data Use Cases 69 4.4.5 Lessons Learned During the development of this use case, there were several lessons learned while interacting with the data in a cloud environment and general data pipeline: • Immediately after a crash, road network performance may be impacted near the crash location and then can begin to spread upstream of the crash, in the opposite direction of the crash, and onto connecting roadways. However, detected changes in performance may lead to falsely identifying that a crash has occurred when, in fact, it has not.
|
From page 70... ...
70 Application of Big Data Approaches for Traffic Incident Management • Color gridded symbology of average speeds reported (raw) versus expected speeds.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.