The National Academies Press

Currently Skimming:

Pages 94-113

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.

From page 94... ... 94 Although most states understand the value of collecting and analyzing data to guide their business decisions, most fail to grasp the scale of the data, the expertise needed for Big Data analytics, and the significant shift away from traditional approaches (including approaches to data collection and analysis, data storage and management, and procurement of IT services) that would be required before the implementation of Big Data. Read the entire page →
From page 95... ... Big Data Guidelines for TIM Agencies 95 and videos; (2) performing advanced statistics; (3) Read the entire page →
From page 96... ... 96 Leveraging Big Data to Improve Traffic Incident Management 6.1 Adopt a Deeper and Broader Perspective on Data Use Traditionally, many organizations have conducted business by relying on business intelligence (often reported on the basis of limited data) , on expert opinions, and even on intuition. Read the entire page →
From page 97... ... Big Data Guidelines for TIM Agencies 97 better the chance of discovering patterns and behaviors that can be tracked, analyzed, predicted, and embedded into organizational decision-making processes. Without enough detailed data, however, Big Data analytics is not possible. Read the entire page →
From page 98... ... 98 Leveraging Big Data to Improve Traffic Incident Management Table 6-1 lists TIM-relevant datasets that could be leveraged to build a data lake. For each dataset, the table provides the readiness for and associated challenges associated with integration of the data into a Big Data data lake. Read the entire page →
From page 99... ... Big Data Guidelines for TIM Agencies 99 One of the foundational aspects of Big Data analytics is the ability to explore and correlate a range of very large datasets to uncover unknown relations and patterns that could lead to an improvement in the state of the practice. If the data is shared in a previously aggregated or summarized form (as opposed to raw form) Read the entire page →
From page 100... ... 100 Leveraging Big Data to Improve Traffic Incident Management the cost and time needed to export or even recreate the data generally is not prohibitive. When dealing with Big Data analytics, however, the much larger size of the datasets involved and the constantly evolving variety of analyses and visualizations that can be performed mean that a Big Data dataset created using a proprietary file format significantly risks future accessibility and value. Read the entire page →
From page 101... ... Big Data Guidelines for TIM Agencies 101 6.4 Use a Common Data Storage Environment A common data storage environment is vital for Big Data. In traditional data analysis, one or more datasets are imported into an analytic tool or platform like a relational database or a statistical software package and processed on the workstation or server where the analytical tool is installed. Read the entire page →
From page 102... ... 102 Leveraging Big Data to Improve Traffic Incident Management Data virtualization could easily allow for siloed datasets across an organization to be organized, managed, and queried without ever having to relocate the data into common physical storage; however, this approach has two main weaknesses. First, virtualized common data stores depend greatly on the performance and quality of the individual (siloed) Read the entire page →
From page 103... ... Big Data Guidelines for TIM Agencies 103 level of service by the cloud service providers. As a result, the prime concern of an organization using cloud infrastructure is no longer to ensure the reliable and sustainable operation and maintenance of the IT infrastructure underlying its data workflows. Read the entire page →
From page 104... ... 104 Leveraging Big Data to Improve Traffic Incident Management In addition to growth-driven variations in processing power and storage, Big Data analytics adds a second layer of power and storage variability, as the analyses involved typically are not processed evenly over time. Dataset processing is rather irregular and includes large spikes driven by human decisions, environmental changes, or the obsolescence of data models, which can occur at any time. Read the entire page →
From page 105... ... Big Data Guidelines for TIM Agencies 105 it does happen to a tiny fraction of the data. In the occurrence of such a rare failure, files generally can be recovered from server backups within a couple of days. Read the entire page →
From page 106... ... 106 Leveraging Big Data to Improve Traffic Incident Management Transportation and TIM agencies have two options for adopting cloud services: • The first option is to use a commercial cloud service provider. This option is also the easiest to implement and would allow the transportation or TIM agency to benefit from an available, very large, and very flexible cloud-based infrastructure at a low cost. Read the entire page →
From page 107... ... Big Data Guidelines for TIM Agencies 107 been stored, these new analyses will not be possible. Storing the data in its raw format allows multiple analysts or researchers to perform differing analyses on the same data at the same time to confirm analytical results, assess the validity of statistical models, or directly compare findings across studies. Read the entire page →
From page 108... ... 108 Leveraging Big Data to Improve Traffic Incident Management used to tie the two sources together. The lack of a common field makes it difficult to integrate the datasets. Read the entire page →
From page 109... ... Big Data Guidelines for TIM Agencies 109 • Because removing sensitive data can negatively affect the ability of the datasets to be mined in detail or merged with other datasets, alternative techniques to obfuscate sensitive data may be considered. Obfuscation methods like hashing techniques and encryption can anonymize personal information, but the methods used need to be sufficiently strong. Read the entire page →
From page 110... ... 110 Leveraging Big Data to Improve Traffic Incident Management resides, without moving it, and the results typically are written to the same location. Consequently, Big Data analytics tools run directly on top of Big Data stores by moving the analytics tools through the data across multiple servers. Read the entire page →
From page 111... ... Big Data Guidelines for TIM Agencies 111 be modified, copied, and redistributed. Software developers can even strip out useful parts from one open-source project to use in their own products. Read the entire page →
From page 112... ... 112 Leveraging Big Data to Improve Traffic Incident Management In comparison, if a proprietary software is used, the necessary flexibility would come at a significant cost, as software licenses would have to be purchased in advance to cover possible spikes and bursts in processing and future growth. By contrast, scaling up proprietary software means purchasing any necessary additional licenses -- which, if overlooked, can lead to exorbitant penalties. Read the entire page →
From page 113... ... Big Data Guidelines for TIM Agencies 113 predict the various ads that website visitors will be interested in seeing. Because the predictions lose accuracy within days or hours, the models are constantly retrained to maintain prediction accuracy over time. Read the entire page →

From page 94...

... 94 Although most states understand the value of collecting and analyzing data to guide their business decisions, most fail to grasp the scale of the data, the expertise needed for Big Data analytics, and the significant shift away from traditional approaches (including approaches to data collection and analysis, data storage and management, and procurement of IT services) that would be required before the implementation of Big Data.