Visualization of Highway Performance Measures (2022)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

88 This chapter includes a summary of the reported key success factors and challenges identified in the synthesis. Areas suggested for future research based on the findings are also presented. Reported Key Success Factors DOTs are developing methods to visually communicate the stories that their performance data tell in ways that work. They are finding ways to tell their performance stories with effective visualizations that resonate with their audience. The key success factors identified by the DOTs that responded to the synthesis survey and participated in the follow-up interviews include the following: • Simplicity in the design of the visualizations. This was one of the strongest associations made with perceived success according to survey and interview responses. Simple charts account for more than 80% of the visualization examples shared by responding DOTs. The phrase “keep it simple” was repeated by representatives from all five state DOTs in the interviews. Success was often reported by those interviewed when they felt a simple visualization effec- tively communicated a clear story that brought the significance of the raw data to life for its intended audience. • A strong foundation of performance measure data. Survey and interview responses indicated that this foundation is necessary to enable analysts to analyze and share information accu- rately and effectively. Complementary to the data is a foundation in the software tools for col- lecting, managing, and creating visualizations. These tools are becoming more sophisticated, easier to use, and more powerful. • Connecting to the intended audience. DOTs report a key success factor is the significance of identifying and connecting to the intended audience. The DOT dashboards examined in detail for this synthesis all deliver dashboard information in a structured system, which the case example interviews expressed were designed intentionally to communicate with multiple levels of clearly defined audiences. The structure begins with the more general- ized big picture (forest) and leads users to more specific measures (trees) and ultimately far more detailed facets, usually only of interest to specialized analysts (leaves). State DOTs interviewed for case examples claimed that sharing the same data, information, and stories with both internal and external partners helped interested parties work better together to accomplish common goals. • Recognizing intended internal or specific stakeholder audiences for success. For example, one common DOT response stated that an information-rich dashboard is an educational tool that allows person-to-person time both within the agency and with external stakeholders to focus on explanation, analysis, and solution strategy planning instead of locating data and find- ing reports. This maximizes increasingly limited face-to-face meeting time in the workplace. C H A P T E R   5 Summary of Findings

Summary of Findings 89   Several respondents indicated that a story-rich dashboard that supports the executive leader- ship’s message can help focus the organization’s work in meeting their stated objectives. • An emphasis on the role of executive leadership. Nearly all of the case examples reflect that a top-down and bottom-up approach more often results in a dashboard that resonates with the intended audiences and drives both the desire for more information and the willingness for more participation. Reported Challenges The synthesis revealed commonly reported challenges DOTs identified in their efforts to create and maintain effective tools for visualizing performance measure data. DOTs indicated that simplicity is a key success factor, and they also indicated that achieving simplicity is hard. Survey respondents indicated that achieving the desired level of clarity and simplicity they identified as their goal required considering many factors such as the text that describes the data story, the clarity of the message, and the effectiveness of the visualization. A strong foundation of performance measure data was identified as a key success factor. The challenge to that success factor is in establishing the data governance. Survey and interview results reflect that many DOTs consider the lack of guidance on best practices regarding the gathering, storage, maintenance, linkage, and usage of the data that underlie the accuracy and effectiveness of all visualizations to be one of the most significant challenges they see. Telling meaningful data-based performance stories is more than just reporting data. Visual- izations that address the broad spectrum of challenges to mobility throughout the state and inform multimodal solutions involve building partnerships within DOTs and between inter- ested stakeholders that enable collaboration. The challenge DOTs acknowledge is that increas- ing amounts of data make it difficult to deliver performance measures effectively to all intended audiences. A key success factor is the role of executive leadership. Engaging their input in shaping the DOT stories that incorporate performance accurately and tell the performance story was com- mented on during the interviews with the state DOTs. Those interviews noted the significance and importance of having executive leadership’s support and engagement, particularly within the DOT. Given the significant demands on the time and resources managed by executive leader- ship, it is a challenge to ensure that essential engagement. A visualization is not effective if it is not seen or remembered. Given the competition for time everyone faces, just because a visualization exists does not mean it is seen. Even when accurate factual data are presented in a compelling story, it still may not be acted upon. For example, even with clear visualizations providing insight to solutions that could solve problems (e.g., visual- izations that show the effectiveness of wearing seat belts or not texting), there are still a sig- nificant number of people who ignore or are unaware of that data and experience fatal crashes they may have survived if they chose to buckle up or not text while they drove. A challenge to determining the effectiveness of visualizations is acquiring feedback on their effectiveness. The survey and interview results reflect that the guidance for evaluating a visualization is not often applied in a systematic review process and that audience feedback that evaluates the effective- ness of a visualization is rarely gathered. DOTs noted the increased transparency resulting from the reporting on performance mea- sures. Most interview responses posited that this increased transparency was an overall positive that they feel contributes to increased trust in the system and its measures. This is particularly the case when the performance story told by the data is not positive. DOTs are having to adapt

90 Visualization of Highway Performance Measures and become more resilient and responsive to the increased transparency that accurate perfor- mance measure reporting creates. Suggestions for Future Research A clear need identified by this synthesis is a set of principles that can be used to evaluate visual- izations internally and externally. A Dashboard and Visualization Toolkit that provides best practices for creating visualizations would better enable evaluations of visualizations internally to a DOT. Future research could provide guidance for gathering survey data that could inform the effec- tiveness of visualizations in reaching their intended audiences and either informing them or helping to influence behavior. It could potentially develop methodologies to calculate a return on investment of visualizations. This would better enable evaluations of visualizations after they’ve been produced and serve as a feedback loop to improve the effectiveness of the visual- izations that could be periodically integrated into the Toolkit noted above. These two research efforts would help DOTs that have been leading visualization efforts and DOTs just beginning to expand visualization capabilities.