Programmatic Issues of Future System Performance (2022)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

17 CHAPTER 3: ANALYTICAL PROCEDURE DEVELOPMENT The research team developed a simple analytical procedure to organize and prioritize the full set of 165 refined research gaps presented in the white papers. This analytical procedure allowed the research team to: • Further refine the full set of research gaps into a more manageable “longlist” of 30–50 research gaps from which a set of proposed RPS was identified through a survey and workshop with SMEs who participated in the listening sessions. • Structure components of a self-assessment tool designed to aid STAs to identify and prioritize critical issues and associated threats and opportunities that characterize the same future transportation challenges embodied by the research gaps. The tool also facilitates the development of strategies to address these challenges in agencies’ programmatic initiatives. This chapter describes the steps in the analytical procedure. Chapter 4 presents the proposed research problem statements, and chapter 5 explains the self-assessment tool. Analytical Procedure Steps Figure 1 outlines the steps in the analytical procedure. The process began with the 165 refined research gaps derived from the analysis of the CIHS and CIT2019 reports, literature review, and listening sessions, and concluded with a “longlist” of 43 research gaps that were further prioritized and developed into 18 RPS. Figure 1: Analytical Procedure.

18 Organizational Elements (Steps 1–3) The research team began by working with the refined research gaps within an “organizational spreadsheet” and proposing and validating a set of research tracks, sub-tracks, and keywords around which to structure a theoretical research program. While the ultimate research problem statements do not explicitly reference this structure, these organizational elements of a theoretical program allowed the research team to prioritize the research gaps to compile a more manageable longlist that targeted 30–50 gaps. The research team used the organizational elements—research tracks, sub-tracks, and keywords—to first categorize each refined research gap in a spreadsheet. The research team identified research tracks within each of the six interest areas starting with the topics of the listening sessions and adjusting them based on what was actually discussed during each listening session. Sub-tracks derived from the unified framework’s key research questions (see Table 3 in chapter 2). Sub-tracks were then validated by (1) developing sets of key words or phrases for each research track to confirm adequate topical coverage, and (2) comparing the sub-tracks against all refined research gaps to ensure each research question could be adequately categorized into at least one research sub-track. Table 4 presents the results of this categorization at the research track level. Some variation exists in the level of coverage; for example, transformational technologies have twice as many research gaps as the others, which range from 18 to 25. Variation also exists at the research track level. However, many issues are cross-cutting issues (as captured among the sub-tracks and keywords listed in Appendix C) that suggest a more even balance of issues than the research gaps by interest area might suggest. An important example is equity, with 19 research gaps identified at the research track level. However, equity issues are also identified among all other interest areas at the sub-track level, illustrating its cross-cutting significance as a future challenge for transportation agencies. Table 4: Number of Research Gaps by Interest Area and Research Tracks. Interest Area Research Tracks Research Gaps Transformational Technologies All transformational technologies (advanced aerial mobility, connected and/or automated vehicles, EVs, freight, personal life technology) 57 System Performance and Condition Performance management 13 System resilience 7 Data management 5 System Use Multimodal connectivity and system integration 13 Socio-demographic factors and land use 11 System Impacts and Externalities Land use policy and planning 7 Mitigating negative social outcomes 1

19 Interest Area Research Tracks Research Gaps Sustainability and greenhouse gas reduction 14 Organizational Capacity and Governance Multi-jurisdictional partnerships 7 Organizational capacity 11 Equity Equity in transportation 19 Total 165 Refined Threats and Opportunities (Step 4) Step 4 refined the threats to and opportunities for system performance and agency effectiveness within the unified framework’s six interest areas (as summarized in the white papers) into a list of more than 160 statements and mapped them back to each interest area. The threats and opportunities were then incorporated into the organizational spreadsheet of research gaps by interest area. Appendix D presents this list of threats and opportunities by interest area. Refined Research Gap Prioritization Criteria and Process (Steps 5–8) For the next step in the analytical procedure, the research team developed the following prioritization criteria with which to identify a targeted top 30–50 refined research gaps for the workshop’s longlist: • The degree to which conducting the research would address relatively more threats and opportunities than other gaps. • The scale of potential impact conducting the research would have (e.g., it addresses multiple interest areas, research tracks, or sub-tracks). • If the anticipated research outcome would address a more immediate need in the 10-year timeframe, rather than the 20-year timeframe (given the range suggested by this project), to focus proposed research on issues with less uncertainty and more implementable outcomes. • Whether the gap is best addressed through “NCHRP research,” as opposed to applying government “policy” or conducting “debate” among stakeholders, as suggested for response options to the questions raised in CIT2019. Working within the research gap organizational spreadsheet, the research team used a two-step internal process to apply the criteria and develop the research gap longlist. First, core members of the research team who are intimately familiar with the development of the refined research gaps; organizational elements; and threats and opportunities applied the criteria and compared their results. These individuals assigned each research gap a ranking as follows: • 1 = top priority, include the research gap

20 • 2 = medium priority, potentially consider the gap based on others’ results • 3 = low priority, exclude the research gap In addition, these team members indicated if a research gap could be combined with an already- selected top priority gap to yield a more comprehensive or cohesive research gap statement that might reduce overlap among gaps and/or capture potentially cumulative benefits into a single prioritized research gap. The second step of the internal process was to meet with the research team’s panel of former state DOT executives, discuss the prioritization criteria and process, and ask each of them to review the results. Panel members confirmed or adjusted the core team members’ priorities, particularly through the vantage point of their prior executive experience and understanding of the likely future needs of agencies. The panel and core research team also considered any imbalances among the selections across interest areas and research tracks (as discussed earlier). At the end of Step 8 the research team arrived at a longlist of 43 research gaps. Engagement with Transportation Subject Matter Experts (Step 9) This longlist was refined to a prioritized shortlist of 18 items through a survey of transportation SMEs who participated in the listening sessions. The survey asked participants to rate each research gap statement based on two criteria: • Feasibility – What is the feasibility (positive) of the research need from your professional perspective? Consider the time sensitivity and criticality of underlying issues and the magnitude of impacts. • Desirability – What is the feasibility of undertaking the research and advancing the work products to implementation from your professional perspective? Consider the outcome’s influence, level of effort, resource needs, organizational constraints, timeline, and readiness for implementation. The research team used the survey results to develop 18 draft RPS for consideration at a virtual workshop with survey participants. The workshop and development of the RPS are discussed further in the next chapter.