Mobility on Demand and Automated Driving Systems: A Framework for Public-Sector Assessment (2022)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

82 CHAPTER 7: IMPACT ASSESSMENT FRAMEWORK IMPACT ASSESSMENT FRAMEWORK SECTION OVERVIEW This section discusses an impact assessment framework and answers key questions including: • What is an impact assessment framework? • What steps make up the framework? • What are the next steps after the implementation of the framework? WHAT IS AN IMPACT ASSESSMENT FRAMEWORK? Pilot projects are laboratories for testing and implementing large-scale change as well as creating a model that other communities can replicate and customize. Pilot projects may occur at different scales including: • Neighborhood: Agencies or other stakeholders specifically design these smaller projects to meet neighborhood needs and contexts. Neighborhoods can include residential areas (e.g., apartment complexes, planned unit developments), universities, office parks, airports, and business improvement districts. • City/town: City or town agencies or other organizations typically institute these pilots in an attempt to respond to city- or town-wide needs. • Regional: Regional entities are frequently responsible for implementing larger-scale regional pilots, which involve a macro-level implementation approach (e.g., includes a broader range of geographies and demographics). Source: Shaheen et al., 2019 Assessment frameworks can predict, identify, and measure potential impacts resulting from pilot projects for, or the integration of, AVs. The framework can assess impact areas including travel behavior, the environment, user demographics, the economy and labor, and social equity. In Understanding How Cities Can Link Smart Mobility Priorities Though Data, Shaheen et al. (2018) developed an impact assessment framework, which serves as the basis for this example. Expert interviews and an extensive literature review provided additional insights on this framework. The framework is applicable across any MOD and/or AV project and provides a consistent yet flexible lens to evaluate the impacts of such projects. Impact Categories The impact assessment framework can assess a variety of impact areas including: • Behavioral shifts, • Economics and labor, • Environment and energy,

83 REPORT • Equity and inclusion, • Goods delivery, • Infrastructure and land use, • Operational efficiency, • Public perception, • Public transit ridership, • Technical accomplishments, • Traffic congestion, and • Traveler safety. Together, these impact categories cover the relevant areas in which public transit agencies may wish to assess AV impacts. The Shared Automated Vehicle Toolkit presents information and ideas on topic areas (e.g., organizational readiness, rights-of-way considerations) that the impact assessment framework may evaluate. Performance metrics and data types enable an assessment of these impact categories. WHAT STEPS MAKE UP THE FRAMEWORK? The impact assessment framework is comprised of six key steps that researchers can use either chronologically or through an iterative process, depending on the context of the project. Different projects and potential contexts can be found in the Shared Automated Vehicle Toolkit: Policies and Planning Considerations for Implementation. Although sequential language is used to describe each step, successfully meeting project goals often requires an approach that accommodates changing requirements or circumstances. Figure 18 summarizes this approach. The framework is designed to incorporate a feedback loop between steps, allowing researchers, public agencies, and other stakeholders the flexibility to refine their methods without compromising milestone progress or project delivery. Figure 18. Implementation Process Framework Map Based on Implementation Scale Researchers, agencies, or other organizations can map a project by the scale of focus from neighborhood to city or town to regional. Mapping a project based on these scales can assist an organization in organizing and understanding influences on the project and identifying the appropriate quantitative approach. Map Based on Scale Define Project Objectives Define Project Hypothesis Define Project Metrics Define Project Data Types and Sources Define Analysis Methods

84 REPORT Define Project Objectives The next step highlights the importance of identifying the project’s (or broader organization’s) main goals and objectives. This is usually a straightforward process that can help clarify what objectives are realistically achievable. If applicable, the researchers should define the objectives’ associated direction (e.g., increase, decrease) and user population or system context (e.g., low- income households). Not all objectives may describe measurable impacts. In these cases, objectives will be defined differently and will likely require a more qualitative treatment rather than a quantitative assessment. Define Project Hypotheses This step encompasses translating the objectives into hypotheses that more concretely state the expected outcome. Researchers are ultimately using the framework to test whether the analysis results agree with the hypotheses and whether the established objectives were met. It can be helpful to reword the hypotheses as questions, especially in cases when the expected outcomes are less clear. This prevents the study from oversimplifying and categorizing results as completely successful or altogether insufficient. The distribution of results does not often neatly fit into one single idea or outcome. The researchers need to define hypotheses and corresponding questions that can capture a variety of responses. Define Project Metrics In the fourth step, the researchers identify what they need to measure to assess whether the established hypotheses are supported and to answer any associated questions. It is essential to define metrics that appropriately capture the ideas presented in the hypotheses and questions. The project metrics identified may not be exhaustive and may need to be supplemented with measures that assess specific components of the pilot project that were not covered within the initial scope of questions. In addition, researchers may need to revisit and redefine metrics to account for gaps in data availability. Researchers may grapple with limited data availability and need to be prepared to propose alternative metrics when data sources are inadequate. Challenges with data granularity and availability may result in the inability to calculate desired metrics. Regardless, this procedural step can help reveal important data gaps to inform future data collection efforts and research. Define Project Data Types and Sources In the fifth step, researchers identify data types that can populate the established metrics. Researchers can use different data types to develop metrics. These are types—as opposed to sources—since the identification of sources is also context specific. Occasionally, the type and source are the same (e.g., census data). In general, though, most data will come from MOD and AV service operators, and the public transit agencies and data types can be specified depending on the pilot project. The availability of complete and accessible data at the required level of granularity for a study is context specific. The data may exist and be accessible for one pilot project conducted in area A

85 REPORT but not for another pilot project conducted in area B. It is at this point in the process that researchers need to determine whether metrics can be computed with the available resources. If the data sources prove inadequate, it is important to understand the extent to which the missing information forms a data gap. In cases where there is a data gap, it may be useful to redefine metrics or search for alternative data sources. These alternative data sources are likely to be less- than-ideal proxies but can provide at least some insight into the hypothesis in question. In addition, metrics often require multiple types of data from disparate sources. In many cases, researchers want to compare before and after data, referring to data produced prior to the implementation of the pilot project and data generated by itself, respectively. These datasets are not distinguished since they are derived from the same data types. In addition, specific metrics are not associated with specific data types. As mentioned in the framework description, metric computation frequently requires multiple types of data from different sources. As a result, the metrics do not fit neatly into one data type or another. Researchers often use the same data types to calculate different metrics and address different questions. Researchers can use the data types listed in Appendix D: Data Types.