Shared Automated Vehicle Toolkit: Policies and Planning Considerations for Implementation (2022)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

77   This chapter provides a toolkit that can be used to evaluate MOD and AV pilots and deployments. Tools for the Pilot and Implementation Framework Tools included in this chapter: • Summary of the evaluation framework. • Description of phases in the evaluation framework. • Example of using the framework. • Next steps after completing an evaluation. This toolkit has provided information on a variety of ways, environments, and uses for MOD and AVs. However, understanding the impacts of the implementation of these innovations is critical for service operations. This chapter presents a framework that can assist public agencies and other stakeholders in developing, implementing, facilitating, and/or evaluating MOD, AV, and goods delivery services. The framework can help provide benefits, such as • Offering strategies for using MOD and AVs to reach community and regional goals; • Assisting communities in deciding which transportation innovation opportunities to support; • Evaluating research and associated impacts from organizations, such as the federal government; • Assisting public agencies in evaluating the impacts of pilot projects (e.g., projects directly implemented by a public transit agency, through public-private partnerships, or private- sector initiatives); and • Identifying necessary changes to current programs and services. By design, the framework is applicable across a variety of transportation projects and provides a method for evaluating project impacts. MOD and AV Planning, Implementation, and Evaluation Framework This framework is based on a three-phase process consisting of planning, pilot, and imple- mentation and evaluation phases. The process involves problem understanding, collaboration, and experimentation to yield potential strategies. Figure 24 illustrates the process. The framework was adapted from an initial framework used for assessing smart cities (i.e., communities using different technologies to collect data to inform decisions) that can be C H A P T E R 1 2 Pilot and Implementation Framework

78 Shared Automated Vehicle Toolkit: Policies and Planning Considerations for Implementation implemented by communities. For further information on the original framework please see A Framework for Integrating Transportation into Smart Cities (Shaheen et al. 2019). e framework builds upon design thinking and communities of practice (CoPs) method- ologies. Design thinking is a human-centered design paradigm focused on meeting the under users’ needs with existing technology and strategic business models (Brown 2008). CoPs are groups of people joined together by a common concern or passion who regularly engage with one another (Wenger and Snyder 2000). Phase 1: Understanding Problems and Community Needs is phase denes the problem(s) and identies the potential community needs that either may arise from MOD and AV projects or may be addressed through MOD and AV projects. Step 1: Initial Assessment e rst phase begins with an assessment based on interviews with stakeholders (e.g., commu- nity representatives, private-sector partners). ese interviews can help establish a baseline for SOURCE: Shaheen et al. 2019. Phase 1: Understanding Problems and Community Needs Step 1. Initial Assessment Step 2. Problem Statement Process Step 3. Design Thinking Application Step 4. Problem Statement Development Step 5. Refinement and Prioritization Phase 2: Creating Institutional Capabilities for Collaboration and Experimentation Step 1. Launch Public Agencies' Communities of Practice (CoPs) for Projects Phase 3: Implementingand EvaluatingPilots Step 1. Prepare for Pilots and Implementation Step 2. Map Pilots Based on Implementation Scales Step 3. Define Pilot Objectives Step 4. Identify Pilot Hypotheses Step 5. Develop Pilot Metrics Step 6. Identify Pilot Data Sources Step 7. Define Methods of Analysis Step 8. Pilot Evaluation Figure 24. Three-phase planning, implementation, and evaluation framework.

Pilot and Implementation Framework 79   problem identification and collaboration. Interviewers can gain a sense of what problems or needs are most urgent by identifying • Opportunities and challenges within a community, • Common characteristics and themes across agencies and stakeholders, and • Possible strategies for public agencies to develop and implement. Step 2: Problem Statement Process During this step, public agencies define challenges related to MOD and AVs in the form of problem statements. Problem statements briefly summarize a critical impact on a community. Problem Statements Problem statements are developed on the premises of 1. Increasing benefits by relating challenges to other communities (e.g., geo- graphically close communities, regions with similar development patterns); 2. Using strategies, such as design thinking, to break down complex problems and address challenges; and 3. Improving operational efficiencies by strategically using resources to address challenges. Step 3: Design Thinking Application Design thinking is a strategy for developing designs based on identifying and understanding user needs to address them more efficiently. Figure 25 summarizes the design thinking process. In the design thinking process, needs are identified through the empathy and definition phases and may be evaluated using simulation or predicted models. The results of these evaluation processes may be measured by metrics established in the ideation phase and by user feedback gathered in the testing phase. In this step, design thinking can be applied to • Determine the core problems (e.g., a set of common issues) related to impacts on the trans- portation system, • Develop problem statements, and • Create strategies to pitch problem statements to potential partners to foster strong partner- ships and secure resources. Step 4: Problem Statement Development During this phase, stakeholders can develop at least five problem statements regarding the specific impact of projects in their community or region. Problem statements can be developed based on stakeholder engagement that was conducted during Step 1: Initial Assessment. Step 5: Refinement and Prioritization Problem statements can then be refined so they can be communicated precisely to a third party and relate to other parties’ concerns. Problem statements are then prioritized based on stakeholder input.

80 Shared Automated Vehicle Toolkit: Policies and Planning Considerations for Implementation The expert interview process and the problem statement process inform the next phase of this framework. In Phase 2, the top five problem statements are employed to define the CoP and the topics for MOD and AV deep dives (these serve as learning exchanges for public agencies). Phase 2: Creating Institutional Capabilities for Collaboration and Experimentation Phase 2 begins the development of multistakeholder col- laboration to support strategic experimentation to implement pilots, policies, and/or programs. SOURCE: Gibbons 2016. Figure 25. The design thinking process.

Pilot and Implementation Framework 81   Launch Public Agencies’ Communities of Practice for Projects Research conducted by Shaheen et al. (2019) on the U.S. DOT Smart City Challenge1 and the Smart City Initiative2 identified the need for P2P information sharing networks to support collaboration and experimentation. This led to the promotion of CoPs, or agency-sponsored cohorts that promote knowledge sharing and collaboration among experts with shared goals (e.g., public agencies). CoPs are developed by engaging a variety of stakeholders by sharing proj- ect priorities that stakeholders also value (e.g., using SAVs to increase mobility for older adults). The purpose of developing CoPs is to provide a collaborative workspace outside of conferences or meetings without pressure from vendors. CoPs typically meet once a month to discuss and share information including building stake- holder support, strategies and tactics for addressing problem statements, best practices, emerging issues, technological developments, and updates on various local initiatives. CoPs may require legislation that supports and enables them through means, such as requiring publicly available information and findings from CoPs, standardizing CoP operations, and facilitating CoPs. One example of a current CoP is between the Smart Cities Lab (a P2P network focused on integrat- ing mobility investments from different cities into a single platform) and the Texas Innovation Alliance (a network of local, regional, and state agencies). Through this CoP, the Texas Inno- vation Alliance has been able to share lessons learned and best practices for equity and acces- sibility, integrated mobility, energy and sustainability, and real-time data. Phase 3: Implementing and Evaluating Pilots This phase involves preparing for MOD and AV pilot implementation and evaluation. Step 1: Prepare for Pilots and Implementation In this step, public agencies should select a pilot project based on their pri- ority problem statements, available institutional resources, and internal expertise. Pilot testing is a tool that can help communities experiment with MOD and AV projects. Pilot projects can help the public sector • Prepare and implement MOD and AV services; • Validate technical and institutional feasibility of new MOD and AV services; • Measure the impacts of specially targeted projects; • Allow for the testing and implementing of large-scale change; • Create models other public agencies can borrow and customize; • Serve as venues for evaluating public policies and regulations that could either support or hinder the operation of MOD and AV services; • Try new MOD and AV services on a limited scale (in terms of time and/or distribution); and • Monitor the success, adapt if necessary, reduce the risk of failures, and maximize the poten- tial for long-term success in a post-pilot project. 1 The Smart City Challenge was a U.S.DOT-sponsored challenge focused on using data, apps, and technology to more quickly, cheaply, and efficiently move goods and people. 2 The Smart City Initiative was a federal investment for research that used technology collaborations to address community challenges (e.g., congestion, growth management).

82 Shared Automated Vehicle Toolkit: Policies and Planning Considerations for Implementation Step 2: Map Pilots Based on Implementation Scales As part of the MOD and AV project planning process, public agencies and private-sector stakeholders need to determine the type of pilot scale they are targeting. MOD and AV pilot projects can be organized by their scale of focus: • Neighborhood: Local needs and contexts. • City: Intercity travel. • Regional: Entire regions and interregional travel, requiring larger-scale community outreach, surveys, and impact analysis. Table 19 discusses these scales in further detail. Focusing on an urban scale can aid in under- standing project dynamics and measuring impacts at the appropriate scale using a quantitative approach. As part of pilot planning and implementation, public agencies may need to evaluate MOD and AV pilot implementations to learn what was successful and what agencies can improve upon in future pilots. When evaluating an MOD or AV pilot project, establishing an evaluation framework can help define project objectives, identify project hypotheses, develop performance metrics, identify sources of data, and define methods of analysis. The following steps describe the process in greater detail. Step 3: Define Pilot Objectives In this step, public agencies establish the MOD or AV pilot project’s goals and objectives (e.g., reducing average travel times by 10 minutes, reducing GHG emissions by 30% within 2 years). If applicable, agencies should define the objectives with an associated direction (i.e., a change up or down) and within some user population or local system context. Some objectives may describe impacts that are measured qualitatively (e.g., safety, perceived accessibility for people with disabilities), rather than quantitatively (e.g., speed, trip time). Pilots typically require tracking outcomes for more than one goal. For example, an AV pilot could use a low-speed automated shuttle to take travelers to and from transit stations. Possible Scale Defining Characteristics Examples N ei gh bo rh oo d Neighborhood MOD and AV initiatives deploy smaller pilot projects specific to neighborhood needs and context. They can include • University campuses for intercampus transportation, • Office parks to connect to offices or nearby facilities (e.g., shopping centers), • Airports to offer connections to and from the terminal (e.g., from parking lots), • Apartment complexes for convenient transportation throughout the complex, and • Business improvement districts to increase access to, from, and through the district. Neighborhood pilots include the development of low-speed, automated shuttles and neighborhood electric vehicle charging infrastructure for electric carsharing. C ity City or jurisdiction-level MOD and AV initiatives are generally undertaken by local governments, typically to respond to broader community needs. City-level pilots include connected wireless technologies for automated vehicles and automated public transportation, citywide transportation equity initiatives, etc. R eg io na l Regional entities generally employ a macro-level approach to implementation (i.e., larger-scale pilots covering regional geography, larger samples, and a regional study population). Regional pilots include integrated fare payment and multimodal mobility platforms, regional pricing initiatives, and 511 information platforms that incorporate real-time transportation system data. Table 19. MOD and AV pilot scales.

Pilot and Implementation Framework 83   goals for this pilot could include reducing collisions around transit stations, improving the geo- graphic distribution of traveler origins and destinations, and reducing overall travel times. Figure 26 illustrates how Steps 3 through 7 of Phase 3 would treat these goals separately. Step 4: Identify Pilot Hypotheses e next step is to develop hypotheses that the evaluation can test (e.g., the average trip time of the population will decrease when using this low-speed automated shuttle). It can be helpful to reword the hypotheses as questions, especially in cases when agencies expect the outcomes to be less clear. is prevents the pilot from oversimplifying and categorizing results in totality as completely successful or altogether insucient. Step 5: Develop Pilot Metrics Aer the objectives and hypotheses have been dened, public agencies can dene perfor- mance metrics for what is being measured. ese metrics illustrate how project objectives have been met. Developing project metrics can also help identify desired data sources and gaps. Sometimes public agencies may need to revisit and redene metrics to account for gaps in data availability. A variety of stakeholders can use performance metrics: • Public agencies can use performance metrics to develop policies or programs to aid in their understanding of local corridors (or regional highway systems). • Municipal and regional service providers can use performance metrics to help implement MOD and AV initiatives. • Residents and non-prots can employ performance indicators to understand the impact of MOD and AV initiatives on quality of life. • Researchers can use performance indicators to quantify and measure the impact and eec- tiveness of MOD and AV initiatives over time. • Public agencies and jurisdictions can compare MOD and AV projects’ and programs’ eec- tiveness when comparable performance metrics are employed. "First and Last Mile" Connection Pilot Program Goal 1: Service's ability to reduce vehicle collisions Implement Steps 3–7 (Define Pilot Objectives, Identify Pilot Hypotheses, Develop Pilot Metrics, Identify Pilot Data Sources, Define Methods of Analysis) Implement Steps 3–7 (Define Pilot Objectives, Identify Pilot Hypotheses, Develop Pilot Metrics, Identify Pilot Data Sources, Define Methods of Analysis) Implement Steps 3–7 (Define Pilot Objectives, Identify Pilot Hypotheses, Develop Pilot Metrics, Identify Pilot Data Sources, Define Methods of Analysis) Goal 2: Expansion of service to include broader geographic distribution of traveler origins and destinations Goal 3: Service's ability to reduce travel times Figure 26. Example goal tracking.

84 Shared Automated Vehicle Toolkit: Policies and Planning Considerations for Implementation Due to the multidisciplinary nature of MOD and AV projects, this toolkit cannot identify all metrics across all MOD and AV impact areas required for performance monitoring. Perfor- mance metrics that are comparable with other MOD and AV pilots can help facilitate multi- disciplinary comparisons across MOD and AV projects. Table 20 lists example metrics that a shared, electric, connected, automated (SECA) vehicle pilot could use. Step 6: Identify Pilot Data Sources The next step can define data sources as needed to evaluate project performance metrics. In this step, partners can work together to develop data sharing and data use agreements early in the pilot or implementation process. If data sources are insufficient, it is important to gain an understanding of the extent to which the missing information forms a data gap. A data gap may be addressed by redefining metrics or searching for alternative data sources. Alternative data sources may not be ideal substitutes but can yield some insight into the relevant hypothesis. Metrics often require multiple types of data from various sources that may be unrelated. Data sources could include but are not limited to • Annual mileage on personal vehicles, • App usage data, • Census data, • Environmental Protection Agency vehicle fuel data, • EV charging data, • Surveys, • Transit agency data (e.g., revenue, costs, payment, ridership), • Traffic sensor data, Impact Category Metric Examples Safety Number of SECA vehicle collisions. Injury and fatality rate of SECA vehicles (per mile). Injury and fatality rate of non-SECA vehicles (per mile). Transportation Unlinked trips. VMT with SECA vehicles. Passenger miles traveled with SECA vehicles. Parking utilization by location. Travel mode share before and after project implementation. Average passenger wait time. Average trip time. Average trip distance. Equity Demographic profile of vehicle owners. Demographic profile of users. Number of users on the mobility platform. Spatial distribution of locations served. Demographic distribution of areas served. Average trip cost per mile. Environment Fuel efficiency of SECA vehicles. Emissions per mile of SECA vehicles. Emissions impact by changes in behavior as a result of the project. Energy Fuel efficiency of SECA vehicles. Energy consumption per mile of SECA vehicles (kilowatt hour). Energy impact from changes in behavior as a result of the project. Congestion Average speed between origins and destinations. Spatial density of vehicles. Hours per day SECA vehicles are moving on the road. Distribution of VMT per hour. Count of vehicles driving per hour. Number of SOV trips avoided by time and by origin/destination. Adapted from Shaheen, Cohen, and Bayen 2018; SFMTA 2016. Table 20. Shared, electric, connected, automated vehicle pilot example metrics.

Pilot and Implementation Framework 85   • User activity data, • Vehicle activity data, and • VMT estimates for regions, states, and the nation. Step 7: Define Methods of Analysis This step defines the methodologies that will guide data analysis. Advanced models or sta- tistical analyses are not always required to answer important questions. Specific metrics can be evaluated through descriptive statistics (e.g., averages, minimums, and maximums) or plotting data to find clear trends. Well-designed metrics and data designs can allow for a variety of dif- ferent techniques to address the same question. Step 8: Pilot Evaluation The last step is to implement the evaluation. Analytic findings collected after the evaluation can be incorporated as feedback in the next development cycle of the pilot. Data Standards, Privacy, and Management Data sharing is another requirement that public agencies may require as a condition for AVs to operate in the public rights-of-way. Standardized and open data (1) allow public agencies to understand AV and MOD impacts; (2) identify gaps in the transportation network; (3) monitor equitable service standards; and (4) offer multimodal, real-time transportation information through smartphone apps, websites, and other platforms. A few examples of industry data standards for MOD include: • General Bikeshare Feed Specification (GBFS) – GBFS makes real-time bikesharing operational data feeds publicly available in a standardized format. GBFS does not include historical usage data or other personally identifiable information. • Mobility Data Specification (MDS) – MDS is a data and application programming interface (API) standard that allows the city to gather, analyze, and compare real-time and historical data from shared mobility service providers. The specification also serves as a measurement tool that helps enable enforcement of local regulations. MDS also allows service providers and public agencies to communicate with each other about their services because it consists of two APIs: (1) a service provider API and (2) a public agency API. MDS includes data such as mobility trips (and routes); location and status of equipment (e.g., available, in-use, and out-of-service); and service provider coverage areas. These data standards could become models for the establishment of data standards specific to AVs (privately- owned and shared). With a growing number of data sources containing sensitive financial and location information, consumer privacy is becoming an increasing concern. The European Union’s General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) are consumer data privacy laws that took effect in May 2018 and January 2020, respectively. GDPR aims to give individuals control over their personal data by requiring companies to implement data management practices, such as anonymizing data, disclosing how data will be used and how long it will be retained, and the right for users to request a copy of their data. CCPA has similar consumer protections as well as prohibiting discrimination against consumers that exercise their privacy rights. In particular, CCPA includes a number of data types that could apply to MOD and AVs. CCPA applies to data categories such as personal and online identifiers, commercial information, biometric information, online activity information, geolocation data, and audio, visual, and photographic information, among others. These laws could raise questions around how MOD and AV data are collected and shared while still complying with consumer data privacy laws and regulations. (continued on next page)

86 Shared Automated Vehicle Toolkit: Policies and Planning Considerations for Implementation Data Standards, Privacy, and Management (Continued) Because AVs and MOD generate a lot of technical and user data, data management is also becoming increasingly important. A variety of research initiatives are exploring data management and AVs. The U.S. DOT’s Federal Automated Vehicle Policy series (i.e., AV 1.0, 2.0, 3.0, and 4.0) synthesize potential data uses and management strategies. Additionally, Workzone Data Exchanges (a U.S. DOT initiative to share work zone data on an accessible platform to increase awareness and safety) also work to gain a better understanding of the best uses of and practices for AV data. These initiatives can provide transportation stakeholders with a better idea of the opportunities, challenges, uses, and limitations of AV data. Some potential use cases for AV and MOD data include • Assisting in understanding travel patterns and behavior, use cases, and opportunities; • Informing plans and policies; and • Supporting MOD and AVs multimodal integration (e.g., MaaS platforms, integrated payment, etc.). However, MOD and AV data can also raise concerns including (1) consumer privacy; (2) trade secrets and proprietary information protection; (3) data safety and security; and (4) usability and storage. Existing transportation data standards and management strategies, such as general feed specifications and secure data sharing platforms, can support the development of future AV data management practices. Next Steps The implementation framework can assist agencies and other stakeholders to understand the potential impacts of select pilot projects and programs. These findings can then inform decisions made by stakeholders, such as whether to continue with a program and what pilot or program changes may need to be made. Depending on the findings of the assessment framework, stakeholders may take the following next steps. • Altering Programs: Based on the findings of the assessment, agencies may want to alter their program. Building on the example metrics in Table 20, if the evaluating organization finds that their service is providing significantly higher numbers of rides for people without disabilities, they may want to alter their service to include more people with disabilities. These alterations could include marketing locations, booking process, and vehicle design. • Disseminating Lessons Learned: The assessment framework may reveal critical lessons learned that can be distributed to other stakeholders to inform their pilot development pro- cesses or practices. • Ending Programs: The findings could also encourage stakeholders to end their program of service. Findings may reveal that the project was not reaching the intended goal or was result- ing in other challenges, such as high operational costs. • Expanding Programs: Assessment findings may also encourage stakeholders to expand their programs to offer new or more robust services, reach a broader demographic, or serve a larger geographic area. • Reevaluating Programs: The assessment findings may encourage stakeholders to reassess pilot projects and programs with a different or more tailored focus to gain a broader under- standing of the project.

Pilot and Implementation Framework 87   Key Takeaways • This framework for MOD and AV pilot planning, implementation, and evaluation can help public agencies – Identify and understand community problems, – Engage and collaborate with stakeholders, and – Experiment with pilot strategies. • The framework consists of three phases: – Phase 1 includes an initial assessment with stakeholders, application of design thinking to problems, and problem statement development to understand community concerns. – Phase 2 includes the development of CoPs to create institutional capabilities for collabo- ration and pilot experimentation. – Phase 3 focuses on MOD and AV pilot implementation and evaluation. • Implementing and evaluating MOD and AV pilots is essential for communities and public agencies to understand the impacts of these services and to replicate what works.