Mobility on Demand and Automated Driving Systems: A Framework for Public-Sector Assessment (2022)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

86 REPORT CHAPTER 8: SAV PILOTS This section reviews AV technologies being proposed and piloted across the United States and internationally. Today, the majority of SAV pilots are targeting SAE Level 4 automation, in which a human operator does not need to control the vehicle if it is operating in a suitable ODD. The ODD in these use cases is defined by key roadway characteristics (i.e., roadway types and speed limits) and roadway conditions (e.g., weather conditions, time of day) an AV is designed to operate in (USDOT, 2017). For this reason, the ODD may be more important than the level of automation when discussing differences between SAV pilots. PILOTS Around the world, numerous pilots are pairing automation with shared mobility. Many of these pilots implement low-speed, automated shuttles that can be called on demand or operate on a fixed route in a controlled environment. A few commercial services offer on-demand mobility in a manner similar to TNCs (also known as ridehailing and ridesourcing). Since 2017, several SAV pilots have emerged across the United States. While companies have continued to launch SAV pilots, industry expectations for consumer facing SAV services without a safety driver have faced delays. Cruise, a private AV technology provider, planned to launch a commercial SAV service by the end of 2019; however, the company has delayed the launch and has not announced a new launch date. Tesla’s Elon Musk stated the company would have a million SAVs on the road by 2020. Musk has since acknowledged that this timing may be incorrect (LeBeau, 2019). Waymo was expected to roll out thousands of AVs to its fleet of around 600 vehicles, and this development is now expected to begin in 2020 (Wiggers, 2019). The assessment conducted in this research reviews the continuously operating SAV pilots as of December 2019 in the United States and classifies them according to whether the pilot is: 1. Serving passengers or testing only, 2. Operating on public or private roads, and 3. Using a low-speed shuttle or a conventional vehicle. Temporary demonstrations or pilots that have ceased operations and those that do not carry passengers (or are very close to carrying passengers) are not included in this assessment. The SAV pilots are classified using these characteristics because they measure how close each pilot may be to deployment and can inform the use cases that the pilot operators are targeting. Broadly, the SAV pilots currently underway can be classified into two types: pilots operating on private roads and pilots operating on public roads. In many cases, AVs on private roads are not subject to state and federal regulations. However, these classifications may evolve as services advance from testing to deploying services on public rights-of-way. The SAV pilots operating on private roads and in planned communities are typically low-speed (under 30 mph) deployments operating in controlled environments. These pilots often focus on serving specific markets, such as office parks, housing developments, retirement communities, and universities. The other main group of SAV pilots is operating on urban streets, typically using conventional vehicles equipped with AV technology to navigate their surroundings and traffic (Stocker and Shaheen, 2018).Table 21 and Table 22 list pilots underway and summarize important attributes, such as the vehicle type and ODD.

87 REPORT Table 21. Select SAV Conventional Pilots in the US Operators Location ODD Description Aptiv/Lyft Las Vegas, NV Public roads and city streets Through a commercial pilot project accessed through the Lyft app, 20 SAVs are servicing popular destinations on the Las Vegas strip (Korosec, 2019). AutoX San Jose, CA Public roads and city streets AutoX, an AV startup testing in California, launched a grocery delivery pilot program in San Jose in 2018 (Green, 2018). In June 2019, AutoX began accepting Early Rider applications for its SAV pilot service, xTaxi, and transported its first customer (AutoX Team, 2019). Cruise/GM San Francisco, CA Public roads and city streets In 2017, Cruise launched its pilot, “Cruise Anywhere,” an SAV service for its employees to use for pre- selected destinations in San Francisco, California. Cruise intended to launch a commercial SAV offering in 2019. The company has since announced it will not be deploying a commercial offering in 2019 and has not announced a new target date (Marshall, 2017; Felton, 2018). Mercedes- Benz/ Bosch San Jose, CA Public roads and city streets In December 2019, Mercedes-Benz and Bosch launched an SAV service in San Jose, California. The SAVs transport passengers in S-class vehicles on a fixed route between West San Jose and the downtown area. A select group of people has access to an app through which rides can be booked (Hawkins, 2019). nuTonomy and Lyft Boston, MA Public roads and city streets NuTonomy has been testing its vehicles in the Seaport neighborhood of Boston since 2017. In June 2018, the vehicles were approved for testing city-wide. In December 2017, nuTonomy partnered with Lyft to provide rides on their AVs through the Lyft app; however, NuTonomy no longer offers this service in Boston. Since 2018, the fleet has slightly expanded and is still operational as of 2019 (Locklear, 2018; Lyft, 2017). Via, Hyundai, and PonyAI Irvine, CA Public roads and city streets Via has recently signed a partnership with Hyundai and Pony.ai to form BotRide, a TNC service that is operating SAVs on public roads. As of November 2019, BotRide will operate a shared TNC service in Irvine, CA (Pony.ai, 2019). Voyage The Villages, San Jose, CA Private roads/ planned communities Voyage operates three SAVs at The Villages retirement community in San Jose, serving 15 miles of road. It has operated in San Jose since 2017 (Camerona, 2017; Voyage, 2019).

88 REPORT Operators Location ODD Description Waymo Phoenix, AZ Public roads and city streets Waymo launched the Early Rider program in early 2017, allowing select Phoenix, Arizona residents to request rides in their automated minivans. Waymo has since launched its commercial SAV service, Waymo One. As of December 2019, Waymo One serves over 1,500 riders every month in the metro Phoenix area. The company plans to pilot new features and capabilities in 2020 as well as expand ridership (Barr, 2018; Szymkowski, 2019).

89 Table 22. Select SAV Low-Speed Shuttle Pilots in the US Operators Location ODD Description EasyMile*/ Virginia Tech Transportation Institute Blacksburg, VA Public roads and city streets Researchers at the Virginia Tech Transportation Institute (VTTI) and North Carolina A&T State University will use an EasyMile EZ10 shuttle to evaluate whether AV shuttles can improve public transit access for vulnerable road users, such as older adults and people with disabilities. The shuttle will operate on a fixed route between a local transit stop and a stop on the VTTI campus. The pilot is scheduled to last one year, from May 2019 to May 2020 (Deekens, 2019). Local Motors – Olli, IBM National Harbor, MD, Germany Public roads and city streets With a 3-D printed, “crowd-funded” design, Olli has had their shuttles in the D.C. area streets since 2016. In October 2019, Maryland granted Local Motors a permit to expand its testing from a 1.5-mile track onto public roads on the outskirts of National Harbor (Graham, 2019; Local Motors, 2018; Warren, 2016). May Mobility Detroit, MI; Grand Rapids, MI; Providence, RI Public roads and city streets May Mobility currently operates three SAV pilots with its six-seater shuttles. In December 2019, the company raised $50 million in a round of financing with the largest investor being Toyota Motor Corporation (Bigelow, 2019). Optimus Ride Boston, MA; South Weymouth, MA; Brooklyn, NY Public roads and city streets Optimus Ride has been testing in Boston since 2017. Optimus Ride now provides SAV services to the Union Point development in South Weymouth and the Brooklyn Navy Yard in New York City (Etherington, 2017; Hawkins, 2019). *EasyMile still lists Babcock Ranch (FL) and Bishop Ranch (CA) as current pilot programs; however, employees at each community stated the pilots were no longer operational. In addition to the purpose of AV pilots (e.g., supporting public transit service, offering transportation options to different communities) ODDs are important components to AV operations. A few important ODDs with AV pilots include: • Closed-campus operations: Many pilots are serving passengers or members of a closed- campus community, such as a university, workplace, or retirement community. • Current public operations: Waymo launched a commercial SAV service in the Phoenix area in December 2018. Initially, the program was only available to former members of the Early Rider program. Now, members of the public can sign up for the Waymo One service. Once accepted off the waitlist, they can request rides through an app. In December 2019, Waymo expanded the service through a new iOS app (Ringle, 2019). • Emphasis on temperate climates: US SAV pilots have largely emphasized coastal and sunbelt (e.g., California, Arizona, and Florida) deployments. However, in the next few years, more AV pilots will likely test vehicles in less favorable weather conditions. A few companies are currently testing vehicles in winter conditions. May Mobility has been testing its shuttles in Detroit through a pilot program since mid-2018, and EasyMile

90 REPORT began a temporary winter pilot with its EZ10s in Minnesota in late 2017 that has since been completed (MnDOT, 201; Stocker and Shaheen, 2019). Other than Waymo’s AV testing efforts that began in 2012, most SAV pilots have been active since 2018 (about half have commenced piloting in 2018). Several SAV pilots in major US communities launched during late 2016 and early 2017 (e.g., Waymo in Phoenix and nuTonomy/Lyft in Boston). For the most part, these programs are making incremental improvements and slowly expanding the areas served during the pilot phase. At least three commercial SAV services have launched since 2018 including Lyft and Aptiv in May 2018, Waymo One in December 2018, and Pony.ai in November 2019. Given these developments, more SAV pilots may emerge soon. In the mid- to long-term (10 to 20 years), local SAV programs may gain a larger market share of US passenger-miles than their low-speed counterparts (Stocker and Shaheen, 2018). As shown in this analysis, large automakers and technology companies are at the early stages of developing SAVs for local or regional deployments. On the other hand, smaller players may continue to target niche markets and use cases allowing for quicker SAV deployment due to specially designed vehicles that do not need to function across a wide range of environments (Stocker and Shaheen, 2018), Potential Developments In October 2019, Waymo announced that it would begin removing safety drivers from behind the wheels of its Early Rider program. However, the vehicles in its commercial SAV service, Waymo One, still operate with test drivers (Hawkins, 2019). For many companies, it is not clear when they will begin removing test staff from their vehicles, and there is no standardized guidance for when safety engineers should be removed from a vehicle. In July 2019, Uber released its self-driving vehicle (SDV) safety case framework in a bid to provide transparency and inform voluntary standard-setting efforts. The SDV safety case framework provides a graphical representation of the company’s safety argument for AV testing and deployment (Uber ATG, 2019). Many companies testing AVs are also developing remote operations capabilities, allowing a remote human operator to take control of an AV in the event of a malfunction or emergency (Stocker and Shaheen, 2018). Appendix D: Data Types as suggestions to help think about data sources that can potentially support metric computation. Once the research team is given the parameters of the specific pilot project, they should revisit the data sources. Appendix D: Data Types contains tables that act as an extension of the impact assessment framework and provides examples of the questions, performance metrics, and data types that are most likely to be relevant to public transit agencies as they begin to engage with MOD and AV services. Define Analysis Methods The final step is defining the methods of analysis. This does not necessarily happen chronologically last as there may be preferred methodologies that guide other steps of the framework. At other times, questions can be answered in several different ways, and specific methodologies are not decided upon until later in the process (e.g., complex statistical

91 REPORT techniques, aggregating or plotting data to find general trends). Less complex techniques can be sufficient for analysis, depending on how comprehensive the research team determines is necessary. While it is important to consider analytical methods throughout the execution of the framework, it is ill-advised to lock in on a specific methodology too early. This can result in researchers overlooking simpler ways to address the same question or may create a reliance on data that was assumed available but was not ultimately accessible. Well-designed metrics and data designs can allow for a variety of different techniques to address the same question. Implementation Framework Table 17 provides an example of how to use the six steps of this framework to identify and develop a project. The example focuses on the launch of an SAV pilot to improve travelers’ access to transit. The example service uses fully AVs to offer an additional transportation option for travelers with disabilities. One goal for this sample project would be to explore the accessibility of the service and whether or not the service is providing an equal or more equitable alternative than non-AVs. For this goal, it is important to measure the trips made by passengers with disabilities as passengers travel to and from stops and stations. This is one example for a hypothetical SAV service. The real-world implementation of SAV services would require the consideration of a variety of goals and objectives, in addition to the accessibility example provided. For example, considering travel times and travel patterns would be important to an SAV pilot. These different goals would need to be addressed individually within the framework under each of the six steps, which Figure 19 illustrates. Table 17. Example Framework Application Step Example Explanation 1. Map Based on Implementation Scale Statement: Town-based scale (based on jurisdictional boundaries). Defining the implementation scale and how it is defined, in this case by jurisdictional boundaries, sets the stage for the remainder of the project. 2. Define Project Objectives Statement: Increase the number of trips made by passengers with disabilities going to and from X transit station For an accessibility-focused pilot, the number of trips taken by people with disabilities may be one of many objectives. 3. Define Project Hypotheses Statement: The number of trips taken per passenger using the new program going to and from X transit is more than trips taken on traditional modes This is the objective translated into a hypothesis. It is important to ask related questions including: “How did the trips taken by passengers with disabilities change?,” “Did the trip time for passengers with disabilities also decline?,” and “To what extent did trips by passengers with disabilities change?” 4. Define Project Metrics Rate: The number of trips by passengers with disabilities compared to trips by passengers without disabilities It is important to define exactly what is meant by “rate” and to make sure there are not any assumptions that may affect the interpretation of the results. For example, the number of trips by passengers with disabilities could mistakenly be defined as the total number of trips by passengers with disabilities per day. However,

92 REPORT Step Example Explanation this does not make a comparison with trips taken by other passengers and, as a result, does not reveal much. It is more appropriate to define the rate compared to trips by other travelers. 5. Define Project Data Sources Data: Number of trips, passengers with disabilities Sources: Traveler information, trip data Stakeholders evaluating pilots need trip numbers and traveler information for SAVs and standard vehicles providing similar services before the pilot and during the pilot. These types of data could come from a variety of sources including vehicle usage and user data. While travelers will likely disclose their disabilities to gain accommodations, it may be difficult to obtain an accurate number of passengers with disabilities particularly when passengers have less visible disabilities. This illustrates a data gap that may result in the need to refine project metrics. New metrics could include an evaluation of the impacts the pilot had on paratransit ridership. 6. Define Methods of Analysis Methodology: Hypothesis test for equality of two proportions We defined the hypothesis as a comparison between two populations: trips taken by passengers with disabilities and trips taken by passengers without disabilities. This lends itself to a test for equality of two proportions to determine whether there is a statistically significant difference between the two, and, if the number of trips taken by passengers with disabilities during the pilot is less than before. This is not a coincidence, as we defined the hypothesis with this statistical test in mind.

93 REPORT Figure 19. Applying Framework to Individual Program Goals WHAT ARE THE NEXT STEPS AFTER IMPLEMENTATION OF THE FRAMEWORK? Implementing the framework can assist agencies and other stakeholders in understanding the potential impacts of select pilot projects and programs. These findings can then inform decisions made by stakeholders, such as whether or not to continue with a program and what pilot or program changes may need to be made. Depending on the findings of the assessment framework, stakeholders may take the following next steps: • Altering programs: Based on the findings of the assessment, agencies may want to alter their program. For the example provided in Table 17, if the evaluating organization finds that their service is providing significantly higher numbers of rides for people without disabilities, they may want to alter their service to include more people with disabilities. These alterations could include marketing locations, booking process, and vehicle design. • Ending programs: The findings could also encourage stakeholders to end their program of service. Findings may reveal that the project was not reaching the intended goal or was resulting in other challenges, such as high operational costs. • Expanding programs: Assessment findings may also encourage stakeholders to expand their program to offer new or more robust services, reach a broader demographic, or serve a larger geographic area. • Revaluating programs: The assessment findings may encourage stakeholders to reassess pilot projects and programs with new project hypotheses and/or performance metrics to gain a broader understanding of the project. "First- and Last-Mile" Connection SAV Pilot Program Goal 1: Service increases accessibility Implement Six Key Steps Goal 2: Service increases spatial distribution of traveler origins and destinations Implement Six Key Steps Goal 3: Service improves (i.e., decreaes) travel times Implement Six Key Steps

94 REPORT KEY TAKEAWAYS • Agencies or other organizations can use the impact assessment framework as a tool to predict, identify, and measure the potential impacts of new service and project implementation (e.g., AVs). • The framework contains six steps: 1) map based on implementation scale, 2) define project objectives, 3) define project hypothesis, 4) define project metrics, 5) define project data types and sources, and 6) define analysis methods. • After implementing the framework, agencies or other organizations can continue to the next logical steps including altering, ending, expanding, and revaluating programs.