Guidebook for Measuring, Assessing, and Improving Performance of Demand-Response Transportation (2008)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

41 Once a DRT system has measured its performance—calculating the various measures selected to gauge how well its operations are performing—it then must assess that performance, analyz- ing and interpreting the results. Assessing DRT performance is the topic of this chapter, which includes the following: • A brief review of different methodologies that can be used to assess DRT performance. • The typology of DRT systems developed through the research project. This typology groups DRT systems into seven categories with similar characteristics that affect their performance. Developing a typology of DRT systems was one of the objectives of the research project, to facil- itate the comparison of a DRT system’s performance to that of similar systems. Such comparisons can be useful for assessing performance, and they are more useful when the comparisons are with systems that share attributes affecting performance. 5.1 Different Ways to Assess Performance There are several different ways that DRT performance can be assessed. Many DRT systems use more than one method to assess their performance, both to monitor and manage DRT oper- ations and to present performance results to their boards or other interested groups. Using a combination of methods may provide a more thorough assessment. Trend Analysis Also called a time series analysis, trend analysis is probably the most common assessment methodology. With this method, a DRT system compares its own performance on the same measures over time, typically on a monthly and annual basis, where the annual data are shown by month to account for the seasonal variability of DRT service. Such an analysis allows the system to monitor its performance and measure progress over time. The DRT system may need to reach certain performance standards, or monitoring may be done to watch trends and iden- tify areas for improvement. When using trend analysis, a DRT system should note any specific time points when signifi- cant changes are introduced or major events occur that impact performance. This will allow sub- sequent assessments to review performance trends in light of the major change or event. For example, if the DRT service area changes or if the system introduces new or different technol- ogy, it is important to document when that event occurs on the trend line in the performance monitoring reports (see Figure 5-1). This will inform the review of the resulting performance and provide a context for any deviations in resulting performance. C H A P T E R 5 Assessing Performance—Typology of DRT Systems

42 Guidebook for Measuring, Assessing, and Improving Performance of Demand-Response Transportation Comparison to Established Norms or Standards With this method, the DRT system compares its performance to an established standard or norm. While there are no hard and fast standards that must be met by all DRT systems, some norms have developed over time. Generally, for example, a norm of 90% on-time performance for trip pick-ups has evolved, though there is no industrywide requirement dictating such performance for DRT timeliness and despite the fact that DRT systems define “on-time” in varying ways. For ADA paratransit, however, regulations establish de facto standards for service in specific areas and these in turn affect performance. Based on the regulations and subsequent court inter- pretations, for example, it has been established that ADA paratransit systems must serve all expected demands for service.1 This has become a standard for DRT systems that provide ADA paratransit service. In addition, a DRT system may set its own standards for performance achievement. This is particularly true when service is provided by contract. Specific standards may be set in contract documents, establishing performance levels that the contractor is expected to meet. This can be beneficial in ensuring contractor attention to performance. Caution is needed, however, in setting those standards, as sometimes they may be unrealistic, setting up a difficult dynamic that may harm the contracting relationship. For contracted service as well as directly operated ser- vice, standards must be evaluated periodically, to ensure they are reasonable and continue to be reasonable in light of any changing conditions that influence performance. State and regional funding organizations may also set standards that must be met by DRT sys- tems for continued funding consideration. The state of California, for instance, has set specific stan- dards for the achievement of farebox recovery for systems that receive certain state transit funds. Comparison to Peers Comparison to peers is another approach to assessing DRT performance and a common per- formance assessment methodology. With this approach, the DRT system identifies a number of 0.00 1.00 2.00 3.00 4.00 5.00 Jan Feb Mar Apr May JulJune SepAug Oct DecNov Pa ss en ge r T rip s/ Re ve nu e Ho ur 2003 2004 2005 Expanded service area to include newly incorporated area. Figure 5-1. Example of DRT productivity trends. 1 While the term “zero denials” is commonly used in relation to meeting ADA paratransit demand, the regulations and subsequent court decisions require that ADA paratransit systems meet all expected demand for service, with recognition that there may be an insubstantial number of trips that cannot be met as long as such denials are not attributable to the design of the paratransit system.

Assessing Performance—Typology of DRT Systems 43 systems that share basic characteristics; researches the performance of those similar systems on selected performance measures; and compares its own performance to that of the peers. Despite widespread use, peer assessments typically include caveats, which state something to the effect that peer system comparisons should be “treated with caution,” as selected DRT systems may share similar characteristics but other attributes, beyond basic similarities, may cause performance differences. This means that direct comparisons are not exactly “apples to apples.” While it may be that peer assessments should be used judiciously, they provide useful infor- mation for a DRT system interested in knowing the performance of others on specific measures, and they show the range of performance achieved by the other DRT systems. Importantly, such comparisons can provide a context for assessing an individual system’s own performance. Although a trend analysis of the DRT system’s own performance over time may provide the “truest” assessment of performance, given that the operating environment and factors affecting performance generally do not change significantly over the short-term, peer comparisons pro- vide a framework for reviewing a DRT system’s own performance. And understanding the dif- ferent factors and characteristics that affect peer systems’ performance may help an individual DRT system consider strategies to improve its own performance. Choosing peers, however, may not always be straightforward, and reported performance may be based on data and data definitions inconsistent across the peer groups. Addressing these issues—which will provide for more accurate peer assessments—was one of the objectives of this research project. 5.2 Developing a Typology of DRT Systems—Which Systems Are Similar? Measuring a DRT system against similar systems can be useful for assessing performance. Given the considerable range in types of DRT systems, however, it may not always be clear as to which systems are appropriate peers. Typology of DRT Systems with Categories of “Similar” Systems One of the objectives of the research for this Guidebook was to develop a typology of DRT systems, so that DRT systems are categorized into groups within which systems share character- istics that influence performance. Using the earlier literature on DRT from the 1970s and 1980s as well as more recent reports and published information, various characteristics and criteria were reviewed, each having some degree of influence on the operations and resulting performance of DRT service, in the attempt to develop a typology of DRT systems. These criteria are identified in Table 5-1 and described briefly in the following sections. Criteria Influencing DRT Performance Ridership Market Served A key characteristic influencing DRT performance is the type of riders that are served (1, 5). In the early years of DRT, the major distinction was between systems that served the general public and those that served specific population segments of the community, often seniors and persons with disabilities. In terms of performance, DRT systems serving the general public can typically achieve higher productivities than systems serving specialized markets, for a number of reasons:

• The pool of potential riders from the general public is larger, creating a higher density of potential demand. • Dwell time at pick-up locations is shorter for general public riders. This includes both the established wait time, set by policy, and the time needed for rider boarding and alighting. Data from several DRT systems in the mid 1990s found that dwell times at pick-ups for ambulatory riders, which make up the large majority of general public riders, averaged 2 to 4 min, while that time for riders using wheelchairs, which may be a significant portion of the specialized rider market, was 4 to 6 min (6). • There tend to be fewer late cancellations and no-shows at systems serving the general public as general public riders are typically less likely to cancel trips on short notice or no-show trips because of health issues and inclement weather. In the years post ADA, a key performance distinction among DRT systems is between those that function as ADA paratransit and those that do not (2, 7, 8). Importantly, the ADA regulations establish requirements that systems must meet, a number of which, essentially, set general or spe- cific standards that affect performance (2). Among these include regulations that specify when and where service is to be provided, the fare structure parameters, that all trip purposes be served, that trip requests be taken for next-day service, and that capacity constraints are not allowed. DRT systems that are not ADA paratransit do not have to meet such requirements and have more flexibility in providing service. These non-ADA paratransit systems can use a variety of policies and practices to ensure service meets available funding levels or locally determined parameters. Such systems may, for example, set days and hours of service without regard to fixed- route service; they may prioritize trips; and they may schedule frequently requested trips to pre- determined times, that is, group trips to achieve higher productivities. This greater flexibility also means that such DRT systems have more latitude to make certain changes to improve perfor- mance. This is a significant difference between ADA paratransit systems and other DRT systems. Service Area or Operating Environment The service area influences DRT performance in several significant ways: number of people living in the service area, geographic size, and distribution of residential areas and trip destina- tion areas. A service area with a larger population will have a larger pool of potential riders. A service area that is large geographically will tend to have longer trip lengths, and a low-density dispersion of residential areas and trip destinations across a service area will also mean longer trip lengths and less opportunity to group trips. Trip length is a particularly important performance factor: DRT systems can serve fewer longer trips in a given amount of time compared to shorter trips, which impacts both produc- tivity and cost per passenger trip (2). Other aspects of the service area may also impact performance, such as geographic features influencing the ease or difficulty of travel throughout the community. In a related way, traffic congestion and the street network complexity in larger urban areas also impact performance, 44 Guidebook for Measuring, Assessing, and Improving Performance of Demand-Response Transportation Ridership market served Service area/ operating environment Type of routing and scheduling Type of operator: public vs. private contractor Dedicated vs. non-dedicated vehicles Advance request vs. immediate response Use of advanced technology Door-to-door vs. curb-to-curb Use of volunteers Provision of Medicaid transportation Table 5-1. Criteria influencing DRT performance.

Assessing Performance—Typology of DRT Systems 45 resulting in long travel times which limit DRT performance in the same way as long trip lengths. Larger urban areas may also have long trip lengths, depending on passengers’ travel patterns. For larger urban areas, then, the compounding effects of traffic congestion, complicated street net- work, and long trip lengths put significant constraints on DRT productivity. Type of Routing and Scheduling The type of routing/scheduling structure has an important influence on DRT operations and per- formance (1, 2, 5, 7, 9, 10). DRT systems with a very flexible or unconstrained routing/scheduling structure will not be as productive as those with a less-flexible, constrained structure. The distinc- tion is often described as “many-to-many,” with many different pick-ups going to many different drop-offs or destinations, versus “many-to-few,” “few-to-few,” or “few-to-one,” which groups rid- ers traveling to only a few drop-offs or just one drop-off. The former type of DRT will have many individualized trips, with less opportunity to group trips and thus fewer trips provided in a given amount of time, whereas the latter have more opportunity to group passengers since there are lim- ited destinations. The grouping of passenger trips will increase productivity and decrease cost per passenger trip. Type of Operator—Public Agency versus Contractor DRT systems can be differentiated by whether they are directly operated by a public agency ver- sus privately operated by contract (2, 8, 11). The major performance distinction between the two is generally considered to be cost. With differences in labor costs between public and private tran- sit entities and the fact that labor is the dominant single component of transit operating costs, it is generally accepted that DRT services that are contracted to private entities will result in some cost savings compared to services directly operated by a public entity. Recent research suggests that cost differences may not be statistically significant, however (12). According to this research, this may be due, in part, to use of financial penalties for contracted service, to the extent that con- tractors may be bidding price structures that cover expected losses due to the penalties, or they may be scheduling vehicles in such way as to avoid conditions that result in the penalties. Dedicated versus Non-Dedicated Vehicles The issue of dedicated versus non-dedicated vehicles is important from several perspectives when differentiating types of DRT service. Use of non-dedicated service may improve cost effi- ciency since the public entity sponsor purchases only that amount of service that is needed. Yet dedicated service provides more control to the sponsoring public agency, which may result in somewhat higher quality service. From a data collection perspective, data collection may be somewhat more difficult with non-dedicated services, particularly taxis, and this may impact efforts of the public entity sponsor in monitoring performance. Advanced Request versus Immediate Request Service From the perspective of a DRT system, there are several performance differences between immediate and advance request service. With immediate request service, a DRT system is able to change and insert trips on a real-time basis, providing the opportunity for higher productivity (13). Cancellations and no-shows are less frequent with immediate response DRT systems (14). With advance reservation service, however, a DRT system can focus on refining the service-day schedule and operators’ manifests, providing an opportunity for performance improvements. Late cancellations and no-shows, though, will negate some of this effort, creating “holes” in opera- tors’ schedules. To the extent that a DRT system can re-fill these “holes” with new or re-scheduled trips, some of the detrimental affects of late cancellations and no-shows can be mitigated. DRT research has found, based on a simulation study of advanced technology, that higher rates of late cancellations negatively impact productivity. A DRT system with a lower rate of cancellations will be able to achieve higher productivity than one with a higher rate of cancellations. Assessing the damage of cancellations, the study found “...the average decrease in vehicle productivity due to can-

cellations appears to be an increasing function of the cancellation rate, with an approximate 4 to 5% decrease in productivity for every 10% increase in the cancellation rate” (15). Use of Advanced Technology Considerable research has been conducted that analyzes the impact of advanced technology on DRT performance, with the general conclusion being that use of advanced technology pro- vides various performance improvements. In particular, the literature suggests that use of CASD systems can improve productivity (12, 16, 17, 18, 19) though reportedly the magnitude of improvement was generally not large. Improvements in such areas as the reservations function, dynamic dispatching, and providing improved information to riders have also been reported (19). In addition to CASD, an AVL system has been found to improve DRT performance, accord- ing to some published accounts and research, including simulation studies (15, 20, 21, 22). Improvements relate to higher on-time performance and productivity gains to the extent that the real-time information provided through AVL can be used to make scheduling adjustments. Door-To-Door versus Curb-To-Curb Service The distinction between DRT that operates as door-to-door and curb-to-curb is another criterion affecting DRT operations and performance. From an operational perspective, door-to-door service is usually considered to increase dwell time, measured as the time that the vehicle spends from the time it arrives at the pick-up or drop-off location to the time that it departs, given that the vehicle operator goes to the door of the passenger’s building at both the pick-up and drop-off to provide assistance to and from the vehicle. However, the paratransit industry lacks good, quantifiable data as to the extent of the effect, and there are some who maintain that door-to-door service may shorten dwell time (at least at the pick-up end) as the operator is helping the rider negotiate, and negotiate more quickly with the assistance, the distance from the pick-up building to the vehicle. A recent study conducted for a large metropolitan ADA paratransit system attempted to analyze the differences in dwell time between ADA systems that are door-to-door versus curb- to-curb (23). Interestingly, based on information obtained from nine large ADA paratransit systems around the country, the study reported that the systems did not collect detailed data on dwell time and none considered dwell time a factor that needed close attention as it related to productivity or efficiency. According to the study, the systems reported similar dwell times regardless of whether they were door-to-door or curb-to-curb (though the study notes that the dwell times were often anecdotally estimated). Use of Volunteers The use of volunteers for DRT is another practice that affects DRT performance. DRT systems, particularly in rural areas and smaller communities, may use volunteers as drivers in conjunction with paid drivers as well as volunteers to serve in other capacities, such as administrative assis- tance (1, 24). Use of volunteers will provide a different cost structure than a system that uses only paid staff. For example, if operating statistics for the volunteer component are included in the system’s total operating data and costs, performance on measures that use operating costs may look “better” given that there will be no labor costs for the volunteers. TCRP Report 91: Economic Benefits of Coordinating Human Service Transportation and Tran- sit Services (25), quantifies specific cost savings for several specialized transportation systems that use volunteers. Provision of Medicaid Non-Emergency Transportation The provision of Medicaid non-emergency transportation is yet another factor that can affect the operations and performance of DRT systems, and its impact on DRT performance is mixed 46 Guidebook for Measuring, Assessing, and Improving Performance of Demand-Response Transportation

Assessing Performance—Typology of DRT Systems 47 (26). Providing Medicaid transportation may have the potential to enhance performance, as the addition of the Medicaid clients adds to the pool of riders, thus allowing the system to schedule more trips per unit of service supplied. Yet, Medicaid trips may also hinder performance. Where such trips are long, to distant medical facilities, performance will be negatively impacted. Other Medicaid-related factors can negatively impact performance, including the level of record- keeping required for Medicaid transportation providers and the eligibility verification process which can be difficult and time-consuming. Additionally, state Medicaid agencies typically require specific software programs to interface with central state records and these programs are expensive to purchase. Significantly, Medicaid’s impacts on DRT service are affected by location, since each state takes its own approach to overseeing the program, with varying requirements on the transportation providers. Typology of DRT Systems Developed for Guidebook The typology of DRT systems developed through the research project builds on earlier attempts to categorize DRT services. It also recognizes that a useful typology will use fewer rather than more categories and that the categories have clear demarcations that can be translated into discrete groupings. Early attempts to classify DRT services began in the 1970s with the early development of DRT (1, 3, 5, 9), with the common criteria used to classify DRT services including: • Ridership market served: general public versus a target market such as seniors or non- ambulatory riders; • Service area or operating environment: urban/higher density versus non-urban/lower density; and • Type of routing and scheduling structure: very flexible or unconstrained (“many-to-many”) versus less flexible or constrained (“few-to-few,” “few-to-one”) on a semi-scheduled basis. More recent attempts to classify DRT include these same criteria among the primary criteria for distinguishing types of DRT service, though in some cases the terminology has changed (2, 7, 8). Significantly, the criterion of ridership market includes the distinction of DRT that provides ADA paratransit service, recognizing the impact that the ADA regulations have on operations and performance. The typology adopted for the Guidebook uses two of these criteria: ridership market and ser- vice area. The third criterion—type of routing and scheduling—has an important effect on performance, but this is difficult to capture with discrete categories since DRT systems are not solely many-to-many or many-to-one or few-to-one and so on. Rather, systems typically have a mix of trips, using various routing/scheduling parameters. Based on the two primary criteria of ridership market and service area, defined by population size of service area, the typology developed for this project for DRT systems is shown in Table 5-2. Small Urban DRT Systems 50,000 – up to 200,000 Population Large Urban DRT Systems 200,000 - 1 Million Population Largest Urban DRT Systems Over 1 Million Population ADA Paratransit Only Limited Eligibility DRT General Public DRT ADA Paratransit Only Other DRT ADA Paratransit Only Other DRT Table 5-2. Typology of DRT systems.

While clearly there are a number of characteristics that influence DRT operations and resulting performance, it was determined that a useful typology would have fewer rather than many cate- gories and that the categories would be relevant in practice, practical in application, and straight- forward to apply. Small Urban DRT Systems Within the “Small Urban” DRT category, there are three types of DRT systems. “ADA Para- transit Only” refers to those systems that are designed and operated as ADA complementary paratransit systems. It would include systems that follow the ADA regulations strictly, as well as those that go somewhat beyond the strict, mandated ADA paratransit requirements, offering, for example, service to ADA-eligible riders beyond the 3⁄4-mi corridors around fixed routes and systems that provide premium services to ADA-eligible riders, such as will-call trips and same- day taxi service. DRT systems in this category, however, are those that are defined predominately by the ADA regulations. “Limited Eligibility DRT” includes DRT systems that serve specialized groups, typically sen- iors and riders with disabilities, sometimes defined as ADA-eligible. Also included in this cate- gory would be systems that serve senior, disabled, and low-income riders, as well as other locally defined specialized rider markets. The third DRT system type within the small urban DRT category is “General Public DRT,” and this is the least common type of urban DRT. However, there are some smaller cities and com- munities that provide general public DRT as the only form of local transit. And some communi- ties that have provided general public DRT service since the 1970s decided to continue such ser- vice, even after introducing fixed-route service. In a number of these communities, the DRT service is re-configured so that it provides DRT service to only ADA-eligible riders within the 3⁄4-mi cor- ridors of the fixed-routes and general public DRT service beyond the ADA-defined corridors. Large Urban DRT Systems The “Large Urban” DRT category includes demand-response systems operating in urban areas with populations of more than 200,000 and up to one million. Within this group are two cate- gories of DRT systems: “ADA Paratransit Only” and “Other DRT.” The “ADA Paratransit Only” systems are defined in the same way as in the small urban DRT group, and “Other” includes all other DRT systems, such as those serving senior and disabled riders as well as those serving spe- cialized markets and the general public. Since there are very few DRT systems serving general public riders in areas of this population size, it was determined that the typology would not include a separate category for general public DRT systems. Largest Urban DRT Systems The “Largest Urban” DRT group includes those DRT systems operating in major cities with more than one million population, and includes the same two categories of DRT systems as in the Large Urban DRT group: “ADA Paratransit Only” and “Other.” The Typology in Practice Using the typology of DRT systems, the next chapter of the Guidebook provides performance data for representative DRT systems within each of the seven categories for the five key per- formance measures selected for the project. These data can serve as benchmarks for other DRT systems interested in comparing their own performance to that of similar systems. 48 Guidebook for Measuring, Assessing, and Improving Performance of Demand-Response Transportation