Guidebook for Measuring, Assessing, and Improving Performance of Demand-Response Transportation (2008)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Using the performance data identified in Chapter 3, this chapter identifies the key performance measures selected for the Guidebook. While there are many different measures that can be used to evaluate DRT, it does not take a long list to capture the important aspects of performance. One of the objectives of the research project was to select a limited number of measures to assess DRT’s efficiency, effectiveness, and quality. 4.1 Performance Measures for DRT—The Many The literature review conducted as part of the research project underlying this Guidebook found more then 60 different measures that have been used to assess DRT service. The more common of these are shown in Table 4-1. For an individual DRT system, selection of the performance measures to be used for moni- toring and assessing service will depend upon the system’s specific goals and objectives, and it may also depend on the type of DRT service that is provided as well as other local factors. DRT systems that provide ADA paratransit service, for example, must collect and monitor a greater range of performance data than would typically be used by a general public DRT system. DRT systems may also need to assess their performance using measures selected by their state departments of transportation. A number of states provide state funding to their transit systems, including DRT, and require that the systems report specified performance data as a condition of the financial support. DRT systems may have other organizations to which they must report performance, such as other funding agencies and, for those operating coordinated services, spec- ified performance data may be required for the coordinated human service agencies. 4.2 Key Performance Measures for DRT—The Few From the many DRT performance measures found through the literature review, a limited number were identified as key performances measures. Using a smaller set of measures to assess transit performance may be preferable to using a long list of indicators (1, 2). This approach allows a DRT system to concentrate on essential elements of service performance. Depending on the results of those limited measures, the DRT system may need to delve deeper into its opera- tions, examining additional and more detailed data to identify reasons behind the performance results. The key performance measures for DRT selected for the Guidebook are shown in Table 4-2 and discussed below. 30 C H A P T E R 4 Performance Measures

Passenger Trips per Revenue Hour Passenger trips per revenue hour measures the productivity of a DRT system. Many consider this the most important single measure of DRT performance, assessing the system’s effectiveness. As a performance measure, productivity captures the ability of the DRT system to schedule and serve passenger trips with similar origins, destinations, and time parameters, using the least number of in-service vehicles and revenue hours. This is the essence of shared-ride, public DRT service. Performance Measures 31 Operating Cost per Passenger Trip Operating Cost per Vehicle Hour Operating Cost per Vehicle Mile Operating Cost per Passenger Mile Passenger Revenue per Total Operating Cost (Farebox Recovery Ratio) Passenger Trips per Vehicle Hour Passenger Trips per Vehicle Mile Accidents per 100,000 Vehicle Miles No-Shows per Scheduled Trips On-Time Pick-Ups to Total Pick-Ups (On-Time Performance) Complaints per 1,000 Passenger Trips Average Trip Length Average Vehicle Travel Time System Speed Response Time (as measured by the minimum time between when service is requested and when provided) Trip Denials per Trips Requested Passenger Trips per Revenue Hour Operating Cost per Revenue Hour Operating Cost per Passenger Trip Safety Incidents per 100,000 Vehicle Miles On-Time Performance productivity = total passenger trips ÷ total revenue hours Table 4-1. Common performance measures used for DRT. Table 4-2. Key DRT performance measures selected for guidebook. But there are various factors that affect the ability of a DRT system to be productive, includ- ing, importantly, the size of the service area, distribution of residential areas and destination areas, and the patterns of riders’ trips. If the service area is large and passengers request trips to distant and dispersed destinations, it will be harder to effectively schedule two or more riders on the same vehicle, and this will mean a lower productivity. If there are limited group trips, that is, opportunities to schedule riders on the same vehicle at the same time for travel to a common destination, this will also mean a lower productivity. Other factors that impact productivity include the level of no-shows and late cancellations; scheduling efficiency; dispatcher skills, and ability to schedule trips in real-time; vehicle opera- tor experience and familiarly with the service area and passengers’ trip-making patterns; and the operating environment including traffic and the roadway network.

32 Guidebook for Measuring, Assessing, and Improving Performance of Demand-Response Transportation The type of DRT service—particularly whether it functions as ADA paratransit or not—also affects productivity, because ADA regulations have effective limits on the flexibility that a DRT system has to maximize shared riding. This also may mean a lower productivity. Some of these are examples of uncontrollable factors affecting the performance of DRT systems as measured by passenger trips per revenue hour. Such factors impacting DRT performance are discussed in more detail in Chapter 5. From a DRT performance perspective, the emphasis on productivity stems in great part from the fact that small changes in productivity can be very cost effective. Larger changes can be even more cost effective. Table 4-3 provides a hypothetical example of a fictitious medium-sized DRT system, operating 75,000 annual revenue hours. If this system could increase its current produc- tivity of 1.8 passenger trips per revenue hour to 2.0, it could accomplish two improvements: (1) it would increase its ridership by 15,000 annual passenger trips within the same operating cost and (2) decrease the operating cost per passenger trip by 10%, from $25.00 to $22.50 per passenger trip (Scenario “A”). On the other hand, if the system has to serve the additional 15,000 passenger trips at the original productivity level of 1.8, it would need to increase revenue hours, with a correspon- ding operating cost increase of over 11% (Scenario “B”). And finally, if a productivity increase from 1.8 to 2.0 enabled the system to decrease its revenue hours, while serving the original 135,000 pas- senger trips, it could hypothetically reduce its annual operating cost by 10% (Scenario “C”). While the productivity measure has been calculated with either revenue hours or vehicle hours, revenue hours have generally been used for demand-response transportation. Revenue hours are also typically used for measuring fixed-route productivity as well. Productivity is sometimes measured as passenger trips per mile. Given the low passenger vol- umes on DRT relative to mileage, this ratio usually results in a number less than one. Such result- ing numbers are not particularly logical given that an actual passenger trip is not less than one; passenger trips per hour is an easier number to visualize. Performance Considerations Poor performance on passenger trips per revenue hour may result from a number of causes: • Scheduling practices that do not effectively group similar passenger trips, • Limited dispatch control that is not able to effectively manage service operations and respond to changes on a real-time basis, • Scheduled revenue hours are not aligned with ridership demand, • High rates of no-shows and late cancels, • Low ridership levels, • Low density of passengers within the service area, • Lengthy passenger trips, and • Vehicle operator inexperience. Revenu e Hour s Annual Scenario Passenge r Trip s Productivity Operating Costs for Revenue Hour s Operating Cost per Revenue Hour Operating Cost per Passenger Trip 75,000 135,000 1.8 $3,375,000 $45.00 $25.00 “A” 75,000 150,000 2.0 $3,375,000 $45.00 $22.50 “B” 75,000 135,000 1.8 $3,375,000 $45.00 $25.00 83,333 150,000 1.8 $3,750,000 $45.00 $25.00 “C” 75,000 135,000 1.8 $3,375,000 $45.00 $25.00 67,500 135,000 2.0 $3,037,500 $45.00 $22.50 Table 4-3. Effects of productivity on hypothetical DRT system.

Operating Cost per Revenue Hour Operating cost per hour is generally considered the key cost-efficiency measure, assessing the financial resources needed to produce a unit of service, defined as an hour of revenue service. In other words, what does it cost the DRT system to put service on the street? This measure, how- ever, does not evaluate use of the DRT service, and, as such, should be assessed in conjunction with the performance measures that evaluate ridership utilization. Performance Measures 33 operating cost per revenue hour = total operating cost ÷ total revenue hours Similar to the productivity measure, practices vary as to whether the measure uses revenue hours or vehicle hours in the denominator. Revenue hours are preferred for several reasons including the following: • It is consistent with the denominator normally used for the productivity measure—passenger trips per revenue hour. • It facilitates incorporation of taxi-based and other non-dedicated provider data. When DRT service is provided by taxis or other non-dedicated vehicles, it is usually trip specific—from a particular origin to a particular destination—and the concept of deadhead mileage and time is not relevant. Non-dedicated providers should be required to provide to the DRT system their mileage and time data for each trip that they provide, regardless of how they are paid, so that revenue miles and time can be computed, and included with data for dedicated service. • Governments, as sponsors and funders of public transit service, are primarily interested in buying revenue hours of service. It is the transit operators’ responsibility to provide them efficiently by minimizing deadhead. Operating cost per mile is another service efficiency measure often used for performance assess- ments, either in addition to or instead of operating cost per hour. However, cost per hour is often the more important measure because the largest proportion of costs (wages and salaries) is paid on an hourly basis (3). Performance Considerations The elements in this measure are the DRT operating costs, with the major components of costs related to staff labor and vehicle operations and their maintenance, and the amount of revenue service, as measured by hours. This latter data element is determined by the established service span; demand for service; allocation of revenue hours as determined by vehicle availability; vehi- cle operator assignments; and scheduling practices. There are various reasons that a DRT system’s performance on operating cost per revenue hour may not meet objectives, including: • Costs for labor, particularly vehicle operators; • Costs for maintenance, from an older fleet, problem vehicles, accidents, costs for fuel; • High proportion of paid vehicle operator hours to revenue vehicle hours; • High costs for administration; and • Inefficient number of revenue vehicle hours, resulting from a poor service design or from scheduling practices. Vehicle operator labor is a major cost center. For the transit industry in general, labor includ- ing fringe benefits may account for up to 70 to 80% of total operating costs, with the majority of employees working in vehicle operations. The labor rates paid to vehicle operators are somewhat controllable, but will depend on the local job market and wages paid for similar positions at com- peting organizations. For some DRT systems, the rates may be influenced by a labor contract.

34 Guidebook for Measuring, Assessing, and Improving Performance of Demand-Response Transportation Maintenance is another important cost center. Based on NTD data for the transit industry in general, vehicle maintenance may account for up to 20% of operating expenses. DRT management has some control over this factor, but costs will also depend on the type of vehicles, their age, and the vehicles’ operating conditions, the latter of which is influenced by service area characteristics and weather. Operating Cost per Passenger Trip Operating cost per passenger trip is a critical cost-effectiveness measure. It combines elements of the first two measures—operating cost per revenue hour and passenger trips per revenue hour, relating productivity to the hourly operating cost. As a composite measure, a DRT system may have low operating costs but if productivity is also low, the operating cost per passenger trip may be high. Conversely, a DRT system may have a relatively high cost on a revenue hourly basis, but if its productivity is high, the cost per passenger trip may be low. operating cost per passenger trip = total operating cost ÷ total passenger trips Operating cost per passenger trip is a measure that decision makers typically look to: what does it cost to provide a trip for one passenger? It is important because it examines a DRT system’s abil- ity to carry out a core function—that is, transport passengers in a cost-effective manner. Performance Considerations Reasons that a DRT system might show poor performance on this measure include the following: • High operating costs – Costs for labor, particularly vehicle operators – Costs for maintenance from an older fleet, problem vehicles, accidents, costs for fuel – High administrative costs. • Low productivity – Scheduling practices that do not effectively group similar passenger trips – Limited dispatch control that is not able to effectively manage service operations and respond to changes on a real-time basis – Scheduled revenue hours that are not aligned with ridership demand – Low ridership levels – High rates of no-shows and late cancels – Low density of passengers within the service area – Lengthy passenger trips. Improving performance requires a reduction in operating costs and/or an increase in the number of passenger trips that are served—productivity—without expansion of resources and therefore costs. Safety Incidents per 100,000 Vehicle Miles Safety is a primary concern for all transit systems, including DRT. While the safety perfor- mance measure may not get the same attention as operating cost per revenue hour or operating cost per passenger trip in a DRT system’s monthly report, when the measure does get attention, it is probably not because of good performance. Safety performance should be included with the set of key performance measures, given its critical role within a DRT system. As a performance measure, the safety incident rate can be seen as one that incorporates an assessment of both service operations as well as passenger service quality. The safety of the DRT

system may not be an attribute that passengers consider each day as they ride the DRT system, but safety is a dimension of customer service quality (4). Calculation Given the different ways the DRT systems define and measure safety and their accident rates, it was determined that the Guidebook use the NTD definitions to assess safety. As described in Chapter 3, these are very specific definitions. The performance measure is the sum of NTD major and non-major safety incidents divided by total vehicle miles times 100,000. The measure compares the raw number of safety incidents to the miles traveled by the system, which places the raw number into the perspective of miles traveled by the system. Performance Measures 35 safety incidents per 100,000 vehicle miles = [(NTD major + non-major safety incidents) ÷ (total vehicle miles) x 100,000] on-time performance = (total on-time trips, including no-shows, + early trips) ÷ (total completed trips + no-shows + missed-trips) In addition to reporting accidents through NTD, DRT systems monitor safety incidents through their own internal procedures and practices, typically assessing preventable versus unpreventable accidents and counting all incidents and accidents, without regard to a pre- determined dollar threshold. The thresholds for reporting NTD-defined safety incidents related to property damage are relatively high, particularly for smaller systems. Performance Considerations A DRT system’s performance on safety can be improved by ensuring that vehicle operators are well trained, vehicles are well maintained, and operating policies and procedures support safe operations day-to-day. Poor performance on safety may result from a variety of reasons: • Limited vehicle operator training and/or retraining; • Inexperienced vehicle operators; • Vehicle issues such as the vehicle type or design and their condition; • Scheduling practices that result in a system speed that forces vehicle operators to rush, which then increases opportunities for accidents; • Environmental factors such as bad weather; and • The system’s commitment to safety. On-Time Performance On-time performance is perhaps the most important measure of service quality from a DRT rider’s perspective. On-time performance measures the reliability of the system: does the vehi- cle arrive for the pick-up when it was promised? Timeliness is often important at the drop-off end as well, though on-time performance at the drop-off end is not routinely measured and reported by DRT systems. DRT systems should assess on-time performance at the drop-off end for time-sensitive trips, those with pre-determined

36 Guidebook for Measuring, Assessing, and Improving Performance of Demand-Response Transportation “appointment” times. This would be a separate assessment, since only those trips with an appoint- ment time would be included for this assessment. As discussed in the last chapter, the definition of “on-time trips” varies among DRT systems. Data collection also varies, with many systems using vehicle operator-reported data and some using MDTs. Also as noted in Chapter 3, the measurement of on-time performance at the des- tination end has been an issue in some cases at DRT systems that are ADA paratransit, given that the ADA regulations do not specifically address destination arrival times. Calculation On-time performance should be calculated based on all completed trips and also on no-shows and missed trips, using definitions discussed in Chapter 3. It is important that procedures be developed to ensure that no-shows are in fact legitimate no-shows. This would include procedures that require vehicle operators to wait for the prescribed waiting period before they claim a rider is a no-show and to check with dispatch before leaving for their next stop. The Guidebook also suggests that early trips be included with on-time trips. However, the count of early trips should also be maintained separately so that early trips can be monitored. As discussed in Chapter 3, many riders are happy if their vehicle comes somewhat early, but riders should not be pressured into early departures if they are not ready to go until the window begins. In terms of calculating on-time performance, some DRT systems base measurement on a sam- ple of trips. This may be the random choice of a pre-determined number of days per month. If such a process is used, it is important that the days are representative of the service. For exam- ple, if service is provided Saturdays and Sundays as well as weekdays, the sample should consider weekend days as well. A number of systems have designed statistically valid sampling techniques. One of the larger DRT systems included in the research uses a statistically based sampling process to measure trip timeliness on a real-time basis. Each day, before the start of service, a random sample of trips is selected for on-time measurement and entered into a computer program written by transit agency staff. At the scheduled time of each selected trip, the dispatcher is prompted by the com- puter program to check the trip by contacting the vehicle operator by radio at the scheduled time of arrival to ask the vehicle operator of his specific location and then checking the AVL at the same time to confirm the operator’s location. The dispatcher then records the resulting data for on-time performance monitoring. The month-end performance statistic that is reported is con- sidered accurate at the 95% confidence level with +/-5% accuracy. The calculation of on-time performance may also be done using all trips. This is facilitated when the DRT system has MDTs and trip-by-trip data collection is automated. Where AVL technology is also available, this can serve to check the vehicle operator-reported MDT data when this is needed. Performance Considerations Poor on-time performance may result from various factors, including the following: • Vehicle operator schedules that are not adequately prepared or that overbook trips, so vehi- cle operators cannot maintain the schedule; • Incorrect information on schedules so that vehicle operators do not have the proper infor- mation for timely service (bad addresses, lack of details on just where to pick up the passen- ger such as back door, side street, etc.); • Limited dispatch practices to make real-time changes to tackle service problems and help vehi- cle operators who are running late; • Staffing issues such as no back-up operators (such back-up vehicle operators are often referred to as the “extra board”), inexperience, or an inadequate number of operators;

• Vehicle breakdowns or road calls, resulting from vehicle design issues or maintenance prac- tices that do not keep vehicles in good working order; and • Passengers’ habits (e.g., excessive dwell time because passengers are not ready to board upon vehicle arrival, use of wrong mobility aide). 4.3 Additional Performance Measures for DRT There are a variety of other performance measures that DRT systems typically monitor. Some of these are interim measures, assessing a specific aspect of DRT service that impacts efficiency, effectiveness, or service quality. The more common of these other measures are identified below, using performance data discussed in the previous chapter. No-Show/Late Cancellation Rate While there is some variation in how this measure is calculated, typically the no-show and late cancellation rate measures the percent of scheduled trips that are not completed due to passenger no-shows and late cancellations. The Guidebook recommends that this be calculated as the sum of passenger no-shows and late cancelled trips (with late cancels defined differently by DRT systems) divided by the total number of scheduled trips. The denominator—number of scheduled trips—is the total of the trips that are placed onto vehicle schedules for service, as defined in Chapter 3. Performance Measures 37 no-show/late cancellation rate = (total no-shows + total late cancellations) ÷ total number of scheduled trips It is noted that the no-show/late cancel rate can be considered an interim measure, monitored because of the important affect that no-shows and late cancellations have on productivity and operating costs. They are combined together for performance measurement purposes as they have a similarly negative impact on DRT operations: for most DRT systems, they represent lost resources with adverse impacts on productivity. In an effort to minimize the negative impacts of no-shows and late cancellations, most DRT systems have adopted policies addressing no-shows and late cancellations. There is considerable variation among these policies, but broadly, they all establish penalties for passengers who repeatedly cancel their trips with little notice or fail to appear for their scheduled trips. Imple- mentation and enforcement of such policies can significantly reduce the occurrences of no-shows and late cancellations; this is discussed in detail in Chapter 7. Cancellation Rate As discussed in Chapter 3, there are different types of cancellations. For thorough monitoring of cancellations, the Guidebook recommends that DRT systems assess the rate of advance can- cellations as well as the rate of same-day cancellations. To assess the degree of advance cancellations, the DRT system can compare all advance can- cellations received over the reservation period to the total number of trip reservations. While advance cancellations do not have the same detrimental affect as a same-day or late cancel, they do represent efforts of the reservations/scheduling staff which do not result in service. One strat- egy to address high rates of advance cancellations is to shorten the reservation window. This is discussed in detail in Chapter 7.

38 Guidebook for Measuring, Assessing, and Improving Performance of Demand-Response Transportation trip denial rate = total trip denials ÷ total number of requested trips advanced cancellation rate = total advance cancellations ÷ total number of reserved trips same day cancellation rate = total same-day cancellations ÷ total number of scheduled trips Missed-trips rate = total missed trips ÷ total number of scheduled trips Monitoring same-day cancellations is more important than advance cancellations from a ser- vice perspective, as these represent trips that are scheduled, but not taken. The same-day can- cellation rate compares total same-day cancels to total scheduled trips. DRT systems may be able to use at least some of the capacity created by same-day cancellations, but this will vary by the system’s scheduling and dispatch practices; how much time the DRT system has to respond to a same-day cancel; and policies related to will-calls and same-day trips. Missed-Trip Rate The missed-trip rate measures the percent of trips that are not completed because the DRT vehicle fails to arrive at the scheduled location, or the vehicle arrives late and the passenger declines to take the trip or is not even there anymore. The measure is computed as the number of missed trips divided by the total number of scheduled trips. As discussed in Chapter 3, it is useful to assess the degree of lateness of missed trips. A missed trip that is only minutes late is not the same as one that is an hour or more late. Performance Considerations Missed trips result from the same causes as poor on-time performance. Missed trips may also result from a lack of rider confidence in the DRT service. When the system is relatively reliable, riders are more likely to wait an extra amount of time for a vehicle that may be running late. However, if the DRT service is often late, riders may be less likely to wait much beyond the on-time window for their vehicle. In such cases, they may find alternative transportation or forego the trip. Trip Denial Rate The trip denial rate has become an important measure for those DRT systems that are ADA paratransit. The ADA’s prohibition of capacity constraints includes the requirement that ADA paratransit systems meet all expected demand for service, though it is recognized that there may be an insubstantial number of trips that cannot be met as long as such trip denials are not attrib- utable to the design of the paratransit system. As noted in Chapter 3, the definition of denials for ADA paratransit is somewhat complicated, and DRT systems that provide ADA paratransit service should ensure that they understand the definitional issues.

Complaint Rate In addition to monitoring and responding to complaints, some DRT systems measure and report their rate of complaints, by comparing the number of complaints received to service pro- vided, such as total service complaints per 1,000 passenger trips. The denominator may be total passenger trips completed or it may be total trips scheduled. Rather than passenger trips, some DRT systems compare complaints to revenue hours of service provided. Performance Measures 39 complaints per 1,000 passenger trips = (total valid complaints ÷ total passenger trips) x 1,000 or complaints per 1,000 revenue hours = (total valid complaints ÷ total revenue hours) x 1,000 The complaint rate can be monitored over time as an indicator of customer satisfaction. It is important that DRT systems maintain a consistently defined measure, so that trends and com- parisons month-to-month or year-to-year are meaningful over time. If the calculation method is modified, this should be clearly noted on any trend line comparison, to ensure proper assessment. Some systems have established a standard related to complaints, for example, the DRT system should have no more than x complaints per 1,000 passenger trips. Such a standard may be included in a contract document for a contracted DRT operator, with associated incentives and liquidated damages. Other DRT systems purposefully exclude a measurement of complaints from their standards, believing that complaints should be facilitated and encouraged as a method of monitoring service. One large DRT system purposefully did not include any measure of complaints when it structured performance standards for its contractors, and made the filing of complaints very easy for riders as a way to monitor contractor performance. Average Passenger Trip Length The size of the DRT system service area, distribution of riders’ origins and destinations, and degree of shared riding will affect the average passenger trip length. This can be a useful measure for a DRT system to monitor as it has an important affect on system productivity, with longer trip lengths having a negative affect on productivity. The average trip length should be monitored over time, as changes may be reflected in system productivity, and it may also be useful in comparisons to peer systems given its affect on productivity. average passenger trip length = total passenger miles ÷ total number of passenger trips Average Travel Time Average travel time is computed as the sum of all passengers’ travel times divided by the total number of passenger trips. average travel time = total passenger travel time ÷ total number of passenger trips This is not a measure that is routinely reported by DRT systems, but it is useful, indicating both the degree of shared riding and service quality for the passengers. One of the premises of

DRT is the grouping of passengers with similar trip patterns—ride sharing—to maximize pro- ductivity. If passengers’ travel times are short, comparable to travel by private vehicle, it indi- cates that the scheduling function has not achieved much ride-sharing. On the other hand, if many passengers’ travel times are long, it may indicate too much ride-sharing, and passengers may be overly inconvenienced with long on-board times to reach their destinations. Balancing ride-sharing with passenger travel times is a key objective of the scheduling function. While the average travel time for a DRT system provides a composite measure across the system, it may also be useful to analyze sampled individual trips on a regular basis. This may be particularly important for DRT systems that provide ADA service, given regulations on capac- ity constraints and specifically trips with excessive lengths. For example, a DRT system may want to review trips that are over an established length of time. For a small DRT system, this might be sampled trips over 45 min. For a large DRT system, this may be sampled trips over 60 or 80 min. 40 Guidebook for Measuring, Assessing, and Improving Performance of Demand-Response Transportation