Guide to Effective Freeway Performance Measurement: Final Report and Guidebook (2007)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

iii TABLE OF CONTENTS LIST OF FIGURES ........................................................................................................................ iv LIST OF TABLES .............................................................................................................................v ACKNOWLEDGEMENTS ........................................................................................................ vi ABSTRACT.......................................................................................................................................vii SUMMARY...........................................................................................................................................1 CHAPTER 1 Introduction and Research Approach ................................................7 Background Scope of the Research Background Research Approach Background Relationship to Current Research Efforts Background Guidebook Development CHAPTER 2 Findings ............................................................................................................ 17 Background Benchmarking Interviews Background Basic Principles for Freeway Performance Measurement Recommended Freeway Performance Measures CHAPTER 3 Interpretation, Appraisal, and Applications ................................. 33 CHAPTER 4 Conclusions and Suggested Research ............................................... 36 Conclusions Suggested Research REFERENCES ................................................................................................................................42 APPENDIX A Results of Benchmarking Interviews ............................................. A-1

LIST OF FIGURES Figure 1. The Same Performance Measures Should Be Carried Across Applications Spanning the Entire Time Horizon .............................. 33 iv

LIST OF TABLES Table 1. Initial Benchmarking Interview Locations ................................................. 12 Table 2. Relationship of NCHRP 3-68 to Other Current Performance Measurement Projects .................................................................................. 12 Table 3. Basic Principles for Freeway Performance Monitoring........................ 22 Table 4. Recommended Core Freeway Performance Measures .......................... 23 Table 5. Supplemental Freeway Performance Measures........................................ 28 Table A.1 Supplemental Freeway Performance Measures..................................... A-2 Table A.2 Reasons for Undertaking Performance Measurement ........................ A-3 Table A.3 Congestion/Mobility Performance Measures Under Consideration in Selected DOTs ........................................................... A-7 Table A.4 Types of Performance Measures Used by Agencies ........................... A-8 Table A.5 Uses of Performance Measures .................................................................. A-14 Table A.6 Data Collection, Analysis, and Quality Procedures........................... A-17 v

ACKNOWLEDGMENTS The authors would like to thank Kenny Voorhies, Chris Hedden, Patricia Hendren, and Jocelyn Hoffman of Cambridge Systematics for their contributions in conducting the practitioner interviews, which were the main input to conducting the research. The research was directed by Richard Margiotta (Principal Investigator) and Tim Lomax (Co-Principal Investigator). vi

ABSTRACT This report documents the research performed on the project. Detailed recommendations and guidance are provided in the Guidebook, which is structured for providing transportation engineers and planners assistance in developing and maintaining a comprehensive freeway performance monitoring program. In the research, multiple aspects of freeway performance were considered, but congestion and mobility performance was emphasized because of the lack of guidance and experience in this area. Other aspects included safety, operational efficiency, ride qual- ity, environmental, and customer satisfaction. A review of current practice was conducted, including a review of the private sector as well as 11 benchmarking interviews with state and local transportation agencies. Based on these results, the Guidebook was structured to answer four primary questions about freeway performance: 1) what measures should be used; 2) how can the measures be developed with data and models; 3) how should freeway performance be communicated; and 4) how can freeway performance measures be used in decision-making. The draft Guidebook was developed as a series of nearly 400 annotated slides which were reviewed in five additional interviews with state and local agencies. The final Guidebook presents step-by-step procedures addressing the four primary issues associated with freeway performance monitoring. Ongoing freeway performance monitoring of recent trends is emphasized although the use of performance measures across project evaluations and analysis also is covered. vii

SUMMARY OF FINDINGS BACKGROUND The use of freeway performance measures has been growing in recent years, and ranges from site-specific operations analysis, corridor-level alternative investments analysis, and area-wide planning and public information studies. In the past few years, the issue of performance monitoring has been elevated by transportation agencies to be responsive to the demands of the public and state legislatures and the Transportation Equity Act for the 21st Century’s (TEA-21). The Safe, Accountable, Flexible, Efficient Transportation Equity Act for the 21st Century – A Legacy for Users (SAFETEA-LU), the most recent Federal transportation authorization legislation, continued this emphasis on performance monitoring, particularly with regard to system operations and management. Simultane- ously, the deployment of intelligent transportation systems (ITS) technologies has the potential to make a vast amount of data available for analysis. However, many challenges lie ahead before freeway performance measurement becomes “standard practice” and is imbedded in the transportation decision-making process. These challenges include the following: • The transportation profession is only beginning to define and measure congestion/mobility performance in objective terms. • More detailed measures than HCM-based levels of service are required to capture the effect of operational strategies, which are often more subtle than capacity expansion projects. • Based on what data are available, congestion is growing in areas of every size. • Freeway performance must be viewed from several perspectives. • The concept of “reliability” is growing in importance. • While advances in freeway performance concepts have been made, data limitations hamper their implementation. • In the short term, some combination of surveillance data, planning data, and modeling must be used to support freeway performance measurement. • Communication of freeway performance monitoring results also is crucial. • How freeway performance measures are to be used in the transportation decision-making process is still evolving. 1

BENCHMARKING INTERVIEWS A major part of this research effort was to ascertain what the more progressive agencies are doing in the area of freeway performance measures. A series of benchmarking interviews with state and local transportation agencies was conducted, resulting in the following findings. Motivations for Undertaking Freeway Performance Measurement Four motivations exist for agencies to undertake freeway performance measurement: 1. Legislative Mandates. State legislatures may require transportation (as well as other state) agencies to engage in a formal performance measurement and reporting process. Freeway performance measures are under- taken initially primarily to feed a mandated reporting process, but managers learn that there is intrinsic value in conducting freeway performance measurement for their own purposes (see reasons 3 and 4 below). 2. Agencywide Performance Measurement Initiatives. Even in the absence of legislative intervention, DOTs and MPOs often initiate department-wide performance measurement programs for a variety of reasons. Usually these are ostensibly linked to the notion of “customer focus” and improved public relations and involve- ment. Like legislative mandates, these efforts result in Annual Performance Reports. Freeway performance is usually couched in terms of congestion/mobility in these reports and is usually summarized at the State or major metropolitan area level. 3. Formal Business Plan Linkage, Particularly for Operations. Several agencies have taken a formal business plan approach to the actions. The undertaking of Business Plan can be dictated by DOT upper management or self-initiated by a champion. 4. Quantification of Benefits for Freeway Programs, Particularly for Operations. Operations personnel are discovering that when it comes to competing for internal resources and visibility, they are at a disad- vantage compared to other functional areas. Infrastructure programs have a long history of documenting the effects their program have users, as embodied in pavement, bridge, and maintenance management systems. “Not having the numbers” makes it hard to argue in favor of programs when others do have the numbers. In two of the interview cities freeway performance measurement has not yet been undertaken, though there were signs that this may change (i.e., they may just be “late adopters”). None of the four motivations currently are present in these cities and local managers are not convinced of the cost-effectiveness of implementing performance 2

measurement. In this sense, they are no different from the other cities – without strategic, legislative, or top man- agement mandates/initiatives, it is doubtful that the other areas would have undertaken performance measurement. Types of Performance Measures Used by Agencies Both outcome and output measures are used by agencies. In the literature of performance measurement, a distinction is made between output and outcome types of measures: • Output measures relate to the physical quantities of items; levels of effort expended, scale or scope of activities; and the efficiency in converting resources into some kind of product. Output measures are sometimes called “efficiency” measures. • Outcome measures relate to how well the firm or agency is meeting its mission and stated goals. In the private sector, outcome measures relate to the “bottom-line” – the financial viability of the firm (e.g., profit and revenue). For transportation agencies, outcomes are more related to the nature and extent of the services provided to transportation users. The research team and the project panel thought that, although the output/outcome dichotomy is well- established, it is confusing to new users. Therefore, an alternative naming convention was adopted: • “Quality of Service” is a more intuitive term for the outcome category of measures, and • “Activity-Based” is more apt for the output category of measures. It is clear that agencies who have undertaken freeway performance measurement have accessed the literature on performance measurement because they use the outcome/output terminology. For outcome measures, derivatives of speed and delay are commonly used by both operating and planning agencies. The Travel Time Index is a popular metric. Level of service as a metric is still in use in both planning and operations agencies, though it is not as widespread as it might have been 10 years ago. Reliability metrics have not yet found their way into widespread use. These metrics are usually formulated for short segments or at key loca- tions. An exception is Seattle where a series of defined “freeway trips” have been defined – these can involve travel over multiple freeway routes for extended lengths. Output measures are used primarily by operating agencies, and then primarily for incident management activities and the operation of field equipment (e.g., sensors, cameras). Many areas are beginning to define more sophisticated measures for measuring congestion/mobility per- formance but have not yet implemented them. Overall, there appears to be a trend away from the general categories of performance (LOS) and toward continuous measures that are based on delay and travel time. Further, 3

consideration of travel time reliability is growing in acceptance, though its implementation is still problematic, primarily due to data requirements. Customer satisfaction measures, where collected, are not used specifically for freeway performance measurement. Rather, they are instituted to gauge overall opinions about how well an agency is dealing with congestion. Use of Performance Measures Development of performance reports appears to be the major use right now for outcome-related freeway performance measures. The frequency of publication varies from weekly to annually, but annual reports are the most common. The linking of performance measures (more specifically, changes in them over time or their level relative to preset targets) and investment decisions is not well established. The best examples of actions taken based on performance measures is the tracking of detailed output measures for incident management programs – there is evidence that agencies act on these to modify activities such as service patrol routing and schedules. However, a linkage between major freeway investments and outcome measures (e.g., freeway delay) was not found. This may be due to the lack of experience with developing and applying the measures rather than with an unwillingness to use them to support investment decisions. It is true, however, that having better information on the scope and causes of congestion tends to lead towards more open thinking about what to fund to improve the situation. What it does not solve is the fact that the state and MPO planning processes choose large investments that are based on long-range needs rather than on short-term changes in performance measures (or the failure to meet current performance targets). State DOTs and MPOs have not yet directly collaborated in joint efforts in developing freeway perform- ance measurement programs. There seems to be a split of responsibility along traditional lines: DOTs tend to han- dle construction and operations while MPOs handle planning activities. Some MPOs and DOTs use common measures, but also develop measures unique to their applications. Data Collection and Analysis Metrics are developed through a variety of methods: operations agencies (whose focus is primarily free- ways) rely heavily on archived roadway surveillance data; the development of formal data archive management systems is on the rise. Planning agencies (whose purview includes all roadways in an area) use a mix of methods, including travel demand forecasting and other models; sample-based travel time runs from floating cars; and 4

overlapping aerial photography. Planning agencies are just now starting to tap ITS data archives as a source of data for performance measures; this occurrence is not very widespread. Universities within a state are commonly used to set up (at least initially) performance measurement pro- grams and data archives. Sometimes these functions are passed on to the DOT, sometimes the universities retain control. Integration of the various data sources available (ITS roadway surveillance, events {incidents, weather, work zones}, and sample-based data) is not well very well advanced. However, there is recognition that this must occur, especially in areas that consider delay by congestion source (e.g., incidents) as an important measure. Collection and use of incident data are becoming more common among freeway management systems. However, every area defines data elements and collects data differently. Work zones are occasionally collected as part of incident data. Collection of weather data is uncommon. Data Quality The quality of data from ITS roadway sensors is a major concern of agencies and has even caused trepida- tion in using the data for freeway performance measurement. Data quality problems can be traced primarily to two sources: 1) improper installation (including initial calibration and acceptance testing of equipment) and 2) inadequate detector maintenance due to funding shortfalls. This is a serious problem for freeway performance measurement, especially since ITS roadway sensors provide continuous data at the small time and geographic increments necessary to support sophisticated measures (reliability, congestion by source). The most extreme case is Houston which basically relies on probe readers for travel time estimates – they do not rely on the roadway sen- sors originally installed. As a result, since volumes are not available, not all the performance measures that are possible can be constructed. When formal archived data management systems are implemented, data quality control checks are insti- tuted. However, these are post hoc in nature – they can test for inconsistencies based on valid ranges, checks against theory, and checks against history, but subtle errors in accuracy still occur and are unknown. (The only way to determine accuracy is to validate field measurements independently.) 5

GUIDEBOOK DEVELOPMENT The Guidebook was structured to deal with the technical and institutional issues identified in the bench- marking interviews. The Guidebook addresses each stage in the freeway performance measurement process with step-by-step procedures for transportations to follow. The scope of the Guidebook covers: • Urban and rural freeways: a comprehensive approach to freeway performance measurement; • A focus on throughput/congestion/mobility of freeways, because of the lack of experience in these areas; • Discussion of additional aspects of freeway performance: o Freeway safety; o Operational efficiency; o Ride quality – Affects quality of traffic flow (link to asset management information systems); o Environmental – Emissions and fuel use; and o Customer satisfaction. A chapter of the Guidebook is devoted to each stage of the freeway performance measurement process: • Rationale for Freeway Performance Measurement: “Why are we doing this?” • Context for Freeway Performance Measurement: “How does it fit in?” • Performance Measures (Metrics): “What measures should be used?” • Supporting Data and Methods: “How are the measures developed?” • Presentation and Communication: “How are the measures best presented?” • Use of Freeway Performance Measures in Decision-Making: “How are the measures used to support decisions?” 6

CHAPTER 1 INTRODUCTION AND RESEARCH APPROACH 1.1 BACKGROUND The use of freeway performance measures has been growing in recent years, and ranges from site-specific operations analysis to corridor-level alternative investments analysis and to areawide planning and public informa- tion studies. In the past few years, the issue of performance monitoring has been elevated by transportation agencies to be responsive to the demands of the public and state legislatures and the Transportation Equity Act for the 21st Century’s (TEA-21). The Safe, Accountable, Flexible, Efficient Transportation Equity Act for the 21st Century – A Legacy for Users (SAFETEA-LU), the most recent Federal transportation authorization legislation, continued this emphasis on performance monitoring, particularly with regard to system operations and management. Simultaneously, the deployment of intelligent transportation systems (ITS) technologies has the potential to make a vast amount of data available for analysis. However, many challenges lie ahead before freeway performance measurement becomes “standard prac- tice” and is imbedded in the transportation decision-making process. These challenges include the following below. The transportation profession is only beginning to define and measure congestion/mobility perform- ance in objective terms. For more than 35 years, the Highway Capacity Manual (HCM)1 has served as the focal point for defining quality of traffic flow. Prior to the 2000 edition of the Manual, performance was defined by broad ranges of “levels of service” (LOS). Even with the publication of the 2000 edition, freeway performance is still largely tied to the level of service concept. The 2000 edition of the Manual is beginning to address the “saturated flow regime” (i.e., congestion) in a comprehensive fashion and to recognize that a single LOS category, (“F”) does not capture the nature and extent of congestion. At the local level, measuring and reporting congestion have often been done anecdotally without the advantage of the limited application of the HCM. Future versions of the HCM will delve into this problem more deeply. More detailed measures than HCM-based levels of service are required to capture the effect of operational strategies, which are often more subtle than capacity expansion projects. Implementing operational strategies usu- ally never eliminate congestion but rather improve it slightly. These effects are not captured with the broad LOS ranges recommended by the HCM. 7

Based on what data are available, congestion is growing in areas of every size. The Texas Transportation Institute’s (TTI) 2004 Annual Urban Mobility Report2 shows more severe congestion that lasts for a longer period of time, and affects more of the transportation network in 1999 than in 1982 in all urban population categories. The average annual delay per person climbed from 11 hours in 1982 to 36 hours in 1999. And delay over the same period quintupled in areas with less than one million people. The time to complete a trip during the congested period also continues to get longer. Further, congestion is consuming a greater part of the day in many metropolitan areas. The concept of a “peak hour” (rush hour) has been rendered irrelevant by travel patterns that have led to “peak periods” – multiple successive hours characterized by congestion. Freeway performance must be viewed from several perspectives. A debate within the profession has arisen over the proper perspective for measuring performance. With regard to mobility performance, some have suggested that the view of the user (traveler) is the most appropriate, while others argue that the view from the facility is the correct perspective. We have found this to be a specious argument: both perspectives are needed. The user perspective is important, because that is how transportation customers experience the system; this relates to characteristics of users’ trips. The facility perspective is important, because transportation professionals mainly manage facilities; trips also are managed by such strategies as traveler information and demand management, but to a lesser degree than facilities. Further, the two perspectives are closely related in computation, data requirements, and the measures that can be applied. With regard to freeway performance, “trips” can be defined over extended segments. Finally, homeland security issues are becoming increasingly important for transportation professionals. Freeway performance measures can be useful in both planning (identifying evacuation routes) and operations (real- time management of evacuations.) The concept of “reliability” is growing in importance. There is growing recognition in the profession that not only does congestion occur on “typical” or “average” days, but it is the variability that occurs day to day that is important. Therefore, freeway performance must include the notion of reliability to be useful to both operators and planners. While advances in freeway performance concepts have been made, data limitations hamper their implementation. As performance concepts become more sophisticated, the data requirements of supporting them become more onerous. In particular, reliability requires that data be collected nearly continuously. Even without considering reliability, more detailed data resolution is required to monitor changes due to operational strategies; 8

and traditional monitoring data, which are scattered and sampled, may be adequate for determining major capacity expansions, but lack the resolution to capture the effects of more modest operational improvements. As our own work has demonstrated, freeway surveillance data generated from ITS technologies can be used effectively for these purposes. But these data bring with them a variety of new issues, among them: • In the short term, some combination of surveillance data, planning data, and modeling must be used to support freeway performance measurement. Since surveillance coverage is not complete and data problems will cause gaps in existing coverage, other means must be used to fill in the freeway per- formance picture. However, the system performance data derived from surveillance data may be sig- nificantly different from other estimates or modeling efforts. Combining freeway surveillance data with other data sources should be conducted only where the differences in each type of data are well understood, and where the need for a combination of data is unavoidable. • Communication of freeway performance monitoring results also is crucial. This involves not only selecting measures that are easily understood by a broad audience, but also conveying the results in formats that can be easily interpreted. Communication to both technical and lay audiences is a major part of our current efforts in this area, and we will build on this experience. • How freeway performance measures are to be used in the transportation decision-making proc- ess is still evolving. Most of the work to date on freeway performance monitoring has been in defining the concepts, measures, and data to support them. However, it is clear that the profession must move beyond the simple reporting of freeway performance trends – performance measures must be used to develop better investment decisions. 1.2 SCOPE OF THE RESEARCH The objective of the research is to produce a practical Guidebook that provides guidance: “… on the effec- tive use of freeway performance measures in operating the system and in meeting the information needs of a large spectrum of potential local, regional, and national users.” The Guidebook presents a comprehensive approach to measuring the performance of urban and rural freeways. Freeways are defined as access-controlled highways char- acterized by uninterrupted traffic flow. The aspects of freeway performance covered by this Guidebook are as follows. 9

The focus of the work is on congestion/mobility performance of freeways. This can be further defined as “quality of traffic flow or traffic conditions as experienced by users of the freeway.” This category includes meas- ures related to typical congestion levels, travel time reliability, and throughput. It also includes supporting measures on the nature of roadway “events” that impede traffic flow: incidents, weather, and work zones. Most of the research conducted in preparing the Guidebook shows how these measures are developed from data and other ana- lytic methods. Mobility is defined differently for urban versus rural freeways, as discussed later in this Guidebook. The rationale for focusing on congestion/mobility performance is that, of the major performance categories, it has the least amount of history with practitioners and is the least well formed. New concepts such as travel time reliability and the deployment of transportation operations strategies in recent years underscore the emerging nature of this area. Despite the focus on congestion/mobility, other aspects of freeway performance also are covered, but not at the same level of detail. These areas (with the exception of customer satisfaction) have a much longer history of performance measurement in the profession. So, rather than “reinvent the wheel,” the Guidebook uses references for much of its material. As a result, data and methods for other aspects of freeway performance are not covered in detail. Rather, the measures for each category are identified and methods for integrating them into a comprehensive freeway performance measurement program are presented, including their use in applications and decision-making. In some cases, these other performance aspects have or are developing their own performance measures, usually applied on an areawide basis. These additional aspects of freeway performance are: • Freeway Safety – Especially safety aspects that are under the direct control of transportation agencies. Safety performance measures are now being considered as part of the recent emphasis on comprehensive highway safety plans. • Operational Efficiency – Measures that relate to the activities and equipment used in freeway management. • Ride Quality – Especially as it relates to the quality of traffic flow. Asset management information systems have long history of ride quality performance measures. • Environmental – Emissions and fuel use are the areas covered in this Guidebook, although many other additional environmental aspects could be covered. 10

• Customer Satisfaction – As transportation agencies adopt a stronger focus on their customers (i.e., users of the system), customer perceptions of performance are becoming important feedback on the effectiveness of transportation programs. 1.3 RESEARCH APPROACH The research undertaken for this project was in several parts. • The research team compiled a list of potential performance measures and their uses by reviewing the literature and compiling their own experiences with Federal, state, and local agencies. • Benchmarking interviews were conducted to ascertain the state of the practice. Agencies from 10 areas were interviewed (Table 1). Six of the interviews included multiple agencies, usually state operations personnel, state, and local planners. Four interviews were conducted with operations personnel only. • An interim report and detailed annotated outline were produced based on the above activities. It included establishing basic principles for freeway performance measures that were used to guide the rest of the project. • The annotated outline was used to construct approximately 400 annotated slides which form the basis for the Guidebook. The slides were distributed to agencies from five areas and these areas were interviewed by the research team to validate the approach and information: o Oregon DOT/Portland Metro; o Arizona DOT/Maricopa Association of Governments; o Minnesota DOT/Twin Cities Metropolitan Council; o Georgia DOT/Atlanta Regional Commission; and o New York State DOT/Capital District Transportation Committee. • Based on the validation interviews, the annotated outline was revised and the draft Guidebook was prepared. This preparation included developing analysis procedures using data from ITS sources. 11

Table 1. Initial Benchmarking Interview Locations Metro Area Agencies Interviewed Multiple Agency Interviews 1. Minneapolis-St. Paul, Minnesota Metro District Operations, Mn/DOT Center for Transportation Studies, University of Minnesota Metro Council 2. Seattle, Washington WSDOT HQ Traffic Office WSDOT NW Region WSDOT HQ Strategic Planning and Programming Puget Sound Regional Council 3. Hampton Roads, Virginia Hampton Roads STC, VDOT Hampton Roads Planning District Commission 4. Milwaukee, Wisconsin WisDOT District 2 Operations WisDOT Central Office University of Wisconsin-Madison WisDOT District 2 Planning 5. Phoenix, Arizona ADOT/Intermodal Transportation Division ADOT/Transportation Planning Division Maricopa Association of Governments 6. Los Angeles, California Caltrans, Freeway Operations, District 7 Southern California Association of Governments Caltrans, Planning Operations Interviews Only 7. Portland, Oregon See Note1 8. Houston, Texas Houston TRANSTAR 9. San Antonio, Texas See Note2 10. Washington, D.C. CHART (Maryland) VDOT Northern Virginia District 11. Atlanta, Georgia GDOT, Office of Traffic Operations 1 No formal interviews were conducted as part of NCHRP 3-68. Rather, the team relied on other work conducted by TTI on performance measures. As part of this effort ODOT assembled a Technical Advisory Committee (TAC) made up of individuals from ODOT sections of traffic management, transportation planning and analysis, transportation data, traffic operations, and internal audit/performance measures. The TAC also included individuals from the metropolitan planning organization (MPO) in Portland (Metro) and the Eugene/Springfield area (Lane Council of Governments), academia, and the local Federal Highway Administration (FHWA) office. 2 Initial conversations with TransGuide indicated that they are not currently using performance measures nor are they planning on developing them in the near future. TransGuide does have an extensive sensor system (485 lane-miles) and a formal incident management program from which detailed performance measures could be developed, however. 12

1.4 RELATIONSHIP TO CURRENT RESEARCH EFFORTS There currently is much activity in the area of performance measurement, particularly for conges- tion/mobility performance. Table 2 summarizes these efforts and indicates their relationship/value to the current project. Many of these projects have been drawn on in later sections to provide examples of freeway performance measurement. Note that only projects that are active have been included; these are the ones that may influence – and be influenced by – the current project. In addition, the Strategic Highway Research Program II (SHRP II) also may include projects related to freeway performance measurement. Of the projects listed in Table 2, NCHRP Project 7-15 is the closest in nature to the research in this report. HRP 3-68 has coordinated with the Project 7-15 team to ensure synergy and avoid duplication. The thrust of Project 7-15 is the use of travel time, delay, and reliability measures in a wide variety of applications undertaken by planners. Note that it is meant to cover all highway types, not just freeways. Much detail on development of per- formance measures is given there, and in some ways the Project 7-15 report can be viewed as a companion document to this one. Specifically, NCHRP 7-15 is geared to estimating travel time, delay, and reliability for the following planning applications: • Application #1. Evaluate Trends In Travel Time, Delay, And Reliability. The objective of this application is to identify and track overall trends in travel time, delay, and reliability for the purposes of preparing a report to the public on agency performance. • Application #2. Identify Existing Deficiencies. The objective of this application is to identify and diagnose existing deficiencies in travel time, delay, and reliability for the purposes of determining appropriate agency actions. • Application #3. Evaluation of Effectiveness of Improvements. The objective of this application is to determine if the implemented improvement actually resulting in the desired travel time, delay and reliability savings for the purposes of the cost-effectiveness of agency of specific actions. 13

• Application #4. Prediction of Future Conditions. The objective of this application is to identify and diagnose future deficiencies in travel time, delay, and reliability for the purposes of determining appropriate agency actions. • Application #5. Alternatives Analysis. The objective of this application is to develop a set of actions to improve facility or system performance. Of these applications, Application #1 has the most overlap with this report; the Guidebook is focused on developing ongoing performance monitoring programs. The Guidebook provides exhaustive detail on this application. The other applications are discussed in the Guidebook, but NCHRP 7-15 covers these in detail. On the other hand, the congestion/mobility performance measures (including reliability) identified in the Guidebook are the basis for the NCHRP 7-15 effort. 1.5 GUIDEBOOK Development The Guidebook was structured to deal with the technical and institutional issues identified in the bench- marking interviews. The Guidebook addresses each stage in the freeway performance measurement process with step-by-step procedures for transportations to follow. In developing this Guidebook, a set of primary questions was developed surrounding freeway performance measurement. Answering these questions from the practitioner’s point of view is the thrust of the Guidebook and serves as the basis for its structure: • Rationale for Freeway Performance Measurement: “Why are we doing this?” Why is freeway performance important and why should its measurement be undertaken? What applications and uses can use freeway performance measures? What is the current state of the practice in freeway performance measurement? (Section 3.0) • Context for Freeway Performance Measurement: “How does it fit in?” How does one establish and maintain a freeway performance measurement program? How does it mesh with other local, regional, state, and national activities in planning, operations, maintenance, and design? How should it evolve over time? (Section 4.0) • Performance Measures (Metrics): “What measures should be used?” What aspects of freeway performance should be measured? What principles should be followed in establishing freeway 14

performance measurement programs? What specific and quantifiable measures (metrics) should be used for each of these aspects? (Sections 5.0 and 6.0) • Supporting Data and Methods: “How are the measures developed?” What data are required to support the development of freeway performance measures? What data collection mechanisms are available now or should be instituted? (Section 7.0) How should the data be processed and combined with analytic methods to create freeway performance measures? (Section 8.0) • Presentation and Communication: “How are the measures best presented?” What are the options for presenting freeway performance measures to other professionals, decision-makers, and the public? How should freeway performance be explained and what is the significance of trends? (Section 9.0) • Use of Freeway Performance Measures in Decision-Making: “How are the measures used to support decisions?” What should stakeholders’ involvement be in using freeway performance meas- ures? What are some examples of how freeway performance measures can be used in the decision- making process for setting policies and guiding investments? (Section 10.0) 15

Table 2. Relationship of NCHRP 3-68 to Other Current Performance Measurement Projects Project Description Relationship to NCHRP 3-68 NCHRP 7-15, Cost-Effective Measures and Planning Procedures for Travel Time, Delay, and Reliabilitya Developing analytic methods to compute travel time reliability measures, including when continuously collected data is not available. Delay by source of congestion also being considered. Reliability measures will be compatible; analytic methods will be of value in computing freeway performance, especially for planning applications. FHWA, Urban Congestion Report (UCR)b Monthly reports on areawide freeway congestion developed from web-based speed maps and data. Provides example of how to track trends at the metropolitan area level and develop performance measures from available data. FHWA, Mobility Monitoring Programc Annual reports (soon to be monthly) on corridor and areawide freeway congestion developed from archived and QC-passed surveillance data. Similar to UCR for tracking trends, although corridors are the basic unit of analysis (more valuable to locals); special studies include “Lessons Learned” and analysis method to decompose congestion by source. TTI, Urban Mobility Studyd Freeway and arterial areawide congestion trends for top 78 metro areas. Long-standing history of congestion trends, widely accepted; pioneered new measures of congestion and develops them from planning-level data. FHWA, Work Zone Performance Measures Highly detailed performance measures and supporting data collection for monitoring work zone performance at the national and state levels. Includes both outcome and output measures for 13 categories of work zone performance. NCHRP 3-81, Strategies for Integrated Operation of Freeway and Arterial Corridorse Project is to develop a manual of recommended strategies for integrating the operation of a freeway and arterial corridor, including their benefits and methods of implementing them. Performance measures used to evaluate effectiveness of various strategies and serve as a basis for implementing them. NCHRP 8-36/Task 47, Effective Organization of Performance Measurement Studying: 1) how transportation organizations structure the performance measurement function; 2) how they organize and deliver performance information; 3) how performance measures are used to guide decisions at levels from top management down to operations; and 4) how measures are used in asset management. Addressing the key issue of how performance measures are used in decision-making. NCHRP 3-85, Guidance for the Use of Simulation and Other Models in Highway Capacity Analyses This project will enhance the guidance in the Highway Capacity Manual for selection and use of simulation and other models. Measures of effectiveness from model outputs are essentially performance measures (see Sections 4.4 and 5.0). ahttp://www4.trb.org/trb/crp.nsf/e7bcd526f5af4a2c8525672f006245fa/62bad24780b7ac4b85256d0b005e07fb?OpenDocument. btrb.org/Conferences/NATMEC/35-Wunderlich.pdf. chttp://mobility.tamu.edu/mmp/. dhttp:/mobility.tamu.edu/ums/. ehttp://www4.trb.org/trb/crp.nsf/e7bcd526f5af4a2c8525672f006245fa/e1818912cb5a8ade85256efd005b6770?OpenDocument. 16

CHAPTER 2 FINDINGS 2.1 BENCHMARKING INTERVIEWS A major part of this research effort was to ascertain what the more progressive agencies are doing in the area of freeway performance measures. From these interviews it was then possible to borrow selected best practices and weave them into the Guidebook. It also was possible to get an understanding of what will and won’t be palatable to agencies in terms of the Guidebook procedures. Details of the interviews appear in the Appendix and are summarized below. Motivations for Undertaking Freeway Performance Measurement Four motivations exist for agencies to undertake freeway performance measurement: 1. Legislative Mandates. State legislatures may require transportation (as well as other state) agencies to engage in a formal performance measurement and reporting process. Freeway performance measures are undertaken initially primarily to feed the mandated reporting process, managers learn that there is intrinsic value in conducting freeway performance measurement for their own purposes (see reasons 3 and 4 below). 2. Agencywide Performance Measurement Initiatives. Even in the absence of legislative intervention, DOTs and MPOs often initiate department-wide performance measurement programs for a variety of rea- sons. Usually these are ostensibly linked to the notion of “customer focus” and improved public relations and involvement. Like legislative mandates, these efforts result in Annual Performance Reports. Freeway performance is usually couched in terms of congestion/mobility in these reports and are usually summarized at the State or major metropolitan area level. 3. Formal Business Plan Linkage, Particularly for Operations. Several agencies have taken a formal busi- ness plan approach to the actions. Most of these are found in the private sector scan of performance measurement, i.e., the Vision-Goals-Objectives-Performance Measures-Targets-Actions sequence. The undertaking of Business Plan can be dictated by DOT upper management or self-initiated by a champion. 4. Quantification of Benefits for Freeway Programs, Particularly for Operations. Operations personnel are discovering that when it comes to competing for internal resources and visibility, they are at a disad- vantage compared to other functional areas. Infrastructure programs have a long history of documenting 17

the effects their program have users, as embodied in pavement, bridge, and maintenance management sys- tems. “Not having the numbers” makes it hard to argue in favor of programs when others do have the numbers. Maryland State Highway Administration (SHA) and Wisconsin DOT are two examples of this. In two of the interview cities – San Antonio and Houston – freeway performance measurement has not yet been undertaken, though there are signs that this may change (i.e., they may just be “late adopters”). None of the four motivations currently are present in these cities and local managers are not convinced of the cost-effectiveness of implementing performance measurement. In this sense, they are no different from the other cities – without stra- tegic, legislative, or top management mandates/initiatives, it is doubtful that the other areas would have undertaken performance measurement. In applying performance measurement concepts, it appears that public agencies mirror those in the private sector. Implementation of the concepts are the difficult part for public agencies. Once the institutional hurdle of establishing a performance measurement program is passed, many technical difficulties still lie ahead, as discussed below. Types of Performance Measures Used by Agencies Both outcome and output measures are used by agencies. It is clear that agencies who have undertaken freeway performance measurement have accessed the literature on performance measurement because they use the outcome/output terminology. For outcome measures, derivatives of speed and delay are commonly used by both operating and planning agencies. The Travel Time Index is a popular metric. Level of service as a metric is still in use in both planning and operations agencies, though it is not as widespread as it might have been 10 years ago. Reliability metrics have not yet found their way into widespread use. (Seattle and Minneapolis are exceptions.) These metrics are usually formulated for short segments or at key locations. An exception is Seattle where a series of defined “freeway trips” have been defined – these can involve travel over multiple freeway routes for extended lengths. 18

Some of the more interesting metrics used by agencies include: • The number of very slow trips (half of free flow speed) that occurs each year by time of day and major trip (Seattle); • Percentage of reduction in incident congestion delay; and • Percent of freeway lane-miles below congested volumes (based on volume per lane). Output measures are used primarily by operating agencies, and then primarily for incident management activities and the operation of field equipment (e.g., sensors, cameras). Many areas are beginning to define more sophisticated measures for measuring congestion/mobility per- formance but have not yet implemented them. Overall, there appears to be a trend away from the general categories of performance (LOS) and toward continuous measures that are based on delay and travel time. Further, considera- tion of travel time reliability is growing in acceptance, though its implementation is still problematic, primarily due to data requirements. Customer satisfaction measures, where collected, are not used specifically for freeway performance measurement. Rather, they are instituted to gauge overall opinions about how well an agency is dealing with congestion. Use of Performance Measures Development of performance reports appears to be the major use right now for outcome-related freeway performance measures. The frequency of publication varies from weekly to annually, but annual reports are the most common. The existence of dashboards was not found in any of the areas, though the Minnesota Department of Transportation (Mn/DOT) is in the development phase. The linking of performance measures (more specifically, changes in them over time or their level relative to preset targets) and investment decisions is not well established. The best examples of actions taken based on performance measures is the tracking of detailed output measures for incident management programs – there is evi- dence that agencies act on these to modify activities such as service patrol routing and schedules. However, a linkage between major freeway investments and outcome measures (e.g., freeway delay) was not found. Washington State seems to be farthest along on this matter, but even there the correlation is not direct. This may be due to the lack of experience with developing and applying the measures rather than with an unwillingness to use them to support investment decisions. It is true, however, that having better information on the scope and causes of 19

congestion tends to lead towards more open thinking about what to fund to improve the situation (at least in the case of Washington State). What it does not solve is the fact that the State and MPO planning processes choose large investments that are based on long-range needs rather than on short-term changes in performance measures (or the failure to meet current performance targets). State DOTs and MPOs have not yet directly collaborate in joint efforts in developing freeway performance measurement programs. There seems to be a split of responsibility along traditional lines: DOTs tend to handle construction and operations while MPOs handle planning activities. Some MPOs and DOTs use common measures, but also develop measures unique to their applications. Data Collection and Analysis Metrics are developed through a variety of methods: operations agencies (whose focus is primarily free- ways) rely heavily on archived roadway surveillance data; the development of formal data archive management systems is on the rise. Planning agencies (whose purview includes all roadways in an area) use a mix of methods, including travel demand forecasting and other models; sample-based travel time runs from floating cars; and over- lapping aerial photography. Planning agencies are just now starting to tap ITS data archives as a source of data for performance measures; this occurrence is not very widespread. Universities within a state are commonly used to at least initially set up performance measurement pro- grams and data archives. Sometimes these functions are passed on to the DOT, sometimes the universities retain control. Integration of the various data sources available (ITS roadway surveillance, events {incidents, weather, work zones}, and sample-based data) is not well very well advanced. However, there is recognition that this must occur, especially in areas that consider delay by congestion source (e.g., incidents) as an important measure. Collection and use of incident data is becoming more common among freeway management systems. However, every area defines data elements and collects data differently. Work zones are occasionally collected as part of incident data. Collection of weather data is uncommon. Data Quality The quality of data from ITS roadway sensors is a major concern of agencies and has even caused trepidation in using the data for freeway performance measurement (e.g., Atlanta). Data quality problems can be traced primarily to two sources: 1) improper installation (including initial calibration and acceptance testing of 20

equipment) and 2) inadequate detector maintenance due to funding shortfalls. This is a serious problem for freeway performance measurement, especially since ITS roadway sensors provide continuous data at the small time and geo- graphic increments necessary to support sophisticated measures (reliability, congestion by source). The most extreme case is Houston which basically relies on probe readers for travel time estimates – they do not rely on the roadway sensors originally installed. As a result, since volumes are not available, not all the performance measures that are possible can be constructed. Several agencies have done formal studies of data quality. One strategy to deal with data quality (identified by the Georgia Department of Transportation; GDOT) is to concentrate calibration and maintenance on “key” detectors, with the idea that these can be used to detect major problems (e.g., at known bottlenecks). The key detectors can then be used to adjust measurements from the remaining detectors. However, it is unclear how the adjustments will be done and how well this procedures will work to improve data quality. Father, developing per- formance measurements from a few isolated detector locations also is highly problematic – since most of the detailed performance measures are based on converting detector measurements to travel times in a corridor, the efficacy of this approach is in doubt. When formal archived data management systems are implemented, data quality control checks are insti- tuted. However, these are post hoc in nature – they can test for inconsistencies based on valid ranges, checks against theory, and checks against history, but subtle errors in accuracy still occur and are unknown. (The only way to determine accuracy is to independently validate field measurements.) 2.2 Basic Principles for Freeway Performance Measurement In order to develop the suite of freeway performance measures, the research team developed a set of basic principles for guidance. Table 3 summarizes the principles. Consistent with the scope of the project, the focus is on measuring the performance of freeways in terms of congestion/mobility and the activities related to improving traffic flow. Detailed discussion of each principle appears in the Guidebook. 21

Table 3. Basic Principles for Freeway Performance Monitoring Principle 1 Mobility performance measures must be based on the measurement or estimation of travel time. Principle 2 Measure where you can – model everything else Principle 3 Multiple metrics should be used to report freeway performance, especially for mobility. Principle 4 Traditional HCM-based performance measures for mobility (V/C3 ratio and level of service) should not be ignored but should serve as supplementary, not primary measures of performance in most cases. Principle 5 Both vehicle- and person-based performance measures of throughput are useful and should be developed, depending on the application. Principle 6 Both quality of service (outcome) and activity-based (output) performance measures are required for freeway performance monitoring. Principle 7 Activity-based measures should be chosen so that improvements in them can be linked to improvements in quality of service measures. Principle 8 Customer satisfaction measures should be included with quality of service measures for monitoring freeway performance. Principle 9 The measurement of travel time reliability is a key aspect of freeway performance measurement and reliability measures should be developed and applied. Principle 10 Three dimensions of freeway mobility/congestion should be tracked with mobility performance measures: source of congestion, temporal aspects, and spatial detail. Principle 11 Communication of freeway performance measurement should be done with graphics that resonate with a variety of technical and nontechnical audiences. Principle 12 Continuity should be maintained in performance measures across applications and time horizons; the same performance measures should be used for trend monitoring, project design, forecasting, and evaluations. 2.3 Recommended Freeway Performance Measures The recommended performance measures fall into two categories: Core (Table 4) and Supplemental (Table 5). The Core measures represent those that should be developed by all agencies involved with freeway performance that have sufficient data available to them to undertake their development. In cases where data currently do not exist, agencies should strongly consider developing the data necessary to compute the Core measures. 3 Volume-to-capacity 22

Table 4. Recommended Core Freeway Performance Measures Performance Metric Definition Units Geographic Scale Time Scale Relationship to NTOC Measures Average (Typical) Congestion Conditions (Quality of Service) Travel Time The average time consumed by vehicles traversing a fixed distance of freeway Minutes Specific points on a section or a representative trip only; separately for GP and HOV lanes Peak hour, a.m./p.m. peak periods, midday, daily Direct correspondence to NTOC measure, but distinction between “link” and “trip” travel time is not used Travel Time Index The ratio of the actual travel rate to the ideal travel ratea None; minimum value = 1.000 Section and areawide as a minimum; separately for GP and HOV lanes Peak hour, a.m./p.m. peak periods, midday, daily Not recommended by BTOC Total Delay, Vehicles The excess travel time used on a trip, facility, or freeway segment beyond what would occur under ideal conditionsb Vehicle-hours Section and areawide as a minimum; separately for GP and HOV lanes Peak hour, a.m./p.m. peak periods, midday, daily NTOC distinguishes between recurring and nonrecurring delay; delay by source recommended by Guidebook as supplements Total Delay, Persons The excess travel time used on a trip, facility, or freeway segment beyond what would occur under ideal conditionsc Person-hours Section and areawide as a minimum; separately for GP and HOV lanes Peak hour, a.m./p.m. peak periods, midday, daily NTOC distinguishes between recurring and nonrecurring delay; delay by source recommended by Guidebook as supplements Delay per Vehicle Total freeway delay divided by the number of vehicles using the freeway Hours (vehicle-hours per vehicle) Section and areawide Peak hour, a.m./p.m. peak periods; daily Not recommended by NTOC Spatial Extent of Congestion No. 1 Percent of Freeway VMT with Average Section Speeds <50 mphd Percent Section and areawide Peak hour, a.m./p.m. peak periods Spatial Extent of Congestion No. 2 Percent of Freeway VMT with Average Section Speeds <30 mph Percent Section and areawide Peak hour, a.m./p.m. peak periods NTOC uses a single measure with different thresholds, but the concept is fundamentally the same Temporal Extent of Congestion No. 1 Percent of Day with Average Freeway Section Speeds <50 mph Percent Section and areawide Daily Temporal Extent of Congestion No. 2 Percent of Day with Average Freeway Section Speeds <30 mph Percent Section and areawide Daily NTOC uses a single measure with different thresholds, but the concept is fundamentally the same Density Number of vehicles occupying a length of freeway Vehicles per lane-mile Section Peak hour/periods for weekday/weekend Not recommended by NTOC Reliability (Quality of Service) Buffer Index The difference between the 95th percentile travel time and the average travel time, normalized by the average travel time Percent Section and areawide Peak hour, a.m./p.m. peak periods, midday, daily Planning Time Index The 95th Percentile Travel Time Index None; minimum value = 1.000 Section and areawide Peak hour, a.m./p.m. peak periods, midday, daily NTOC recommends a “buffer time” which is the difference between the 95th percentile travel time and the average; conceptually the same as the Guidebook a. Travel rate is the inverse of speed, measured in minutes per mile. The “ideal travel rate” is the rate that occurs at the free flow speed of a facility, or a fixed value set for all facilities that is meant to indicate ideal conditions or “unconstrained” (see text for discussion of the ideal/unconstrained/free flow speed). b. See text above for definition of “ideal.” c. See text above for definition of “ideal.” d. A freeway “section” is length of freeway that represents a relatively homogenous trip by users. Logical breakpoints are major interchanges (especially freeway-to-freeway) and destinations (e.g., Central Business District). The term “section” is sometimes used to describe this, but it usually implies additional parallel freeways and/or transit routes. 23

Table 4. Recommended Core Freeway Performance Measures (continued) Performance Metric Definition Units Geographic Scale Time Scale Relationship to NTOC Measures Capacity Bottlenecks (Activity-Based) Geometric Deficiencies Related to Traffic Flow (Potential Bottlenecks) Count of potential bottleneck locations by typee Number Section and areawide N/A Not recommended by NTOC Major Traffic- Influencing Bottlenecks Count of locations that are the primary cause of traffic flow breakdown on a highway section, by type Number Section and areawide N/A Not recommended by NTOC Throughput (Quality of Service) Throughput – Vehicle Number of vehicles traversing a freeway in vehicles Vehicles per unit time Section and areawide Peak hour, a.m./p.m. peak periods, midday, daily Direct correspondence to NTOC measure Throughout – Persons Number of persons traversing a freeway Persons per unit time Section and areawide Peak hour, a.m./p.m. peak periods, midday, daily Direct correspondence to NTOC measure Vehicle-Miles of Travel The product of the number of vehicles traveling over a length of freeway, times the length of the freeway Vehicle-miles Section and areawide Peak hour, a.m./p.m. peak periods, midday, daily Not recommended by NTOC Truck Vehicle-Miles of Travel The product of the number of trucks traveling over a length of freeway,f times the length of the freeway Vehicle-miles Section and areawide Peak hour, a.m./p.m. peak periods, midday, daily Not recommended by NTOC Lost Highway Productivity Lost capacity due to flow breakdown – the difference between measured volumes on a freeway segment under congested flow versus the maximum capacity for that segment Vehicles per hour Section and areawide Peak hour, a.m./p.m. peak periods, midday, daily Not recommended by NTOC Customer Satisfaction (Quality of Service) Worst Aspect of Freeway Congestion (Defined by question) 1) happens every work day; 2) incidents that are not cleared in time; and 3) encountering work zones Areawide or statewide Annually; tied to survey frequency Not recommended by NTOC Satisfaction with Time to Make Long-Distance Trips Using Freeways (Defined by question) 1) very satisfied; 2) somewhat satisfied; 3) neutral; 4) somewhat dissatisfied; 5) very dissatisfied; and 6) do not know Areawide or statewide Annually; tied to survey frequency Direct correspondence to NTOC measure e. Bottleneck types are: Types A-C weaving areas (see HCM and Section 7.0); left exits; freeway-to-freeway merge areas; surface street on-ramp merge areas; acceleration lanes at merge areas <300 feet; lane drops; lane width drops >= 1 foot; directional miles with left shoulders <6 feet; directional miles with right shoulders <6 feet; steep grades; substandard horizontal curves. The shoulder categories are included because of the ability of more than 6-foot shoulders to shelter vehicles during traffic incidents. f. Trucks are defined as vehicles with at least six tires, i.e., FHWA Classes 5-13 plus any larger vehicles as defined by a state. 24

Table 4. Recommended Core Freeway Performance Measures (continued) Performance Metric Definition Units Geographic Scale Time Scale Relationship to NTOC Measures Safety (Quality of Service) Total Crashes Freeway crashes as defined by the State, i.e., those for which a police accident report form is generated Number Not recommended by NTOC Fatal Crashes Freeway crashes as defined by the State, i.e., those for which a police accident report form is generated, where at least one fatality occurred Number Not recommended by NTOC Overall Crash Rate Total freeway crashes divided by freeway VMT for the time period considered Number per 100 million vehicle-miles All safety measures computed areawide; section level may be computed if multiple years are used All safety measures computed annually Not recommended by NTOC Fatality Crash Rate Total freeway fatal crashes divided by freeway VMT for the time period considered Number per 100 million vehicle-miles Not recommended by NTOC Secondary Crashes A police-reported crash that occurs in the presence of an earlier crashg Number Not recommended by NTOC Ride Quality (Quality of Service) Present Serviceability Rating (PSR) The general indicator of ride quality on pavement surfacesh (Internal scale) Section and areawide Annually Not recommended by NTOC International Roughness Index (IRI) Cumulative deviation from a smooth surface Inches per mile Section and areawide Annually Not recommended by NTOC Environment (Quality of Service) Nitrous Oxides (NOx) Emission Rate Modeled NOx attributable to freeways divided by freeway VMT Number Section and areawide Annually Not recommended by NTOC Volatile Organic Compound (VOC) Emission Rate Modeled VOC attributable to freeways divided by freeway VMT Number Section and areawide Annually Not recommended by NTOC Carbon Monoxide (CO) Emission Rate Modeled CO attributable to freeways divided by freeway VMT Number Section and areawide Annually Not recommended by NTOC Fuel Consumption per VMT Modeled gallons of fuel consumed on a freeway divided by freeway VMT Number Section and areawide Annually Not recommended by NTOC g. See text for discussion. h. See: http://www.fhwa.dot.gov/policy/1999cpr/ch_03/cpg03_2.htm. 25

Table 4. Recommended Core Freeway Performance Measures (continued) Performance Metric Definition Units Geographic Scale Time Scale Relationship to NTOC Measures Incident Characteristics (Activity-Based) No. of Incidents by Type and Extent of Blockage Self-explanatory Type: 1) crash; 2) vehicle breakdown; 3) spill; and 4) other. Blockage: Actual number of lanes blocked; separate code for shoulder blockage Section and areawide a.m./p.m. peak periods, daily Not recommended by NTOC Incident Durationi The time elapsed from the notification of an incident to when the last responder has left the incident scene Minutes (median) Section and areawide a.m./p.m. peak periods, daily Direct correspondence to NTOC measure Blockage Duration The time elapsed from the notification of an incident to when all evidence of the incident (including responders’ vehicles) has been removed from the travel lanes Minutes (median) Section and areawide a.m./p.m. peak periods, daily Not recommended by NTOC Lane-Hours Loss Due to Incidents The number of whole or partial freeway lanes blocked by the incident and its responders, multiplied by the number of hours the lanes are blocked Lane-hours Section and areawide a.m./p.m. peak periods, daily Not recommended by NTOC Work Zones (Activity-Based) No. of Work Zones by Type of Activity The underlying reason why the work zone was initiated: 1) resurfacing only; 2) RRR; 3) lane addition w/o interchanges; 4) lane additions w/interchanges; 5) minor cross- section; 6) grade flattening; 7) curve flattening; 8) bridge deck; 9) bridge superstructure; 10) bridge replacement; and 11) sign-related Number Section and areawide Daily Not recommended by NTOC Lane-Hours Lost Due to Work Zones The number of whole or partial freeway lanes blocked by the work zone, multiplied by the number of hours the lanes are blocked Lane-hours Section and areawide a.m./p.m. peak periods; midday; night; daily Not recommended by NTOC Average Work Zone Duration by Type of Activity The elapsed time that work zone activities are in effect Hours Section and areawide Daily Not recommended by NTOC Lane-Miles Lost Due to Work Zones The number of whole or partial freeway lanes blocked by the work zone, multiplied by the length of the work zone Lane-miles Section and areawide a.m./p.m. peak periods, daily Not recommended by NTOC i. Since in many cases the actual time the incident occurred is unknown, the notification time is used to indicate the official “start” of the incident. On most urban freeways, through the use of cell phones by the public, the time between when the incident occurs and when it is first reported is very small. 26

Table 4. Recommended Core Freeway Performance Measures (continued) Performance Metric Definition Units Geographic Scale Time Scale Relationship to NTOC Measures Weather (Activity-Based) Extent of highways affected by snow or ice Highway centerline mileage under the influence of uncleared snow or ice multiplied by the length of time of the influence Centerline-Mile-Hours Section and areawide Daily Not recommended by NTOC Extent of highways affected by rain Highway centerline mileage under the influence of rain multiplied by the length of time of the influence Centerline-Mile-Hours Section and areawide Daily Not recommended by NTOC Extent of highways affected by fog Highway centerline mileage under the influence of fog multiplied by the length of time of the influence Centerline-Mile-Hours Section and areawide Daily Not recommended by NTOC Operational Efficiency (Activity-Based) Percent Freeway Directional Miles with (traffic sensors, surveillance cameras, DMS, service patrol coverage) Percentage (xxx.x%) Section and areawide Annually Not recommended by NTOC Percent of Equipment (DMS, surveillance cameras, traffic sensors, ramp meters, RWIS) in “Good” or Better Condition Percentage (xxx.x%) Section and areawide Annually Not recommended by NTOC Percent of total device- days out-of-service (by type of device) One measure for each type of equipment deployed in an area Percentage (xxx.x%) Section and areawide Annually Not recommended by NTOC Service patrol assists Self-explanatory Number Section and areawide Annually Not recommended by NTOC 27

Table 5. Supplemental Freeway Performance Measures Performance Measure Definition Units Geographic Scale Time Scale Relationship to NTOC Measures Average Congestion Conditions (Quality of Service) Bottleneck (“Recurring”) Delay Delay that is attributable to bottlenecksj Vehicle-hours Section and areawide Peak hour, a.m./p.m. peak periods, midday, daily Incident Delay Delay that is attributable to traffic incidents Vehicle-hours Section and areawide Peak hour, a.m./p.m. peak periods, midday, daily Work Zone Delay Delay that is attributable to work zones Vehicle-hours Section and areawide Peak hour, a.m./p.m. peak periods, midday, daily Weather Delay Delay that is attributable to inclement weather Vehicle-hours Section and areawide Peak hour, a.m./p.m. peak periods, midday, daily NTOC defines two categories: recurring and nonrecurring; see text for discussion Ramp delay (where ramp metering exists) Delay that occurs at ramp meters Vehicle-hours Individual ramps and section as a minimum Peak hour, a.m./p.m. peak periods Abnormal Volume-Related Delay Delay caused by abnormal high volumesk Vehicle-hours Section and areawide Peak hour, a.m./p.m. peak periods, midday, daily Volume-to-capacity ratio The ratio of the demand volume attempting to use a short segment of freeway divided by the freeway’s capacity, as defined by the HCM None Bottleneck locations only (freeway interchanges, lane-drops, bridges) Peak-hour volume/peak- hour capacity Peak-period volume/peak- period capacity Not recommended by NTOC Traffic Demand Indicator Ratio of actual traffic demand (volume) to average traffic demandl None Section and areawide Peak+Shoulder Periods Not recommended by NTOC Delay per Capita Total freeway delay divided by the population of the area being studied Vehicle-hours per person Areawide and statewide Peak hour, a.m./p.m. peak periods; daily Not recommended by NTOC Average speeds by hour of the day (used primarily as an indicator of air quality) The miles traveled by vehicles over a distance divided by the time it took to travel that distance (space mean speed)m Miles per hour Section and areawide Peak hour, a.m./p.m. peak periods; daily NTOC defines “speed” as the time mean speed Reliability (Quality of Service) Reliability: Failure Measure No. 1 Percent of trips (section or O/D) with space mean speeds <= 50 mph Percent Section and areawide Peak hour, a.m./p.m. peak periods, midday, daily Not recommended by NTOC Reliability: Failure Measure No. 2 Percent of trips (section or O/D) with space mean speeds <= 30 mph Percent Section and areawide Peak hour, a.m./p.m. peak periods, midday, daily Not recommended by NTOC Planning Time Index 95th percentile travel time divided by the free flow travel time N/A Section and areawide Peak hour, a.m./p.m. peak periods, midday, daily Not recommended by NTOC Throughput (Quality of Service) VMT per capita Freeway VMT divided by the population of the study area N/A Section and areawide Peak hour, a.m./p.m. peak periods, midday, daily Not recommended by NTOC j. Delay is the excess travel time used on a trip, facility, or freeway segment beyond what would occur under ideal conditions; see text for a discussion of “ideal” conditions. k. May be due to either special events or normal variation due to daily/seasonal fluctuations in demand. l. See text for a more complete explanation. m. Although the Guidebook calls this space mean speed, depending on how the measurements are taken, it may be a “synthesized” space mean speed. That is, if the basic measurements are from point detectors, theoretically speaking, it is closer to being a time mean speed. See Section 9.0 for more discussion. 28

Table 5. Supplemental Freeway Performance Measures (continued) Performance Measure Definition Units Geographic Scale Time Scale Relationship to NTOC Measures Customer Satisfaction All customer satisfaction measures apply areawide or statewide (Quality of Service)n All customer satisfaction measures developed every 1-3 years Biggest concern about transportationo Defined by survey question Percent Not recommended by NTOC Most important thing the Department could do to improve congestionp Defined by survey question Percent Not recommended by NTOC Usage rates and percent of favorable response to broadcast video images Defined by survey question Percent Not recommended by NTOC Usage rates and percent of favorable response to traveler information about 1) congestion and 2) work zones Defined by survey question Percent Not recommended by NTOC Usage rates and percent of favorable response to DMS messages Defined by survey question Percent Not recommended by NTOC Usage rates and percent of favorable response to service patrols Defined by survey question Percent Not recommended by NTOC Percent of favorable response to work zone management Defined by survey question Not recommended by NTOC Percent of favorable response to freeway planning process Defined by survey question Percent Not recommended by NTOC Percent of favorable response with completed projects Defined by survey question Percent Not recommended by NTOC Percent of favorable response with air quality Defined by survey question Percent Not recommended by NTOC Percent of favorable response with long- distance travel Defined by survey question Percent Not recommended by NTOC Percent of favorable response with pavement condition Defined by survey question Percent Not recommended by NTOC Percent of favorable response with highway safety (how safe it is to travel?) Defined by survey question Percent Not recommended by NTOC Percent of favorable response with amount of salt used on main rural highways Defined by survey question Percent Not recommended by NTOC Percent of favorable response with environmental aspects of road construction Defined by survey question Percent Not recommended by NTOC Percent of favorable response with environmental aspects of road planning and design Defined by survey question Percent Not recommended by NTOC n. Usually included in statewide surveys of public’s attitudes towards transportation and service provided; also may be done at the local level. o. 1) Congestion, 2) poor road and bridge condition, 3) highway crashes, 4) transit not available. p. 1) Build more roads, 2) clear incidents faster, 3) reduce time that work zones are needed, 4) more effective snow removal, 5) better inform travelers about congestion they will encounter on their trips. 29

Table 5. Supplemental Freeway Performance Measures (continued) Performance Measure Definition Units Geographic Scale Time Scale Relationship to NTOC Measures Customer Satisfaction All customer satisfaction measures apply areawide or statewide (Quality of Service)q All customer satisfaction measures developed every 1-3 years Safety (Quality of Service) All safety data defined by state police accident report (PAR) All safety measures computed areawide; section level may be computed if multiple years are used All safety measures computed annually No safety measures recommended by NTOC Number of fatal, injury, and PDO crashes – total and by: 1) type of collision; 2) time of day; 3) relation to ramps; and 4) ”first harmful event” (fixed object, rollover, etc.) Number; distribution percents within each category High-crash locationsr Specific locations or short segments of freeway Alcohol-involved crashes (fatal, injury, total) Number Commercial vehicle crashes (total and hazmat involved) Number Commercial vehicle crash rate Total number of commercial vehicle crashes divided by commercial vehicle VMT Rate Crashes where speed was a contributing factor Number Total Work Zone Crashes, Injuries, and Fatalities Number Total Weather-Related Crashes, Injuries, and Fatalities Number Incident Management (Activity-Based) First Responder Response Time Time difference between when the incident was first detected by an agency and the on-scene arrival of the first responder Minutes Section and areawide a.m./p.m. peak periods, daily Not recommended by NTOC Notification Time Time difference between when the incident was first detected to when the last agency needed to respond to the incident was notified Minutes Section and areawide a.m./p.m. peak periods, daily Not recommended by NTOC Total Response Time Time difference between when the incident was first detected by an agency and the on-scene arrival of the last responder Minutes Section and areawide a.m./p.m. peak periods, daily Not recommended by NTOC Clearance time Time difference between when the first responder arrived on the scene and blockage of a travel lane is removed Minutes Section and areawide a.m./p.m. peak periods, daily Not recommended by NTOC On-Scene Time Time difference between when the first responder arrives and the last responder leaves an incident scene; also may be computed for individual responders Minutes Section and areawide Not recommended by NTOC q. Usually included in statewide surveys of public’s attitudes towards transportation and service provided; also may be done at the local level. r. Most states have procedures for identifying high-crash locations. Additional guidance may be available through software packages such as FHWA’s SafetyAnalyst. 30

Table 5. Supplemental Freeway Performance Measures (continued) Performance Measure Definition Units Geographic Scale Time Scale Relationship to NTOC Measures Customer Satisfaction All customer satisfaction measures apply areawide or statewide (Quality of Service)s All customer satisfaction measures developed every 1-3 years Linger Time Time difference between when the blockage of a travel lane is removed and the last responder leaves the incident scene Minutes Section and areawide Not recommended by NTOC Traffic Influence Time Time between when an incident was first detected and the last responder leaves the incident scene Minutes Section and areawide a.m./p.m. peak periods, daily Not recommended by NTOC Detection Method (citizens, police, other agencies) per month The method which incidents are detected or reported Locally defined Section and areawide a.m./p.m. peak periods, daily Not recommended by NTOC Service patrol assists (total and by incident type) Section and areawide a.m./p.m. peak periods, daily Not recommended by NTOC Work Zones (Activity-Based) Traffic volume passing through work zones Self-explanatory; AADT estimates may be used in place of actual counts Vehicles Section and areawide Daily No work measures recommended by NTOC Average Time Between Rehabilitation Activities by Type of Activity Type of activity: 1) resurfacing only; 2) RRR; 3) lane addition w/o interchanges; 4) lane additions w/interchanges; 5) minor cross-section; 6) grade flattening; 7) curve flattening; 8) bridge deck; 9) bridge superstructure; 10) bridge replacement; and 11) sign-related Months Areawide N/A Average Number of Days Projects Completed Late “Late” is any time after the scheduled completion Days Areawide N/A Ratio of Inactive Days to Active Days “Active” is when some work zone activity was performed during a day N/A Areawide Annually Crashes per lane-mile lost Work zone crashes divided by the number of lanes lost N/A Section, areawide, and statewide Annually Average Work Zone Duration by Work Zone Type by Lanes Lost Time length of work zone activities by their severity in terms of traffic impact; Lanes lost = 0, 1, 2, 3, 4+ Hours Areawide Annually Average Number of Days That a Contract Work Zone is Active “Active” is when some work zone activity was performed during a day Days Areawide Annually Weather (Activity-Based) Number of incident responses during weather- related events Self-explanatory Number Areawide Monthly and annually Lane-miles and freeway miles officially closed due to weather or flooding Self-explanatory Lane-miles Areawide Monthly and annually Number of freeways with reduced speed limits by MP3 reductions Self-explanatory Number Areawide Monthly and annually Number of freeway ramps closed due to weather by weather event Self-explanatory Number Areawide s. Usually included in statewide surveys of public’s attitudes towards transportation and service provided; also may be done at the local level. 31

Table 5. Supplemental Freeway Performance Measures (continued) Performance Measure Definition Units Geographic Scale Time Scale Relationship to NTOC Measures Customer Satisfaction All customer satisfaction measures apply areawide or statewide (Quality of Service)t All customer satisfaction measures developed every 1-3 years Weather (Activity-Based) Time between 2 inches of snow accumulation and plowing (clearance) Self-explanatory Minutes Areawide (lane-mile weighted) Annually Lane-miles pretreated with chemical snow/ ice control Self-explanatory Lane-miles Areawide Annually Lane-miles pretreated with chemical snow/ ice control that experienced snow or ice conditions Self-explanatory Lane-miles Areawide Annually Weather event VMT ratio VMT during event: VMT for recent same DOW N/A Areawide Annually Weather event delay ratio Delay during event: Delay for recent same DOW N/A Areawide Annually Delay per lane-mile affected by major weather events Self-explanatory Rate Areawide Annually Crashes per lane-mile affected by major weather events Self-explanatory Rate Areawide Annually Operational Efficiency (Activity-Based)u Service patrol vehicles in operation per shift Self-explanatory Number Section and areawide User-specified Percent freeway miles with (electronic data collection, surveillance cameras, DMS, service patrol coverage) Self-explanatory Percent Areawide User-specified Number of messages placed on DMSs Self-explanatory Number Section and areawide User-specified Individuals receiving traveler information by source (511, other direct means) Self-explanatory Number Section and areawide User-specified Percent of equipment (DMS, surveillance cameras, sensors, ramp meters, RWIS) in “good” or better condition Self-explanatory Percent Section and areawide User-specified Percent of total device-days out-of-service (by type of device) Self-explanatory Percent Section and areawide User-specified Incident detection method Self-explanatory Number Areawide User-specified No. devices exceeding design life Self-explanatory Number Section and areawide User-specified MTBF for field equipment (by type of device) Self-explanatory Days Section and areawide User-specified Number of freeway miles instrumented with traffic data collection devices Self-explanatory; directional miles Miles Areawide User-specified Freeway construction projects completed within 30 days of scheduled completion Self-explanatory Number Areawide User-specified t. Usually included in statewide surveys of public’s attitudes towards transportation and service provided; also may be done at the local level. u. A multitude of other operational efficiency measures resides in asset management information and performance measurement systems. 32

CHAPTER 3 INTERPRETATION, APPRAISAL, AND APPLICATIONS The Guidebook takes a comprehensive view of freeway performance measurement and monitoring. Its guidance can be applied across the spectrum of applications, as shown in Figure 1, to the extent that the same per- formance measures should be used across all applications. By maintaining this consistency, linkage to the goals and objectives of the agency is achieved. Figure 1. The Same Performance Measures Should be Carried Across Applications Spanning the Entire Time Horizon Time Horizon Past Performance Real-Time Short-Range Mid-Range Long-Range Applications – Trend Analysis Current versus Historical Performance Evaluation of Project Alternatives Preliminary Design; TIP Estimates Long-Range Plan (Systemwide Performance of Alternative Plans) The Guidebook touches on all aspects of the project time horizon shown in Figure 1, but the emphasis is clearly on trend analysis. The reason for this is twofold: 1. During the benchmarking interviews, this was revealed to be the area that agencies were struggling with the most. Many agencies had received directives from legislatures or top DOT management to institute performance measurement programs, yet there were little or no precedents to follow. Further, agencies were use to applying performance measures for conducting analyses and evaluations, although they were usually called “measures of effectiveness.” 2. NCHRP Project 7-15, which was still in development at the time of this report, was focusing on how to develop and apply congestion/mobility performance measures for short-, mid-, and long-range applications. The research teams coordinated their work so as to avoid duplication and concentrate resources where they would do the most good. 33

Using the same performance measures across the time horizon is not as difficult as it sounds. Most models used in transportation analyses produce as output some variant of travel time for measuring congestion/mobility performance. These outputs can be easily transformed to the recommended performance measures recommended here, although it may require post-processing of model outputs. In terms of the categories of performance that should be measured, congestion/mobility is the category that receives the most attention in the Guidebook. As mentioned previously, the other areas of performance are generally well-covered elsewhere but there has been little consensus in the transportation profession about how to measure and report congestion. The Guidebook goes into great detail (which will not be repeated here) on how to measure congestion, what data and analytic methods should be used to develop them, how they should be reported and communicated, and how they may be used in the decision-making process. 34

CHAPTER 4 CONCLUSIONS AND SUGGESTED RESEARCH 4.1 Conclusions The research conducted for this study came in two parts: 1) an assessment of current practice and the determining the unmet needs and 2) development of data processing, analytic techniques, presentation methods, and guidelines for applying freeway performance measures to decision-making. From this work, several broad-ranging conclusions can be reached. • Performance measurement of all kinds (not just that related to freeways) is growing in impor- tance and is becoming institutionalized within transportation agencies. Transportation agencies are increasingly adopting a customer focus in their activities, i.e., a more “business-like” approach to doing business. While the motivations of private firms and public agencies are different, many of the tools and principles have equal merit in both worlds. Performance measurement, which has been used in the private sector for some time, is one of these tools. Additionally, practitioners are recognizing that using performance measures allows them to improve their functions in several ways: o Deficiencies are identified with better precision and improvement strategies can be better tailored to the deficiencies. o Public relations is enhanced; just by the fact that statistics are reported, it provides a view to the public that professionals understand the nature of the problems. o In the congestion/mobility realm, having information on the outcomes of investments provides high-level input to transportation programming decisions. • Collection of quality data is required to build the foundation of a freeway performance measure- ment program. There’s no getting around the fact that a comprehensive freeway performance measurement program needs a large amount of detailed and accurate data to be effective. Much of these data already are being collected by ITS deployments in some form. However, data quality has proven to be a major issue, and some of the required data will need new data collection programs. • Congestion and mobility performance measurement on freeways has been the largest gap in knowledge. Based on the benchmarking interviews and the research team’s experiences on other 35

projects, the biggest need for support is in the area of congestion and mobility. This support is likely to be required into the future as agencies engage more actively in freeway performance measurement. Customer satisfaction is a relatively immature area as well, but in general its implications are not necessarily focused on freeways. • Travel time reliability is being recognized as characteristic of congestion that is on equal footing with average congestion levels. With regard to congestion/mobility performance measurement, sub- stantial emphasis is placed on the concept of travel time reliability, which is emerging as a key issue for operators and planners. Travel time reliability is becoming a major theme for operators because it relates directly to the events that cause travel conditions to vary from day to day. As planners take on a larger role in operations (“planning for operations”), they too are recognizing that just dealing with physical capacity-related congestion is only part of the congestion puzzle. The Guidebook notes there are two closely related views of what constitutes travel time reliability, namely, variability in travel times and number of “failures” to meet established travel time thresholds. Both can be related to the underlying distribution (or history) of travel conditions experienced by users, leading to a general defi- nition of travel time reliability: Travel time reliability is defined as the level of consistency in travel conditions over time, and is measured by describing the distribution of travel times that occur over a substantial period of time. Traditionally, mobility/congestion has been defined in terms of “average” or “typical” condi- tions (e.g., HCM applications). One way to look at the measurement philosophy proposed in this report is that “mobility” is analogous to average or typical travel time, while “reliability” is analogous to variability or inconsistency in travel times. • Measuring average congestion and overall travel time reliability is only the start of understanding congestion and crafting strategies to deal with it. Quality of service (outcome) measures are extremely important for agencies because they represent the “bottom-line” for their cus- tomers (travelers). However, measuring total delay and travel time reliability is really just the starting point for freeway performance measurement – deciphering what causes travel times to be unreliable is the next step. The transportation profession has traditionally used the terms recurring and nonrecurring 36

congestion to get at this issue, and the Guidebook acknowledges this, but these terms only supply a limited amount of additional information. Instead, the Guidebook prefers to decompose congestion into “seven sources”: physical bottlenecks, traffic incidents, inclement weather, work zones, traffic control devices, special events, and variable demand.3 These sources interact in complex ways to pro- duce total congestion. Quantifying how much congestion is due to these sources is still problematic, although the Guidebook provides interim advice on how to do it. • Quality of service (outcome) and activity-based (output) performance measures must be linked together and tied into the mission of the transportation agency. Development and reporting trends in performance measures at one level of detail usually begs the question, “why did this happen?” Practitioners need to be willing to dig deeper into this question by constructing lower levels of per- formance measures (activity-based) that are linked to the upper levels (quality of service). Continuing the example of congestion by source, each source has its own characteristics and treatments that should be tracked. For example, traffic incident-related delay is determined by the nature of traffic incidents and the effectiveness of incident management strategies. Tracking these characteristics and activities indicates what aspects of incident management need to be improved, and provides answers to outside requests to explain the situation. • Some experimentation and deviation from the Guidebook on the part of practitioners is war- ranted. Because local issues are all slightly different, the Guidebook is ultimately a reference rather than a prescriptive document. Further, because the state of the art in reporting performance measures is still not mature, practitioners should be free to try new forms of graphics and presentation techniques. 4.2 Suggested Research Benchmarks and Levels of Service for Reliability and Event Performance Measures Use of performance measures is usually associated with setting of performance targets for the measures. However, what is considered to be “good,” “acceptable,” or “poor” for individual performance measures currently is highly subjective. Also, should performance targets be set so as to be achievable, or should they aspire to be lofty “stretch” goals (e.g., zero fatalities)? There is a need to provide guidance to practitioners as to what the performance targets should be, or at least a process for setting them should be defined. For key quality of service (outcome) 37

measures of interest such as congestion level and reliability as well as key activity-based (output) measures such as incident duration (and other incident component times), work zone duration, etc., benchmarks should be established. These can be set in a manner similar to the “level of service” concept used in the HCM. Consistent and Comprehensive Event Data Collection and Use The collection of data on events (incidents, weather, work zones, special events), especially at traffic management centers (TMC) is very scattered. The data that does get saved is a function of the features that were built into the TMC operating software, and these features do not follow any standard practice. Sometimes the information doesn’t even exist in true database format – it’s just archived text messages. This research would: • Identify the data that needs to be collected at TMCs to capture event characteristics. Coordinate with current FHWA efforts on traffic incident management, the National Transportation Operations Coalition (NTOC), and existing ITS data dictionaries. Develop a “TMC Event Data Dictionary” that can be implemented by TMC software. • Identify the policies that need to be in place to ensure that TMC personnel capture data on all events under their purview. • Develop a prototype system for use by a TMC. This will have to be done in conjunction with a TMC software upgrade – the data capture system needs to be integrated with the existing software. Compatibility of Travel Estimation Techniques Travel times are the basis for the congestion/mobility performance metrics recommended in the current research, especially reliability metrics. Travel times can be estimated directly by measuring the passage on individual vehicles over time, synthesized from point detection of spot speeds, or computed with models. Because there is no universal coverage of a single method in urban areas, it is likely that multiple methods will be used in the foreseeable future. The question is: do these methods produce compatible results so that corridor performance can be compared and areawide statistics can be developed by combining them. The issue will become more significant as cell phone and vehicle-based monitoring become more pervasive. This research would collect travel time data at a detailed level using closely spaced floating cars traveling in a heavily instrumented freeway corridor, and then compare the travel time estimates to statistics developed from the roadway-based detectors and models (e.g., HCM). 38

Application of Recommendations in the Field: Use of Performance Measures to Guide Operations and Planning Investment Decisions Many agencies are developing performance measures and producing reports on trends in performance (e.g., dashboards, quarterly reports like WSDOT’S Gray Book.). The issues now become: • How can changes in performance measures (trends) or deviations from goal-based benchmarks lead to changes in investments and policies? • Can changes in performance measures be keyed to specific actions (e.g., a shift in the percentage of congestion due to a nonrecurring event type, such as incidents)? • How detailed do performance need to be to allow this linking to types of actions (e.g., do we need to measure the activities of personnel at a very detailed level)? • How are reliability metrics used? Are areawide or corridor levels useful in reporting performance? • How does tracking recent performance trends modify: o Short- and long-range transportation plans? o Daily operating policies? o Annual capital budgeting for operations? • How are congestion/mobility performance measures used in conjunction with asset management and safety measures to prioritize projects? Do they help operations compete on a “level playing field”? Estimating Reliability and Congestion Source Performance Measures There is a need for operators and planners to make estimates of reliability where continuous data do not exist. A current FHWA effort is considering how existing models can be used to estimate reliability, but there is still lacking an empirical basis for these applications, especially when trying to develop current year estimates of reliability. This study would use continuous ITS data on freeways to relate reliability levels to easily obtainable data that are known to influence the characteristics of events, the cause of unreliable travel. These include: • Incident/crash rates; • Shoulder presence; • Number of lanes; • Base congestion level (AADT/C, V/C); 39

• Bottleneck presence/severity; and • Average weather conditions over a year. Additionally, improved methods for calculating delay by source of congestion are needed. The Guidebook promotes the use of delay by source and documents some existing methods that may be considered “interim,” yet definitive methods for calculating the delay due to the seven sources of congestion from empirical data do not exist. Traffic Data Quality for Real-Time and Performance Monitoring Applications If ITS-derived traffic data are to be used to detect the often subtle changes in traffic conditions due to implementing operations strategies, it is crucial that the error bounds on the data are small enough to allow accurate comparisons. High-quality data also is essential for advanced operations strategies such as providing travel time estimates to highway users. Also, if the private sector becomes a major supplier of data in the future (via cell phones, in-vehicle tracking, or roadway-based sensors), what performance targets for data quality should be imposed? A related issue is if travel time data is provided via the private sector, is there still a need for agencies to maintain sensors as a check on data quality, backup in case of outages or private firms disappearing from the market, and to provide volume data for integrated corridor management, evacuation management, and performance measures? This research would: • Compile best agency practices in maintaining quality sensor data at the field level, including accep- tance, calibration, routine maintenance, and communications. Costs for maintaining data at different levels of quality also will be compiled. • Recommend testing procedures for certifying that data provided by the private sector meet preset quality targets. 40

REFERENCES 1 Highway Capacity Manual, Transportation Research Board, National Research Council, Washington, D.C., 2000. 2 Schrank, David, and Lomax, Timothy, The 2004 Urban Mobility Report, Texas Transportation Institute, September 2004. 3 Cambridge Systematics, Inc. et al, “Providing a Highway System with Reliable Travel Times”, NCHRP 20-58(3) Contractor’s Report, Transportation Research Board, National Research Council, Washington, D.C., September 2003. 41

APPENDIX A RESULTS OF BENCHMARKING INTERVIEWS A-1

Table A.1 Agencies Participating in the Benchmarking Interviews Metro Area Reasons for Selection Agencies Interviewed Multiple Agency Interviews 1. Minneapolis-St. Paul, Minnesota Mn/DOT has been collecting and using performance and efficiency data for many years – they were an early leader in performance measures, earning themselves the title of “Land of 10,000 performance measures.” Mn/DOT Ops aggressive at using operations data and information for decision-making. Long history of active freeway management and data collection. Location of high-profile public debate on operational policy (ramp metering), and location of significant transit technology test (buses on narrow shoulders) Metro District Operations, Mn/DOT Center for Transportation Studies, University of Minnesota Metro Council 2. Seattle, Washington WSDOT very actively pursuing performance measures as a means of selling O&M program. Very active public reporting process, active experimentation in performance measure development., and freeway performance is a key subject in proposed ballot initiatives. WSDOT HQ Traffic Office WSDOT NW Region WSDOT HQ Strategic Planning and Programming Puget Sound Regional Council 3. Hampton Roads, Virginia Field Operational Test aimed at developing a comprehensive archive data management system (ADMS) being conducted here. Multiple stakeholder groups actively engaged in use of the archive; performance measurement at different levels a major thrust. CS is leading the evaluation. Hampton Roads STC, VDOT Hampton Roads Planning District Commission 4. Milwaukee, Wisconsin WisDOT embarked on aggressive use of information for operations and planning WisDOT District 2 Operations WisDOT Central Office University of Wisconsin-Madison WisDOT District 2 Planning 5. Phoenix, Arizona MAG beginning a performance measurement program (planning); Maricopa County DOT also interested in warehousing data and providing performance reports. MAG has recently provided significant funding for improved data collection for performance reporting. Arizona DOT Traffic Operations Center currently monitors 50% of freeways in Phoenix and Tucson metro areas (100 centerline miles). Real-time information is provided to web site and 511. A quarterly report is published internally on freeway congestion in the Phoenix area and how well departmental objectives are being met. ADOT currently spends $2.25 million a year on its Traffic Operations Center plus $1.25 million a year on detector maintenance. ADOT/Intermodal Transportation Division ADOT/Transportation Planning Division Maricopa Association of Governments 6. Los Angeles, California Caltrans HQ actively involved in performance measurement for both Operations and planning activities. Caltrans currently is funding development of an arterial monitoring system to supplement the freeway monitoring system. Real-time freeway congestion information currently is posted to the web. Caltrans, Freeway Operations, District 7 Southern California Association of Governments Caltrans, Planning Operations Interviews Only 7. Portland, Oregon Portland has a very active freeway management effort, an ongoing performance measure development effort with Portland State University, and considerable public pressure to improve roadway performance. See Notea 8. Houston, Texas TxDOT/TRANSTAR currently prepares an annual performance report. Houston TRANSTAR 9. San Antonio, Texas TxDOT Operations has shown interest in better exploiting their data resources. See Noteb 10. Washington, D.C. CHART very active in incident management performance measures; expansion of Hampton Roads ADMS being planned for Northern Virginia CHART (Maryland) VDOT Northern Virginia District 11. Atlanta, Georgia NaviGAtor actively using incident management performance measures. Business Plan being developed tied to multiple performance measures GDOT, Office of Traffic Operations a No formal interviews were conducted as part of NCHRP 3-68. Rather, the team relied on other work conducted by TTI on performance measures. As part of this effort ODOT assembled a Technical Advisory Committee (TAC) made up of individuals from ODOT sections of traffic management, transportation planning and analysis, transportation data, traffic operations, and internal audit/performance measures. The TAC also included individuals from the metropolitan planning organization (MPO) in Portland (Metro) and the Eugene/Springfield area (Lane Council of Governments), academia, and the local Federal Highway Administration (FHWA) office. b Initial conversations with TransGuide indicated that they currently are not using performance measures nor are they planning on developing them in the near future. TransGuide does have an extensive sensor system (485 lane-miles) and a formal incident management program from which detailed performance measures could be developed, however. A-2

Table A.2 Reasons for Undertaking Performance Measurement Metro Area General Background Information Motivation for Conducting Performance Measurement 1. Minneapolis-St. Paul, Minnesota TMC confirms traffic incidents with nearly 285 Closed-circuit TV (CCTV) cameras posted along 210 miles of metro-area freeway. Information on incident location and resulting traffic back-ups are relayed to travelers via Traffic Radio, Traffic TV, various Internet sites and a telephone service. The RTMC provides traffic information to local radio and television traffic reporters as well. Travelers also are alerted to traffic problems via 70 electronic message signs placed throughout the freeway system. TMC staff also operates 430 ramp meters and 4,000 loop detectors (traffic sensors). In an annual Departmental Results report, Mn/DOT tracks a number of performance measures statewide. Performance measures have therefore become part of an institutional reporting process. 2. Seattle, Washington WSDOT has a very active freeway management program, including: freeway ramp meters throughout most of the instrumented freeway system; an active, roving, service patrol program; a coordinated, multiagency incident management program, including designated WSDOT incident management staff; a very active traveler information system, including a 511 call-in line, a heavily used Web site that displays a congestion map and access to both still images and streaming video. WSDOT has adopted “WSDOT’s Congestion Measurement Principles” which are as follows: • Use real-time measurements rather than computer models whenever possible; • Measure congestion due to incidents as distinct from congestion due to inadequate capacity; • Show whether reducing congestion from incidents will improve travel time reliability; • Demonstrate both long-term trends and short- to intermediate-term results; • Communicate about possible congestion fixes by using an “apples to apples” comparison with the current situation; and • Use plain English to describe measurements. Original request from state legislature resulted in the annual performance report known as Measures, Markers, and Mileposts, the “Departmental accountability” report published by WSDOT each quarter to inform the legislature and public about how the Department is responding to public direction and spending taxpayer resources. Agencies now using performance measures as part of everyday practice to help make informed decisions. 3. Hampton Roads, Virginia The Smart Traffic Center (STC) is the Virginia Department of Transportation’s (VDOT) high-tech, customer service approach to regional freeway traffic management and communications. The Freeway Traffic Management System installed at STC consists of an extensive computer controlled, fiberoptic-based communications and control network installed along 31 miles of the area freeways (I-64, I-264, I-564, and I-664), 80 closed circuit television cameras plus access to 36 additional cameras in the tunnels and bridges, over 85 dynamic message signs and over 1,050 vehicle detectors strategically positioned across the entire Hampton Roads region, Wide-Area Highway Advisory Radio System (HARS), and Freeway Incident Response Teams (FIRT) patrolling over 70 miles of interstate in the region. The Smart Travel Lab (STL) at UVA is responsible for gathering and archiving performance data from the region’s loop, radar, and acoustic detectors. The STL does not report travel time and speed performance measures on a regular basis, but uses the Hampton Roads Smart Traffic Center loop data for research. From the research, more direct use of performance measures in day-to-day operations is hoped to be achieved. Output measures are more developed and in use than outcome measures. A-3

Table A.2 Reasons for Undertaking Performance Measurement (continued) Metro Area General Background Information Motivation for Conducting Performance Measurement 4. Milwaukee, Wisconsin The TOC archives many different types of operations data. As part of planned enhancements to their data archiving system, they plan to include a performance reporting “module” in their new data warehouse. A formal performance measurement program, the Freeway System Operational Assessment (FSOA) program, bas been instituted to provide better information to operators, public officials, and travelers. The impetus for FSOA came from the MONITOR traffic operations center, where WisDOT engineers were dealing with operational problems created by a “project” mentality in the planning and project development process. FSOA was created to provide a comprehensive, systemwide assessment of the safety and operational performance of all freeways in the Waukesha District, and to provide a framework in which geometric and/or operational improvement projects could be considered in the current project development process. The impetus for developing an operations performance monitoring process is two- fold: 1) The TOC wants to communicate the benefits of operations to WisDOT managers/administration as well as other nontechnical leaders and elected officials; 2) The TOC already has significant archived data resources that could be used, and there was an opportunity to develop this capability as part of planned enhancements to their data archiving system. Data from the MetaManager system drives many project development decisions. Thus, the operations group would like to develop the traffic analogy to MetaManager, which would essentially be a freeway performance reporting system based upon archived traffic operations data. 5. Phoenix, Arizona There are approximately 100 miles of High-Occupancy Vehicle (HOV) lanes in the area. The HOV lanes are restricted during peak traffic hours between 6:00 and 9:00 a.m. and 3:00 and 7:00 p.m. During these hours, travel on the HOV lanes is limited to vehicles with two or more occupants. Vehicles travel over 22.5 million miles on Phoenix’s freeway system everyday according to the CY2002 Highway Performance Monitoring System (HPMS) Report. This volume translates to approximately 8.22 billion annual VMT on the 189 miles of freeway. As a result, recurring bottlenecks and congested corridors are significant problems in the area. The Maricopa Association of Governments has identified 16 congested segments in the preliminary draft working paper of the MAG Regional Freeway Bottleneck Study. A freeway management system (FMS) that uses intelligent transportation technologies to collect freeway data and to monitor freeway conditions to optimize traffic flow covers approximately one-half of the freeway system in the Phoenix metropolitan area. TTG plans to extend the coverage area in the future. The original impetus was to support ADOT’s Strategic Action Plan, which is performance-based. Performance measures are at the core of this effort. Agencies are discovering uses for performance measures beyond fulfilling the requirements of the Strategic Action Plan. A-4

Table A.2 Reasons for Undertaking Performance Measurement (continued) Metro Area General Background Information Motivation for Conducting Performance Measurement 6. Los Angeles, California The Division of Operations of Caltrans 7 is responsible for constructing and maintaining all interstate and state highways in the Greater Los Angeles Area. It has developed an Advanced Traffic Management System (ATMS), which “integrates recurrent/nonrecurrent incident detection, verification, incident response, planned events of freeway management, and field element operational control.” ATMS uses electronic devices, such as loop detectors, to collect freeway performance data. The information collected by ATMS is fed to the District’s Transportation Management Center (TMC) and forms the basis for many performance measures. Caltrans aims to optimize traffic flow by managing existing traffic operations and anticipating future demands. Preliminary internal or “agency” performance measures for operations have been drafted. Two implicit policies in developing the measures are: 1) measures need to be monitored as well as forecast; and 2) measures should be modally and jurisdictionally blind whenever possible. In addition, Caltrans follows a system management philosophy. As such, freeway performance is not evaluated independent from the rest of the system. SCAG is required to produce a Regional Transportation Plan (RTP). As part of this effort, SCAG has developed goals and performance measures that aimed to evaluate the performance of the Plan. SCAG also takes a holistic, modally blind approach, where the goals and measures are applied to the entire transportation system. While freeway data are collected in such a way that they may be singled out for evaluation, they are not generally being reviewed apart from the whole system. 7. Portland, Oregon (Part of statewide study of performance measures). ODOT had a historical performance measurement process that was in need of updating to capture the effect of operational treatments on congestion/mobility. 8. Houston, Texas The hub for traffic operations in the greater Houston area is the Houston TranStar [Greater Houston Transportation and Emergency Management Center]. The Houston District of TxDOT is responsible for State maintained roadways within a six county area encompassing 5,948 miles with a population of approximately 4.9 million persons, 3.6 million of which are in Harris County. Houston TranStar generally covers about 235 centerline miles of the freeways located within Harris County only. Operations are typically focused on peak periods. There are 4 operators on duty in the Center during each weekday peak, 2 in the midday and one in the overnight and weekend periods. The Motorist Assistance Patrol and most other operational treatments operate only on weekdays. This is focused on the urban areas with very little interaction with rural areas around Houston. The performance measures are primarily derived to develop a “deficiency” report and to provide data for the changeable message signs and web site. An Annual Report also is prepared to describe the activities, actions and benefits from TranStar. The “deficiency” report identifies the technologies and their operating status. It is used to guide maintenance activity, especially in the case of dangerous potholes or inoperable signs or signals. There are some emerging programs, principally the Texas Metropolitan Mobility Plan, which may require more extensive performance reporting, but there is no Central Office-type mandate for measures or monitoring data. A-5

Table A.2 Reasons for Undertaking Performance Measurement (continued) Metro Area General Background Information Motivation for Conducting Performance Measurement 9. Washington, D.C. Maryland CHART currently has traffic sensors at 1.0- to 1.5-mile spacing along some sections of I-70, I-83, I-95, I-270, I-495, I-695, I-795, and U.S. 50. Many sections do not have any instrumentation. There also are video cameras in the same areas as the sensors. Sensor and camera coverage is shown on the CHART web site located at http://www.chart.state.md.us The traffic sensors provide volume, occupancy and speed. U. of Maryland is starting a process to archive this VOS data. CHART recently awarded a contract to a vendor to design reports for presenting the VOS and incident data. Currently, sensors are only used to update the real-time traffic maps on the web site and in the operations centers. Virginia The Northern Virginia (NOVA) District operates over 800 signalized intersections and automatically receives data from over 11,000 loop detectors. NOVA has both presence detectors (6’ x 40’ located at the stopbar) and system detectors (6’ x 6’ located 200-300 upstream from the stopbar). All loops are set up to detect volume, occupancy and speed. Maryland SHA and CHART participate in the Maryland DOT “Managing for Results” program. The program was initiated in 1999. It is linked to a Business Plan: goals, objectives, and performance measures, and strategies to achieve goals are linked. Virginia VDOT beginning a department-wide performance measurement program. 10. Atlanta, Georgia The NaviGAtor Program is the Georgia Department of Transportation’s (GDOT) high-tech, customer service approach to regional freeway traffic management and communications. The NaviGAtor system consists of an extensive computer controlled, fiberoptic-based communications and control network installed along 222 miles of the area freeways (I-20, I-75, I-85 and I-285 and GA 400), 319 closed circuit television cameras on the freeways plus 211 additional cameras on the arterial system managed by local jurisdictions, over 100 dynamic message signs and over 1,100 vehicle detectors strategically positioned across the Atlanta region, and Highway Emergency Response Operators (HERO) patrolling over 250 miles of interstate in 55 vehicles in the region. Development of an Operations Business Plan, which follows the “vision-goals- objectives-performance measures-targets-actions” sequence is driving the implementation of performance measures at NaviGAtor. A-6

Table A.3 Congestion/Mobility Performance Measures Under Consideration in Selected DOTs State Performance Measures Florida • Person-miles traveled; • Truck-miles traveled; • Vehicle-miles traveled; • Average speed; • Average delay per vehicle; • Average door-to-door trip time; • Variance of average travel time or speed(“Reliability”; not yet defined); • Vehicles per hour per lane during peak hour (“Maneuverability”); • Percent highway miles at LOS E or F; • Percent VMT at LOS E or F; • Vehicles per lane-mile (“Density”); and • Lane-mile-hours at LOS E or F (“Duration of Congestion”). Oregon • Roadway miles at V/C >0.70 during peak period; • Hours of delay from nonrecurring congestion; • Delay per incident; and • Hours of delay per system user (multimodal). A-7

Table A.4 Types of Performance Measures Used by Agencies Metro Area Operation Agencies Planning Agencies 1. Minneapolis-St. Paul, Minnesota • Average incident duration; • Percent of highway miles w/peak-period speeds <45 mph; • Travel Time Index; and • Travel times on selected segments, including mean, median, and 95th percentile. • HOV usage; • Roadway congestion index; • Percent of daily travel in congestion; • Percent of congested lane-miles in the peak period; • Percent of congested person-miles of travel; • Annual hours of delay; • Change in citizen’s time spent in delay; • Congestion impact on travel time; and • Travel Time Index. 2. Seattle, Washington Real-Time Operations TSMC staff use displays of three primary sets of information to determine the performance of the freeway system. These three displays include: • A “congestion map” based on vehicle-lane occupancy; • A set of computed travel times for 30 representative “trips” on the freeway system; and • A bank of television monitors displaying various CCTV images. Operations Planning/Output Measures Many output measures tracked; a sample includes: • The number of loop detectors deployed; • The number of loops functioning currently; • The percentage of loops functioning during a year; • The number of service patrol vehicles currently deployed; • The number of hours of service patrol efforts supplied by WSDOT; • The number of motorist assists provided by those service patrols by type of assistance provided; and • The number, duration, and severity of incidents by location (roadway segment) and type of incident. Operations Planning/Outcome Measures • The average vehicle volume by location and time of day and by type of facility (HOV/GP lane); • The average person volume by location and time of day and by type of facility; • The frequency of severe congestion (LOS f) by location; • The average travel time by corridor and major trip (O/D pairs); • The 95th percentile travel time by corridor and major trip (also reported as Buffer Time); • The number of very slow trips (half of FFS) that occur each year by time of day and major trip; • The amount of lost efficiency (speeds <45 mph) by location; and • The number of times that HOV lanes fail to meet adopted travel time performance standards. • The percentage of HOV lane violators observed by monitoring location The planning process incorporates all of the performance measures discussed above. In addition, the planning process uses the additional summary measures of: • Person-hours of travel; • Vehicle-hours of travel; • Person-hours of delay; • Vehicle-hours of delay; • Vehicle-miles of travel; and • Person-miles of travel. A-8

Table A.4 Types of Performance Measures Used by Agencies (continued) Metro Area Operation Agencies Planning Agencies 3. Hampton Roads, Virginia Outcome • Speed; • Volume; and • Occupancy. Output • Numbers of total incidents by type; • Number of vehicles involved; • Number of lanes blocked; • FIRT vehicle mileage; • TOC staff availability; • Numbers of variable message signs; • Average times for incident duration; • Average times for response; • Miles of STC coverage; • Numbers of messages placed on the signs; • Availability of STC field equipment; • Percent of surveillance devices responding; • Turnover rate; • Number of hours worked – Operators; • Number of hours worked – FIRT drivers; and • Number of miles driven – FIRT drivers by route. HCM LOS ranges, developed with a combination of models and roadway surveillance data; specifically: • Lane-miles operating at LOS F. 4. Milwaukee, Wisconsin Still under development, but several concepts are being applied: • Average congestion level (exact definition still being discussed); • Congestion duration (to quantify peak spreading); • Travel reliability (exact definition still being discussed); • Road safety index (exact definition still being discussed); • Measures for freeway service patrol (such as decrease in related incident congestion and secondary crashes); and • Impacts of work zones. • Benefit/Cost Ratio – Quantitative monetized benefits include: 1) travel time savings versus a no-build scenario; 2) value of improved traveler information; and 3) crash reduction. All benefits and costs are projected using 3 scenarios: best case, midrange, and worst case. • Environment – Qualitative assessment of changes. • Interconnection – Qualitative assessment of benefits for multimodal transportation and nondrives. • Partnerships – Qualitative assessment of likely sources of public support and/or opposition. Consideration of Other Performance Measures WisDOT has considered the use of travel time reliability in their models and may implement a monetized benefit value for reliability in the future. At this point, however, the Project Appraisal Report does not directly address travel time reliability. Freight-specific measures are not included, but freight is considered under the “Interconnection” qualitative assessment. A-9

Table A.4 Types of Performance Measures Used by Agencies (continued) Metro Area Operation Agencies Planning Agencies 5. Phoenix, Arizona Outcome Measures • Speed; • Average percent of freeways reaching LOS E or F on weekdays; • Traffic volumes/counts; and • Vehicle occupancy. Output Measures • Numbers of total incidents; • Numbers of Level 1 incidents; • TOC staff availability; • Numbers of variable message signs; • Average times for acknowledgment; • Average times for response; • Average times of closure; • Miles of FMS; • Number of traffic interchange signals connected to central system; • Number of hits to TTG’s traveler information web site; • Number of calls to TTG’s traveler information phone system; • Number of entries into HCRS; • Number of sites with HCRS; • Numbers of messages placed on the signs; • Availabilities of FMS; • Availability of HCRS; • Percent of surveillance devices responding; • Percent of time 511 available; • PC system availability; • Dollar of mandatory employee training; • Percent of mandatory supervisor training; • Percent of employee with >32 hours of training; • Years of ADOT experience; • Turnover rate; • Number of injuries; • People attending TOC tours; • Number of 511 comments; and • Percent of responses within 10 days to constituents. • Congestion Index (percent of posted speed); • Travel time; • Segment delay (seconds/mile); • Stop delay (<3mph) (seconds/mile); • Average speed (percent of posted speed); • Average speed (mph); • Average HOV lane speed (mph); • Running speed ((length/travel time)-stop delay); • Total volume; • HOV lane volume; • General purpose lane volume; • Percent peak-period truck volume; • Percent peak-period volume; • Lane-mile operating at LOS F; and • Hours operating at LOS F. A-10

Table A.4 Types of Performance Measures Used by Agencies (continued) Metro Area Operation Agencies Planning Agencies 6. Los Angeles, California With the exception of HOV facilities, Caltrans District 7 currently does not use a formal system to measure performance. Therefore, the type of performance measures it collects and the uses of such data are limited. However, with the introduction of the Transportation System Performance Measures (TSPM) Program, performance measures will likely become more important in the near future. The proposed TSPM includes a number of travel time- and throughput-based measurements as well as other nonmobility measurements. It is proposed that interregional travel time in key travel corridors be monitored with actual origins and destinations. This shall include both people- and goods-movements. Total person hours of delay also shall be measured. A standard delay definition that can be applied to all modes has yet to be developed. TSPM also includes a reliability metrics to highlight the variability in travel time between origin and destinations. HOV Facilities Output Measures • Total length of HOV facilities; and • Net change in lane-miles. Outcome Measures • Level of Service (LOS) during peak periods; and • Travel time savings per mile. Daily and hourly volume on HOV facilities both in terms of the number of vehicles and of people. Caltrans A Highway Congestion Monitoring Program (HICOMP) is mandated by the State of California to help achieve Caltrans’ stated objectives of increasing efficiency and reducing delays on the State’s freeway system. Each Caltrans district monitors freeway congestion level based on the following basic travel time parameters: • Magnitude (vehicle-hours of delay per day (vhdpd)); • Extent (congested directional miles (cdm)); and • Duration (hours). HICOMP defines recurrent congestion as a condition lasting for 15 minutes or longer and vehicular speeds are 35 miles per hour or less during peak commute periods on a typical incident-free weekday regardless of the posted speed limit. The total delay per segment is calculated by multiplying hourly vehicle volume by the duration of congestion in hours by the travel time exceeding that when traveling the same distance at 35 miles per hour. SCAG • Total delay (vehicle-hours and person-hours); • Total VHT; • Total VMT; • Average System Speed, “Q” (=VMT/VHT); • Percent variation in travel time; • Percent p.m. peak work trips within 45 minutes of home (Accessibility); and • Percent capacity utilized during peak conditions. 7. Portland, Oregon N/A • Travel Time Index (TTI); • Travel Delay; • Buffer Index (BI); • Volume-to-capacity Ratio (V/C); • Travel Time; and • Speed. A-11

Table A.4 Types of Performance Measures Used by Agencies (continued) Metro Area Operation Agencies Planning Agencies 8. Houston, Texas • Speed; • Delay; • Incident response time for large truck incidents; and • Web site usage and comments. N/A 9. Washington, D.C. Maryland (CHART) Goal: IMPROVE MOBILITY FOR OUR CUSTOMERS. Objective: Reduce congestion delay and associated costs caused by incidents by 1 percent annually. Performance Measures: Output: • Average incident duration; and • Number of incident responses and complete reports. Outcome: • Percentage of reduction in incident congestion delay; • Reduction in user costs ($ million) associated with incidents; and • Reduction in truck user costs ($ million) associated with incidents. Maryland (Statewide) • Percent of freeway lane-miles below congested volumes per lane. N/A A-12

Table A.4 Types of Performance Measures Used by Agencies (continued) Metro Area Operation Agencies Planning Agencies 10. Atlanta, Georgia Outcome • Hampered by data quality concerns. Currently experimenting with a two categories of congestion: Moderate (speeds between 30-45 mph) and sever (<30 mph). Considering additional performance measures, including reliability. Output • Traveler Information Calls: – Total calls; – Calls per day; – Calls per route; – Calls by type of call; – Average call length; and – Average answer time. • Incidents managed: – By category; – Detection method; and – Impact levels (general categories). • Number of construction closures; • Device Functioning; • Percent time devices are available; • Number of media communications by outlet; • Web site visits by type of information requested; • Service patrol assists: – By shift; – By type; – By detection type; and – By route. • Service patrol service times (auto versus truck): – Response time; – Clear time; and – Notification to clear time. N/A A-13

Table A.5 Uses of Performance Measures Metro Area Operation Agencies Planning Agencies 1. Minneapolis-St. Paul, Minnesota The operations outcome measures of travel speed and its derivatives of travel time and reliability are just now being developed for use by the RTMC staff. The primary reason for the previous nonusage is data quality as described in the Data Quality section and the recent move to the new RTMC Building. The RTMC staff uses the output measures for incident and staff efficiency to provide benefits information to Mn/DOT management and to the public. The incident data is published monthly by RTMC staff and distributed to Mn/DOT management staff. The staffing measures are used to measure personnel performance, to adjust staff size and hours and to better define the operators shift hours. The FIRST incident data is used to adjust individual patrol routes for the FIRST drivers and to define the FIRST divers need per shift. Performance data also serve as a basis for Metro Council plans and reports such as the Regional Transportation Plan, the Transportation Improvement Plan, the annual update of the Transportation Systems Audit and various operations studies. Although performance measures generally are not linked directly to specific investments, the findings and recommendations of the plans ultimately play a part in influencing investment decisions. 2. Seattle, Washington WSDOT uses performance measures to help allocate resources, determine the effectiveness of a variety of programs, and help plan and prioritize system improvements, primarily from an operations perspective. A variety of measures are computed. Not all of these measures are routinely reported outside of the Department, but key statistics that describe either the current state-of-the-system, trends that are occurring, or the effectiveness of major policies are reported quarterly as part of the Department’s efforts to clarify why it is taking specific actions and to improve its accountability to the public and public decision-makers. Statistics allow comparisons of the relative performance of various corridors or roadway sections under study. These aggregated statistics also can be converted to unit values (e.g., person hours of delay per mile) to further improve the ability to compare and prioritize the relative condition of corridors or roadway segments. 3. Hampton Roads, Virginia The operations outcome measures of travel speed and its derivatives of travel time and reliability currently are not used in real time by the STC or the HRPDC staff. The primary reason for the non- usage is that the data quality is often inadequate as a basis for making operations decisions. The STC staff uses the output measures for incident and staff efficiency to provide benefits information to VDOT management and to the public. The incident data is published monthly by the STC contractor and distributed to VDOT management staff. The staffing measures are used to measure personnel performance, to adjust staff size and hours and to better define the operators shift hours. The FIRST incident data is used to adjust individual patrol routes for the FIRT drivers and to define the FIRT divers need per shift. Operations managers in the STC also use the data for diagnostic purposes in evaluating the effectiveness of implemented strategies under varying conditions. This activity also may be tied to training of STC operations personnel. The field equipment maintenance data is collected and saved, but it is not yet used for management purposes. The current system is stored by use of Automated Maintenance Management software and all equipment is tracked by cabinet location. HRPDC provides regional planning and policy decisions in areas of transportation, air quality, and regional development. The performance measures allow member agencies to make informed decisions on matters concerning not only the local jurisdictions but the Hampton Roads region as a whole. Historical performance measures reporting also is an important use of the metrics. The cyclical studies enable trend analysis for the travel demand model development and special reports for congestion and air quality; therefore, care is taken when metrics are developed or enhanced to ensure compatibility with historic data. Trend analysis is useful to pinpoint problem areas in both the long and short term. Performance data also serve as a basis for HRPDC plans and reports such as the Regional Transportation Plan, the Transportation Improvement Plan, and various bridge and tunnel operations studies. Although performance measures generally are not linked directly to specific investments, the findings and recommendations of the plans ultimately play a part in influencing investment decisions. A-14

Table A.5 Uses of Performance Measures (continued) Metro Area Operation Agencies Planning Agencies 4. Milwaukee, Wisconsin WisDOT plans to use performance measures for the following applications: 1. Communicating the benefits of and marketing operations to WisDOT managers/administration; 2. Benchmarking performance for and providing feedback to control room operators; and 3. Use for operations management and planning (e.g., fine-tuning ramp meter timing, scheduling lane and ramp closures, etc.). The FSOA program has developed a “Project Appraisal Report” (see pages 3 to 5 for an example) that could be used to: 1) compare various alternatives in a particular project or 2) prioritize or rank various projects for programming and funding. The Project Appraisal Report includes 1 quantitative measure and 3 qualitative (“intangible”) measures. 5. Phoenix, Arizona Performance measures are used by TTG for operations, emergency response and traveler information applications. Each measure is employed to achieve the objectives set forth in the ADOT/ITD Strategic Action Plan. The monitoring of speed and volume using FMS data allows TTG to measure the average percentage of Phoenix freeways reaching level of service “E” or “F” on weekdays to determine if the Group’s objective of operating 60 percent of the freeways at a level “D” or better during rush hour is met. The speed and volume data also are used for ramp metering. Although TOC staffing level is measured, it is unclear that the information is being used to adjust work schedules. There also is no indication that the data are disseminated to field operations so that steps can be taken to rectify measures that do not meet stated objectives, i.e., clearance time. For freeway construction, performance measures are used in three ways. First, the measures help bolster priorities for freeways versus other transportation projects. They also provide justification of the one-half cent sales tax for construction of controlled-access highways. Lastly, they are used by ITD to prioritize implementation. The performance measures allow member agencies to make informed decisions on matters concerning not only the local jurisdictions but the region as a whole. One of the key purposes of the Travel Speed Study was to validate MAG’s planning model as required by EPA. Therefore, the measures were developed specifically to meet this need. Historical performance measures reporting also is an important use of the metrics. The cyclical studies enable trend analysis; therefore, care is taken when metrics are developed or enhanced to ensure compatibility with historic data. Trend analysis is useful to pinpoint problem areas in both the long and short term. The bottleneck study was commissioned to address the problem areas identified in the Traffic Quality report. Performance data also serve as a basis for MAG plans and reports. 6. Los Angeles, California The primary use of performance measure is performance reporting. Performance is documented in two reports: the state mandated HICOMP report and the HOV report. The information is then used for planning and program purposes on the State and local levels. Performance measures are generally not linked to investment decisions. However, they are sometimes used to justify specific programs. When fully adopted, PeMS has the potential to dramatically change the way congestion is monitored and performance is measured. PeMS retrieves information from real-time and historic database and presents the information in various forms. Its value lies in allowing planning and operations staff to base their decisions on real system performance data without spending an undue amount of resources on data collection. SCAG performance measures are developed during the RTP process to evaluate alternatives and select the best ones for inclusion in the Plan. The performance measures are tied directly to at least one of six established RTP goals. The goals and performance measures do not emphasize the freeway system but include it as an integral part of a comprehensive transportation system that includes all modes. 7. Portland, Oregon N/A The implementation of the measures and methodology will allow ODOT decision-makers to compare operations program benefits with other programs (e.g., safety, bridge, maintenance). The project provides the operations program with a process for estimating benefits, and this will help the program to identify places for additional study and investment. Finally, these methods will help define the return on operational investments. A-15

Table A.5 Uses of Performance Measures (continued) Metro Area Operation Agencies Planning Agencies 8. Houston, Texas Very few performance goals. Some operations equipment reliability and timely repair standards are used. These goals are measured. Most use of performance measures are for real-time management of the system. N/A 9. Washington, D.C. Maryland Used for annual reporting and the development of specific strategies to meet mobility targets N/A 10. Atlanta, Georgia The NaviGAtor staff uses the output measures for incident and staff efficiency to provide benefits information to GDOT management and to the public in a weekly newsletter format. The incident and traveler information data is published monthly and distributed to GDOT management staff and others. The staffing measures are used to measure personnel performance, to adjust staff size and hours and to better define the operators shift hours. The HERO (service patrol) incident data is used to adjust individual patrol routes for the HERO drivers and to define the HERO divers need per shift. N/A A-16

Table A.6 Data Collection, Analysis, and Quality Procedures Metro Area Data Collection and Analysis Procedures Data Quality Procedures 1. Minneapolis-St. Paul, Minnesota Operations performance data are being collected by the RTMC using in-pavement loop detectors. The data is collected and stored in 30 second intervals. Incident detection and verification are done by CCTV monitoring by RTMC operators, radio calls from FIRST drivers and by calls from Minnesota State Patrol, who receive 911 cell phone calls from travelers. RTMC also shares video with the Minnesota State Patrol, Mn/DOT Metro District Maintenance, Metro Transit, cities, counties, and all local television stations. UMN conducts an annual ramp metering evaluation for Mn/DOT. The evaluation is through simulation and it is related to freeway performance. UMN is considering using some cost-based measures, such as a mainline wait time versus a ramp wait time cost comparison for the next ramp meter evaluation. UMN-Duluth maintains the speed data archive, they receive raw data flat files daily from Mn/DOT. The data is collected and stored in 30 second intervals. Mn/DOT and UMN worked together to develop the data quality process for Mn/DOT. The data quality checks detect outlying and missing data. The data collection algorithm notes loss of communications, flags data that is outlying from expected values or data that is nonchanging over a specified time period, notes missing or off-line detectors and assign a substitute (fake) loop. This process is done only for historical data, not real-time data. 2. Seattle, Washington The Seattle performance measures effort is driven primarily by the existence of a significant archive of inductance loop data, which are collected by WSDOT’s freeway ramp metering algorithm. This system consists of 620 loop stations comprising over 4,080 individual loops, of which 1,020 are paired into dual loop stations and the rest are either single loops located in the freeway mainlines or on- ramps. An archive of the 20-second data is maintained at the University of Washington. In addition to the inductance loop data, WSDOT undertakes four additional, significant data collection efforts: • Vehicle occupancy data collection; • Transit ridership data collection; • Incident occurrence and response reporting; and • Public opinion surveys. Each of WSDOT’s data collection programs has a variety of quality control steps designed to increase the quality of the data collected, as well as to identify and remove from further analyses those data that do not accurately describe actual roadway conditions. The software that operate in the Type 170 traffic controllers used by WSDOT produces an eight-bit error status code with each 20-second data packet transmitted from the field to the TSMC. All vehicle occupancy data used by WSDOT are collected manually by student work crews. Data from the field are entered into personal data assistants (PDA) carried by the data collection staff as each observed vehicle passes the count location. Each field entry is time stamped as it is entered into the PDA. To ensure the quality of these data, a series of software programs has been written to determine whether staff are actually entering real observations. The two basic checks compare the speed at which timestamps are entered (too many too quickly mean that the data collector is “inventing” vehicles) and compare records against continuously repeated numbers (which is usually an indication that a data entry key is stuck in the “on” position). Once the data pass through the initial set of checks, the summary vehicle occupancy rates are checked against previous data collected at this same location. A-17

Table A.6 Data Collection, Analysis, and Quality Procedures (continued) Metro Area Data Collection and Analysis Procedures Data Quality Procedures 3. Hampton Roads, Virginia Surveillance data are compiled with incident data, and traffic data from selected arterial loop detectors in the Hampton Roads Archived Data Management System (ADMS). This recently developed system provides access to real-time and historical traffic volume, speed, and incident data for selected regional corridors. The ADMS server will be moved to the VDOT Central Office in Richmond in the near future. The data are managed using SQL Queries. They are available currently to registered users, including STC and HRPDC staff as well as any user requesting access for research purposes. Data analysis currently is conducted by Smart Travel Lab staff. The data quality of STC-generated information has been hampered by loop detector failures. At any given time, only about one-half of the sensors are operational and the STC does not conduct a systematic sensor calibration process to validate the accuracy of the individual sensors. Further, their maintenance priority is the lowest among field equipment – both CCTV and VMS are a much higher maintenance priority. The operations staff does not use the speed data because of the low quality and travel speeds are reported on the traveler information web site. Realizing this problem impairs the effectiveness of the entire STC, VDOT is holding the Phase 2 contractor to much higher standards of installation quality than in Phase 1 and requiring detector calibration as part of the system delivery. The STC has developed a budget to repair, replace and calibrate defective loop detector installed in Phase 1, however that effort has not yet been funded. The Smart Travel Lab, as part of the ADMS project management task, conducts data quality checks on the detector data. The checks can determine if a detector is on-line and is reporting data continuously. Traffic count data is collected by VDOT through a statewide count program. The current system uses traffic count machines at specified locations for two days of counting per year. Future plans may include use of STC data in areas where instrumentation is available. HRPDC obtains the traffic count data from VDOT for the Hampton Roads area. HRPDC conducts a CMS analysis in the region once each three years. The travel speed collection is through floating cars. HRPDC plans to incorporate GPS technology in the next update of the CMS. STC data will be used in future updates when the ITS data quality is maintained. Since HRPDC obtains its data from VDOT, they do not conduct independent validation of data. The agency, however, does perform internal checks and cleaning of the data based on historical data and knowledge of local conditions. 4. Milwaukee, Wisconsin WisDOT is enhancing and upgrading their current ITS data archiving system to support a number of initiatives, one of which is the freeway performance reporting. This upgrade is referred to as the “Data Extractor” project, and the data warehouse. The Data Extractor project is expected to make the TOC’s data resources much more accessible, and will include scheduled (e.g., monthly) and ad hoc reporting functions. The core data types that will be included in the Data Extractor upgrade include: • Traffic detector data; • History of traffic detector failures; • History of traffic detector configurations; and • Lane and ramp closures. Complementary University Activities The Traffic Operations and Safety Lab (Traffic Lab, or TOPS) at the University of Wisconsin-Madison is developing the TransPortal system, a data warehouse in which they hope to integrate numerous types of disparate data, such as: • Road-weather information; • Public safety/incident management; • Traffic data; and • Static information (e.g., historical crash records). The vision is that a system like this, once implemented, could be used to support operational decisions and possible state-to-state data exchanges (such as in GCM). Loop Detectors As with most traffic operations centers, loop detector maintenance is an issue for WisDOT. The majority of their loops are double-loops for speed measurement, and they are typically installed in conduit. This installation practice helps somewhat. A-18

Table A.6 Data Collection, Analysis, and Quality Procedures (continued) Metro Area Data Collection and Analysis Procedures Data Quality Procedures 5. Phoenix, Arizona Operations performance data are being collected by FMS using in-pavement loop detectors and passive-acoustic detectors. Incident detection and verification are done by close-circuit television (CCTV) and by 911 cell phone calls from travelers. The system is linked to ADOT’s Highway Condition Reporting System (HCRS) is the main conduit for reporting highway conditions to the public. The powerful computer system has the capability to automatically retrieve weather forecast and advisory from the National Weather Service. It also collects special events, road closures and detour information and communicates with TOC’s central computer system to obtain incidents and roadway conditions information. Communications take place via the Internet, wide-area network, and dial-up. Data analysis is usually conducted by ADOT staff. Whenever possible, traffic counts are based on data collected by FMS. When loop detectors are not available, either because the system is down or because the survey area is outside the FMS coverage area, radars are used. The annual short-term traffic counts are done over a 48-hour period at 15-minute intervals. MAG uses two different data collection methods for the Traffic Quality report and the Traffic Speed Study: 1) overlapping aerial photography and 2) floating cars. The data quality of FMS-generated information has been hampered by loop detector failures. At any given time, only about 65 to 85 percent of the sensors are operational. Further, their maintenance priority is the lowest among field equipment. To account for poor detector data, an ITD staff does manual screenings. Techniques published by the Texas Transportation Institute have been helpful for this purpose. The data quality is documented and is available to those using the data. 6. Los Angeles, California Caltrans Caltrans has spearheaded a data archival, processing and analysis system known as Freeway Performance Measurement System (PeMS) to facilitate performance measures calculations and analysis. PeMS includes data from roadway sensors and incident data from the California Highway Patrol and the Caltrans Traffic Accident Surveillance and Analysis System (TASAS). However, the application of incident data is not fully developed. The performance measures collected by ATMS and processed by PeMS are being fed into a newly developed Regional Integrated Intelligent Transportation System (RIITS). RIITS is a collaborate effort by Caltrans, the Los Angeles County Metropolitan Transportation Authority, and the City of Los Angeles. Its goal is to integrate traffic information from a variety of sources into one central system so to facilitate information sharing. It is accessible through the Internet (www.riits.net). The data also are being used by SCAG as basis for its transportation system analysis. SCAG SCAG has launched a Regional Transportation Monitoring Information System (RTMIS). RTMIS is a planning tool designed to “assist staff in monitoring and assessing the performance of the current transportation system against regional goals.” RTMIS consists of four modules: highway, real-time traffic, and mapping. Four other modules are planned for future implementation. They include transit, aviation, nonmotorized and maritime. SCAG relies heavily on Caltrans as its source of performance data. RTMIS has two input components: 1) HPMS and 2) PeMS. Base year data are established for each mobility-related performance measure. Travel demand model is then used to project future speed and delay and calculate the travel time savings that would result if recommended improvements are made. The data quality of ATMS-generated information has been hampered by loop detector failures. At any given time, only about 70 percent of the sensors are operational. While there is a Detector Fitness Program in place, it receives no dedicated budget. Freeway maintenance activities are prioritized according to 1) safety, 2) roadway preservation, and 3) others. Loop maintenance is in the lowest priority category. Some detectors are not fixed for weeks or months. For this reason, it will remain a challenge to collecting quality data. Data go through a normalization process. Both sets of data, raw and normalized, are archived. Through PeMS, users can obtain information on data quality and detector health. An application has been developed in PeMS to account for poor detector data. A-19

Table A.6 Data Collection, Analysis, and Quality Procedures (continued) Metro Area Data Collection and Analysis Procedures Data Quality Procedures 7. Portland, Oregon The Oregon statewide procedure is based on using the HERS-ST model and an augmented HPMS data set. No special data quality procedures undertaken beyond what normally occurs for HPMS. 8. Houston, Texas The data for the real-time traffic map are collected using vehicle equipped with AVI tags that are generally used for electronic toll collections on the network of the Harris County Toll Road Authority. Archived raw data has been kept since the system came on-line in 1993. The incident portion of the database is very extensive and provides details of each incident such as which lanes were closed, incident duration, and the actions taken to resolve the incident to name a few. Weather is not archived in the TranStar system. Work zone information has been archived since May 2002. Special events, work zone and emergency road closure information are posted on the web site for each day, and for a few days in advance when known. These are text files, not numeric or database files. On most days 100% of the AVI reader stations are operational at a given time. Some sensors may need to be temporarily removed because of freeway construction activities, but changes can easily be made in the structure of the look-up table of locations for matched pairs to estimate travel times such that continued data collection is not interrupted. TxDOT has a contractor provide maintenance services for the infrastructure needed to collect the AVI data; measures are in place to assure that any nonoperational sites are repaired within certain time limits. The equipment has been extremely reliable both in the field as well as in the office. A vast majority of short-term outages are a result of loss of communication or interruption of electrical service to the field sites. 9. Washington, D.C. Maryland CHART currently has traffic sensors at 1.0- to 1.5-mile spacing along some sections of I-70, I-83, I-95, I-270, I-495, I-695, I-795 and U.S. 50. The traffic sensors provide volume, occupancy and speed. Incident data is stored in a single Excel file record. U. of Maryland archives this data and compiles an annual operations evaluation report for CHART. Maryland SHA annually conducts customer surveys for the entire agency. A couple of questions regarding CHART are always included. CHART does not conduct a separate customer survey. U. of Maryland has developed a methodology for estimating the benefits of incident management programs, including estimating the amount of incident-related delay. This methodology is the first cut and is being improved. Virginia Virginia Transportation Research Center (VTRC) in Charlottesville receives all sensor data and archives it. VTRC conducts data quality checks. This function is being transferred to VDOT HQ. The Archived Data Management System developed for Hampton Roads is being extended to Northern Virginia. Several performance reports are available within the ADMS Maryland Data quality checks are not yet being conducted for the sensor data. Virginia As part of the ADMS development, an extensive series of quality control checks are being performed on the sensor data. VTRC tested the accuracy of the loops in 2002. They were found to 95% accurate. The speed data was off on many detectors and it was found that the installation quality was poor (installers didn’t measure the length and widths). The improperly installed loops have been corrected. A-20

Table A.6 Data Collection, Analysis, and Quality Procedures (continued) Metro Area Data Collection and Analysis Procedures Data Quality Procedures 10. Atlanta, Georgia Operations performance data are being collected by the NaviGAtor system using a video detection system (VDS). The current detection system covers 222 centerline miles and consists of cameras covering each mainline travel lane at one-third-mile spacing. There are approximately 1,100 VDS detectors. The sensors collect data at 20 second intervals. Incident detection and verification are done by close-circuit television (CCTV) monitoring by TMC operators, calls to *DOT customer service representatives, radio calls from HERO drivers and by calls from 911 centers, who dispatch the local public safety responders. The TMC provides traffic and incident data in real time to the NaviGAtor web site. The web site information also is available to be sent to PDA or cell phone users upon request. The Archived Data Management System is being upgraded by GDOT currently. The primary focus of the archived is the speed detectors (VDS). The archived data management system will eventually include the incident management system records along with the detector records. When fully operational this system will enable the various sources of data (VDS, NaviGAtor actions, HERO activities, construction activities and weather information) to be integrated and geo-located. GDOT recently completed an analysis of the VDS data quality. The findings of that analysis are summarized as follows: • 90% accuracy is required to support desired applications; • The VDS manufacturers specifications allow that level of accuracy; and • Field tests found that individual camera accuracy was highly variable. The analysis concluded that several improvements should be made to the VDS: • Revise system design to provide more accurate data; • Identify and treat systematic errors to achieve accuracy and coverage to support desired data products; • Generate metadata to clearly identify data availability and validity of data sample for the user; • Update maintenance procedures and make maintenance more frequent; • Integrate other data (incident data, speed data from other sources such GPS and toll tag readers); • Identify stations that have higher probability of reporting true values, move focus from single camera accuracy to station and segment accuracy – take advantage of redundancy and connectivity in the system; • Develop and use a data cleaning process; and • Generate truck percentage, VDS allows identification of trucks, but it currently is not used. GDOT currently is implementing the report’s recommendations through an in-house VDS upgrade process. A-21

Guide to Effective Freeway Performance Measurement