Pilot Testing of the TCAPP Decision Guide and Related Capacity Products: Charlottesville-Albemarle Metropolitan Planning Organization, Virginia (2014)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

SHRP 2 Capacity Project C39A2 Pilot Testing of the TCAPP Decision Guide and Related Capacity Products: Charlottesville-Albemarle Metropolitan Planning Organization, Virginia Sarah Rhodes and Marie Scheetz Charlottesville-Albemarle Metropolitan Planning Organization John Miller and Peter Ohlms Virginia Center for Transportation Innovation and Research Thomas M. Guterbock, Deborah Rexrode, and Matthew Braswell University of Virginia’s Center for Survey Research TRANSPORTATION RESEARCH BOARD Washington, D.C. 2015 www.TRB.org

© 2015 National Academy of Sciences. All rights reserved. ACKNOWLEDGMENTS This work was sponsored by the Federal Highway Administration in cooperation with the American Association of State Highway and Transportation Officials. It was conducted in the second Strategic Highway Research Program, which is administered by the Transportation Research Board of the National Academies. The research reported herein was performed by the Charlottesville-Albemarle Metropolitan Planning Organization (MPO): Sarah Rhodes, MPO Program Manager, and Marie Sheetz, Transportation Planner. Other authors of this report are the Virginia Center for Transportation Innovation and Research: John Miller, Principal Research Scientist, and Peter Ohlms, Research Scientist. The authors appreciate the insights offered by Robin Bebel, formerly of the University of Virginia’s Center for Survey Research, Amy O’Leary of the Virginia Center for Transportation Innovation and Research, David Shreve of the University of Virginia’s Center for Survey Research, and Stephen Williams, formerly of the Thomas Jefferson Planning District Commission. COPYRIGHT INFORMATION Authors herein are responsible for the authenticity of their materials and for obtaining written permissions from publishers or persons who own the copyright to any previously published or copyrighted material used herein. The second Strategic Highway Research Program grants permission to reproduce material in this publication for classroom and not-for-profit purposes. Permission is given with the understanding that none of the material will be used to imply TRB, AASHTO, or FHWA endorsement of a particular product, method, or practice. It is expected that those reproducing material in this document for educational and not-for-profit purposes will give appropriate acknowledgment of the source of any reprinted or reproduced material. For other uses of the material, request permission from SHRP 2. NOTICE The project that is the subject of this document was a part of the second Strategic Highway Research Program, conducted by the Transportation Research Board with the approval of the Governing Board of the National Research Council. The Transportation Research Board of the National Academies, the National Research Council, and the sponsors of the second Strategic Highway Research Program do not endorse products or manufacturers. Trade or manufacturers’ names appear herein solely because they are considered essential to the object of the report.

DISCLAIMER The opinions and conclusions expressed or implied in this document are those of the researchers who performed the research. They are not necessarily those of the second Strategic Highway Research Program, the Transportation Research Board, the National Research Council, or the program sponsors. The information contained in this document was taken directly from the submission of the authors. This material has not been edited by the Transportation Research Board. SPECIAL NOTE: This document IS NOT an official publication of the second Strategic Highway Research Program, the Transportation Research Board, the National Research Council, or the National Academies.

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. On the authority of the charter granted to it by Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. C. D. (Dan) Mote, Jr., is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Victor J. Dzau is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. C.D. (Dan) Mote, Jr., are chair and vice chair, respectively, of the National Research Council. The Transportation Research Board is one of six major divisions of the National Research Council. The mission of the Transportation Research Board is to provide leadership in transportation innovation and progress through research and information exchange, conducted within a setting that is objective, interdisciplinary, and multimodal. The Board’s varied activities annually engage about 7,000 engineers, scientists, and other transportation researchers and practitioners from the public and private sectors and academia, all of whom contribute their expertise in the public interest. The program is supported by state transportation departments, federal agencies including the component administrations of the U.S. Department of Transportation, and other organizations and individuals interested in the development of transportation. www.TRB.org www.national-academies.org

Contents 1 Executive Summary 5 CHAPTER 1 Introduction 5 Overview of the TCAPP (PlanWorks) Decision Guide 7 Contribution of TCAPP to the Charlottesville-Albemarle MPO 7 Contribution of the Charlottesville-Albemarle MPO to TCAPP 9 CHAPTER 2 Purpose and Scope 10 CHAPTER 3 Methodology 10 Task 1. Initial MPO Stakeholder Survey 11 Task 2. Follow-Up MPO Stakeholder Survey 13 Task 3. Successes and Challenges of Applying TCAPP 14 Task 4. Surveys of the General Public 16 CHAPTER 4 Results—The Most Influential Performance Measures 20 CHAPTER 5 Results—The Influence of Performance Measures on Decision Making 21 Impacts of Changes in Performance Measures on Respondents’ Support for a Scenario 24 Explanation of Why Support Varied by Performance Measure 28 Reference 29 CHAPTER 6 Results—TCAPP (PlanWorks) Assessment 30 LRP-1: Approve Scope of LRTP Process 31 LRP-2: Approve Vision and Goals 33 LRP-3: Approve Evaluation Criteria, Methods and Measures 35 LRP-4: Approve Transportation Deficiencies 37 LRP-5: Approve Financial Assumptions 38 LRP-6: Approve Strategies 40 LRP-7: Approve Plan Scenarios 43 LRP-8: Adopt Preferred Plan Scenario 44 LRP-10: Adopt LRTP by MPO 46 CHAPTER 7 Conclusions 49 CHAPTER 8 Recommendations A1 APPENDIX A Initial Surveys B1 APPENDIX B Follow-Up Surveys

Executive Summary The Transportation for Communities—Advancing Projects through Partnerships (TCAPP), now known as PlanWorks, Decision Guide lays out a logical process for moving through the many decision steps for planning, prioritizing, and developing improved transportation system capacity. The first phase of this process, long-range [transportation] planning (abbreviated as “LRP” within TCAPP), plays a critical role in the overall effort. The Charlottesville-Albemarle Metropolitan Planning Organization (MPO) sought to implement TCAPP while updating its Long-Range Transportation Plan (LRTP). Throughout this document, LRP refers to the long- range planning component of the TCAPP Decision Guide and LRTP refers to the MPO’s Long- Range Transportation Plan process or document. A significant component of TCAPP is the use of evaluation criteria, also known as performance measures. While they are developed in the step titled, “Approve Evaluation Criteria, Methods and Measures” (LRP-3), they are also the lens through which all transportation projects under consideration are assessed. These criteria, and by extension any associated analysis, have tremendous influence on which projects are ultimately included in an LRTP. Because such criteria depend on the values of individuals (e.g., the criterion “access to transit” and the criterion “hours of delay” may differ in importance for a given individual), MPO staff surveyed the general public to identify the most important performance measures. A subsequent survey of MPO stakeholders (i.e., individuals with some involvement in the planning process) identified the extent to which these evaluation criteria influenced stakeholders’ support for transportation investments. MPO staff worked with its project partners, the Virginia Center for Transportation Innovation and Research and the University of Virginia’s Center for Survey Research, to implement these surveys. The value of this information is twofold: to show other MPOs which performance measures were of interest and to demonstrate the extent to which changes in these measures could influence decisions. The MPO also assessed the implementation of TCAPP at each step of the Decision Guide. With each step the MPO staff noted how it chose to implement the step, key successes, and key challenges. This information may be of interest to entities that wish to know which steps are likely to be “practice ready” in other locations and which steps may require additional guidance. For example, in its application of LRP-3, the MPO staff found that the use of evaluation criteria enabled a focus on regional benefits that would not have been the case without the step. However, MPO staff also noted that as the criteria were a relatively new concept for this particular region, the linkage of criteria (e.g., reductions in crashes) to more fundamental goals (e.g., improved safety) was a challenge due in part to data limitations. This report thus assesses three major components: the performance measures (e.g., evaluation criteria) that are most important to stakeholders, the extent to which changes in these performance measures influence stakeholders’ evaluation of potential investments, and lessons learned from the MPO’s implementation of TCAPP. Details of the methodology and results are given in the body of the report, and the central findings are summarized as follows: 1

Performance measure areas matter more than specific performance measures within those areas. There are large nominal differences, in terms of importance to stakeholders, between performance measure areas. For example, whereas more than 80% of stakeholders rated each of three areas—land use, safety, and the environment—as “very important” or “extremely important,” only 40% of respondents gave the freight area such a designation. Whereas approximately 70% of the general public rated the two areas of travel times and safety as “extremely important” or “very important,” only approximately 40% gave such a designation to the two areas of “bus systems” and “ease of walking and biking.” Within each area, the relative importance of specific performance measures was influenced by how respondents had rated the area’s overall importance. For example, recalling that only 40% of stakeholders had rated freight as extremely important or very important, it was not surprising that the percentage of stakeholders who rated individual freight measures as “extremely important” or “very important” ranged from 15.5% (“importance of the buffer index for key corridors”) to 38.9% (“number of double-stack rail restrictions”). By contrast, because land use was viewed as such an important area overall, the percentage of stakeholders rating individual land use performance measures as “extremely important” or “very important” ranged from 56.1% (“number of historical and archaeological sites affected”) to 86.1% (“impact on street connectivity”). Performance measure values influence respondents’ support for a project. Overall, roughly half of responses (48%) indicated that changing the value of the performance measure from its expected value would “definitely” or “probably” influence respondents’ support for a scenario. Further, there appear to be three conditions that increase the likelihood that a change in a given performance measure will influence respondents. One condition is a higher-than-expected value of the performance measure: most respondents (57%) indicated that the performance measure would influence their support for a scenario when this measure was higher than expected (compared with just 38% when the measure was lower than expected). A second condition is a measure that shows a relatively large percentage change. For example, only 22% of responses indicated that the passenger mobility measure would definitely or probably influence their support for a scenario, and a contributing factor may have been that whether this measure had a higher-than-expected value or lower-than-expected value, the difference was less than a single percentage point. In fact, some free responses suggested that when the percentage change is small, the performance measure is not likely to influence support for a scenario. A third condition, partly related to the above subsection, is that the performance measure should reflect an area of importance to stakeholders. For example, an increase in safety affected support for a project for more than two-thirds of respondents: notably more than 80% of respondents had ranked safety as “extremely” or “very” important. 2

Most TCAPP steps, as implemented, add value to the planning process. A majority of the TCAPP steps in the LRP series add value to the planning process. For example, the omission of a portion of LRP-1 (Approve Scope of LRTP Process) created some confusion that a full implementation of LRP-1 would have avoided. Generally, LRP-3 (Approve Evaluation Criteria, Methods and Measures), LRP-4 (Approve Transportation Deficiencies), LRP-8 (Adopt Preferred Plan Scenario), and LRP-10 (Adopt LRTP by MPO) all initially worked as intended. For example, given the six months of MPO discussion for LRP-1 through LRP-7, the adoption of a preferred scenario (LRP-8) was relatively smooth. It was also remarked that the use of scenarios in LRP-6 (Approve Strategies) and LRP-7 (Approve Plan Scenarios) led to a less politically driven project list than had been the case with previous long-range plans. The availability of evaluation criteria (LRP-3) encouraged stakeholders—both from the MPO and the general public—to view the alternative scenarios in an objective fashion, directly considering impacts across four major areas: mobility (e.g., hours of congestion), economy (e.g., average travel time to work), the environment (e.g., percentage change in annual tons of particulate matter), and the community (e.g., the percentage of households with an income under $25,000— which for this area are environmental justice populations—with access to transit). Thus this effort generally found that the LRP steps as given may add value to the planning process. Recommendations There are four critical areas where application of TCAPP demonstrates a need for additional guidance. The body of the report details challenges in applying these steps, and this additional guidance that is needed is summarized as follows: • TCAPP may need an intermediate step that relates high-level goals to quantifiable performance measures. A disconnect was noted between the higher-level goals in LRP-2 (Approve Vision and Goals) and specific performance measures in LRP-3 (Approve Evaluation Criteria, Methods and Measures). For example, regarding the goal “develop a multimodal transportation vision” from LRP-2, the MPO had focused on two objectives: (1) create more flexible roadway designs and (2) work toward developing a regional transit authority. While these two objectives were action items for MPO staff, there was not a comparable performance measure from LRP-3 that quantified progress toward these objectives. TCAPP may thus need to emphasize to participants that LRP-2 and LRP-3 should be closely coordinated. • TCAPP may need to clarify that the influence of a performance measure derives not just from its meaning but its range of impacts. There were some measures that an initial survey showed to be important to respondents (e.g., change in the auto mode share) but which were later found not to be influential owing to their relatively small impact (e.g., the different scenarios considered changed the auto mode share by less than half a 3

percentage point). Thus, either a more influential measure can be considered or expert interpretation of the measure can be provided. (For instance, using this example, one could replace auto mode share with percentage growth in pedestrian mode share, or an expert could explain the utility of a small percentage change in the auto mode share). • TCAPP may need strategies for developing better performance measures from incomplete data sets. While the value of the performance measures is ostensibly that they provide a data-driven way of measuring progress, there are some data sets that are incomplete or which are so disaggregate that substantial staff time is needed to relate them to specific goals. For example, although safety was one of the 16 evaluation criteria, it was difficult to take advantage of the available crash data for this criterion. • MPOs may need (1) greater flexibility (when deficiencies are defined), and (2) educational outreach (when scenarios are generated). Stakeholders thought that the list of deficiencies identified in LRP-4 (Approve Transportation Deficiencies) should be expanded beyond the two categories of road congestion and transit access. The use of a scenario—that is, evaluating transportation projects as a set rather than individually, where the set shares some common strategy—was confusing for some stakeholders, and some elected officials outside the decision-making body of the MPO thought that the use of scenarios was misleading. 4

CHAPTER 1 Introduction The development of a long-range transportation plan (LRTP) is always challenging for a metropolitan planning organization (MPO) because the process is outcome based rather than prescriptive. While Section 450.322 of the Federal Code of Regulations provides a list of requirements that must be achieved with the development of any LRTP, these requirements do not provide structure or advice as to how to achieve these required elements. Such flexibility is reasonable as MPOs vary substantially in terms of size, needs, and resources, but it also means that while there is clarity about what a plan must have, there is not detailed guidance on how to achieve that content. Overview of the TCAPP (PlanWorks) Decision Guide The Transportation for Communities—Advancing Projects through Partnerships (TCAPP) Decision Guide was developed under the second Strategic Highway Research Program (SHRP 2) Capacity Program and focuses on developing a framework that would foster a collaborative decision-making process in the development of transportation improvements. From the perspective of the Charlottesville-Albemarle MPO the guide provides anchor points for key decisions that must happen as each transportation improvement develops. For this project the MPO only assessed the long-range transportation planning portion of the TCAPP Decision Guide, that is, Steps LRP-1 through LRP-11. (The guide covers key transportation decisions for long-range transportation planning, the focus of this report, as well as three other processes: programming, corridor planning, and environmental review/NEPA merged with permitting.) In its entirety, the guide covers 44 key decisions of the transportation development process. The first 11 steps of the TCAPP Decision Guide are outlined in Table 1.1. It should be noted that the Charlottesville-Albemarle MPO is in an attainment area; therefore, Steps LRP-9 and LRP-11 were not assessed by the MPO as conformity analysis is not required. Also, note that throughout this document, LRP refers to the long-range planning component of the TCAPP Decision Guide and LRTP refers to the MPO’s Long-Range Transportation Plan process or document. 5

Table 1.1. LRP Portion of the TCAPP (PlanWorks) Decision Guide STEP TITLE DESCRIPTION LRP-1 Approve Scope of LRTP Process The scoping key decision involves a broad assessment of the data, decisions, and relationships to consider, acquire, or make throughout the entire LRTP process. LRP-2 Approve Vision and Goals At this key decision, the community's values, whether stated as a vision and goals or simply agreed upon by the stakeholders for the planning area, are used to guide the transportation-specific vision and goals. LRP-3 Approve Evaluation Criteria, Methods and Measures At this key decision the evaluation criteria, methods, and measures are approved that will allow decision makers to compare scenarios to the vision and goals and to one another. LRP-4 Approve Transportation Deficiencies The approved list of specific corridors, roads, and areas that are deficient identified at this key decision serves as a basis for problems and opportunities addressed in both the corridor planning and environmental review processes. LRP-5 Approve Financial Assumptions At this key decision information from the Programming /Fiscal-Constraint Phase is introduced into the LRTP decision- making process. LRP-6 Approve Strategies Strategies are developed to address the deficiencies identified in LRP-4. A strategy is a specific tactic or policy employed or recommended by an organization. Strategies could include road or multimodal improvements, land use changes, and other means of addressing deficiencies. LRP-7 Approve Plan Scenarios Scenarios are based on approved strategies and are compared using the evaluation criteria, methods, and measures. LRP-8 Adopt Preferred Plan Scenario At this key decision, a preferred plan scenario is adopted for inclusion in the draft LRTP. LRP-9 Adopt Finding of Conformity by MPO (not assessed) Air quality conformity analysis is done within the air quality process in order to validate that the preferred scenario meets current conformity requirements. LRP-10 Adopt LRTP by MPO At this key decision a final plan is adopted by the MPO board. This plan includes the preferred scenario, fiscal constraint, and air quality conformity. LRP-11 Approve Conformity Analysis (not assessed) This is a legally required decision consisting of the federal approval of air quality conformity of the LRTP. Source: http://transportationforcommunities.com/shrpc01/framework_application_kdps/21/0. 6

Contribution of TCAPP to the Charlottesville-Albemarle MPO MPO staff believed that TCAPP offered two clear opportunities to enhance the long-range transportation planning process: improved structure and greater objectivity. The first opportunity was structure. With the implementation of previous LRTPs, the method of developing the plan was not typically structured with a beginning, middle, and end clearly identified. The key requirements outlined by the Federal Highway Administration (FHWA) were always the focus, but the methods for achieving these requirements were developed as more of an ad-hoc process. Much of the LRTP effort focused on the development of plan goals or the development of a public process for informing various groups about the plan. Very little discussion actually focused on the specific projects that needed to be included in the LRTP and how those projects related to one another and the overall transportation system. With the 2040 LRTP update, MPO staff knew that they wanted to focus the planning effort on the future transportation system and any needed improvements. The TCAPP Decision Guide provided the framework for such a focus. The second opportunity was objectivity. MPO staff felt strongly that all projects under consideration for inclusion in the LRTP should be assessed by using evaluation criteria. As performance measurement is becoming more commonplace in the development and later assessment of transportation improvements, MPO staff felt strongly that this component should be a key part of this update. As the development of evaluation criteria is one of the key decisions in the TCAPP Decision Guide, MPO staff encouraged the MPO committees to adopt the TCAPP process. Contribution of the Charlottesville-Albemarle MPO to TCAPP For the Charlottesville-Albemarle MPO, the process of updating its LRTP for the year 2040 was already under way prior to receiving funding related to TCAPP. While some elements of the plan were moving forward, there was no clear structure as to the development of this plan. In trying to establish this structure, the MPO began researching what other communities had done. This research process led to the TCAPP Decision Guide. Fortunately the MPO’s LRTP process fell in line with a funding cycle from the Transportation Research Board’s Strategic Highway and Research Program’s second round. In November 2012 the Charlottesville-Albemarle MPO was selected as a funding recipient to test the TCAPP Decision Guide in conjunction with the implementation of its LRTP. Formally this project was called “Additional Pilot Tests to Improve TCAPP” and required an assessment of TCAPP. The MPO recognized that an assessment should not simply state whether TCAPP was or was not useful. Rather, there were three areas of exploration related to TCAPP where the MPO felt it could make a contribution. First, the “evaluation criteria” section of the TCAPP Decision Guide was the single most influential component that most affected the decisions of the public and the MPO stakeholders. Because there are a wide variety of evaluation criteria, also known as performance measures, that are available, the MPO sought to determine which specific measures 7

were most important to stakeholders, realizing that this information might be helpful to those who are implementing TCAPP elsewhere. Second, once such measures had been identified, it seemed appropriate to determine whether these measures could influence decisions. For example, one evaluation criterion is the percentage of motorists who, as a result of a set of investments, use alternative modes. (This criterion is directly supported by one of the main goals of the LRTP: a “fully integrated transportation system that allows people to choose from an array of modes to meet their mobility needs.”) Accordingly, the MPO sought to determine whether a forecasted shift toward alternative modes motivates individuals to select a given scenario. Third, it was also appropriate to determine which aspects of TCAPP were practice ready and which aspects might require additional effort. Some of the LRP steps appeared to dovetail well with the existing planning process, whereas others, such as the direction to evaluate sets of projects—known as scenarios—rather than individual projects, were newer. Thus, identifying these challenges was included as part of the TCAPP assessment. 8

CHAPTER 2 Purpose and Scope The Charlottesville-Albemarle MPO’s pilot test of TCAPP focused on three key research questions: • Which performance measures are most important to stakeholders? Because TCAPP emphasizes the use of performance criteria throughout the LRP steps, being able to identify which measures, as well as which performance areas, were most valuable to stakeholders was an initial motivation for this work. Stakeholders included both experts familiar with the process, such as planning staff or elected officials, and members of the general public. • Does the application of performance measures tangibly shift priorities that individual actors bring to the long-range transportation planning process such that changes in these measures could change the actors’ stated priorities? That is, can changes in calculated values of performance measures alter the transportation priorities at the most fundamental level—the individual? • Are there steps within TCAPP where additional research, guidance, or outreach might be needed to facilitate implementation? That is, which steps in the LRP sequence can be directly undertaken, and which steps might be challenging for MPOs to implement? The scope of this research was limited to information that could be gleaned through the Charlottesville-Albemarle MPO’s LRTP update and related stakeholder and public surveys. 9

CHAPTER 3 Methodology The case study approach was used, where the TCAPP process was applied to a single region in central Virginia. Four specific tasks were undertaken and are detailed in this chapter: 1. Survey MPO stakeholders initially to determine the most important performance measures. 2. Survey MPO stakeholders again to determine whether the most important measures influence an individual’s support for proposed investments. 3. Document the successes and challenges from applying TCAPP to the process of updating the LRTP for the Charlottesville-Albemarle MPO. 4. Conduct surveys of the general public to identify additional insights regarding the use of performance measures. Task 1. Initial MPO Stakeholder Survey In the spring of 2013, the MPO disseminated a survey to MPO stakeholders to determine which evaluation criteria were most important to them when considering potential project scenarios for the LRTP. Stakeholders included all 11 of the elected officials that serve the City of Charlottesville and Albemarle County, all 16 planning commissioners from the City of Charlottesville and Albemarle County, the MPO Technical Advisory Committee, and about 15 representatives from a wide variety of community groups that focused on issues ranging from the environment to economic development. This survey was a web-based survey administered to the stakeholders via e-mail. Stakeholders also received phone calls to encourage them to complete the survey. Because this survey focused on individuals who were more aware of the long-range transportation process than the general public, this survey had more depth than the survey of the general public (described in Task 4). A total of 44 responses were received: 8 from elected officials, 17 from appointed officials (including planning commissioners), 10 from planning or engineering staff, 4 from advocacy groups, and 5 from stakeholders identifying themselves as the general public. Prior to the survey’s release, the MPO worked with its project partners to develop said surveys. Partners include the Virginia Center for Transportation Innovation and Research, which assisted the MPO in developing the surveys’ overall purpose and drafting survey questions; and the University of Virginia’s Center for Survey Research (CSR), which implemented all surveys. It was decided that the surveys should assess a wide variety of evaluation criteria, even if some of the criteria were not ultimately used in the long-range transportation planning process. In starting the discussion of which criteria should be included, the project partners selected seven categories based on experience, planning requirements, and typical approaches. These seven categories are listed below. 10

• Environment • Land Use • Social Justice, Community, and Alternative Modes of Transportation • Economy • Public Safety • Passenger Mobility • Freight Mobility From these seven categories, the MPO and its project partners worked to develop specific evaluation criteria within these categories that assessed elements for the categories differently. For example, some measures considered positive impacts and some measures assessed adverse impacts. Judgment was required to group the measures. For instance, items in the third category (“social justice, community, and alternatives modes of transportation”) were grouped based on how the MPO structured its evaluation criteria in the long-range transportation planning process. As another example, the seventh category (“freight mobility”) was a stand-alone category but could also have been placed under “economy.” (Freight mobility is generally not a part of the MPO’s evaluation criteria but was included given national interest in freight research.) For a full list of the performance measures that were used, see Appendix A. The survey also asked stakeholders to indicate the extent to which positions held by various types of organizations, such as those that advocate for economic development, would influence stakeholders’ support for a given transportation investment. Task 2. Follow-Up MPO Stakeholder Survey A second survey of MPO stakeholders, conducted in February 2014 after completing TCAPP, sought to determine by how much the evaluation criteria would have to change in order to alter a stakeholder’s support for a given scenario. This directly supported the original Charlottesville- Albemarle MPO project application, which stated the following: This project will test the impact on the transportation priorities of individual decision makers and stakeholders in an LRTP project as identified in TCAPP LRP‐3, LRP‐4, LRP‐5, LRP‐6, LRP‐7, and LRP‐8 through in‐depth survey research conducted prior to the initiation of the LRTP process and also one year later at the completion of the process. For each of the seven performance areas, this second survey focused on the performance measure that was deemed most important in the first survey. MPO stakeholders were asked whether their support for hypothetical Scenario X (a grouping of transportation projects) would be influenced by a change in this performance measure. For example, in the passenger mobility area, the most important measure had been “amount of change in passenger-miles traveled on non-auto modes.” The survey first asked respondents whether a change in passenger-miles 11

traveled on auto modes, which in this case had been calculated to be a 0.4% reduction, influenced respondents’ support for the scenario. The survey then asked whether “low” or “high” values (which were given for each measure) influenced support for the scenario. (For example, in the case of auto passenger-miles traveled, a low value would have been a reduction of 0.2%, thus respondents were asked whether this would influence their support for Scenario X). Stakeholders were also asked if the information was helpful to them, how confident they were in this information, and how precise the evaluation criteria needed to be to affect their decisions. Generally, the “low,” “high,” and “Scenario X” values for each performance measure were calculated from eight different scenarios analyzed by MPO staff. That is, from these eight different scenarios and for each performance measure, a Scenario X value (calculated as the mean of the eight scenarios), a low value (usually calculated as the mean minus one standard deviation), and a high value (usually calculated as the mean plus one standard deviation) were given in the survey. The advantage of this approach is that the survey uses performance measure values that could be expected to be seen in practice. For example, from the eight scenarios and for the land use performance measure defined as “reduction in travel time to work,” the mean reduction minus one standard deviation yields 1.4%, and the mean reduction plus one standard deviation yields 3.6%. Thus, the survey uses a “low” value of 1.4% and a “high” value of 3.6%. There are two critical outcomes of this approach: • The range of “low” and “high” values differs by performance measure. For example, whereas 1.4% to 3.6% was the range for the aforementioned land use measure, the range for the social effects performance measure (essentially a measure of the size of the bicycle network) was from a “low” of 13.5% to a “high” of 24.5%. • The Scenario X value differs by performance measure. For example, the Scenario X improvement for the land use measure was a 2.5% reduction in travel time, whereas the Scenario X improvement for social effects was 19%. The advantage of this approach is that the “low,” “high,” and ”Scenario X” values presented in the survey are realistic, in terms of being based on scenarios examined during the LRTP process, rather than arbitrary values. (For example, there is no acceptable scenario of transportation projects the authors can envision that would yield, say, a 75% reduction in travel time.) However, an artifact of this approach is that the extent of influence of scenarios on a given performance measure will vary. As will be shown in Table 5.2, the Scenario X values for the performance measure ranged from being smaller than 1% (notably for the safety and passenger mobility measures) to being as large as 19% (for the social effects measure noted above). As was the case with Task 1, MPO stakeholders were contacted via e-mail, and follow-up phone calls were administered to those participants who had not completed the survey initially. A total of 41 individuals responded to the follow-up survey: 7 elected officials, 15 appointed officials, 9 planning or engineering staff, 4 from advocacy groups, and 6 identifying as the 12

general public. An effort was made to distribute the survey to the same respondents who had completed the first round MPO survey. Task 3. Successes and Challenges of Applying TCAPP The MPO documented the successes and challenges from applying each LRP component of TCAPP to the process of updating the LRTP. These LRP steps served as anchor points for the long-range transportation planning process. The goal was to move the public, the MPO advisory committees, and the MPO’s decision-making body (the MPO Policy Board) through these steps methodically in order to make the process and the decisions made during the process clear and accessible to all involved. The overall implementation of TCAPP took approximately 18 months, beginning in January 2013 and ending in May 2014. The time it took to implement TCAPP was comparable to the implementation of previous LRTPs for the MPO. However, given that TCAPP was a completely new LRTP development process, it would have been beneficial to take more time during implementation. With the deadline for the five-year update of the LRTP set for May 2014, the MPO focused its efforts on developing the elements of the planning process not previously seen by the community, specifically evaluation criteria and project scenario development. Figure 3.1 outlines the MPO’s approach to the development of its LRTP. The titles are how the MPO referred to each step; in parentheses are the TCAPP guide steps showing how the MPO’s process connected to the TCAPP guide. Once the goals for the LRTP were established, the MPO focused on the development of evaluation criteria and the identification of transportation deficiencies. Evaluation criteria provide the lens through which all project scenarios would be compared, and the identification of transportation deficiencies provided the starting point from which to develop new transportation projects. Once these deficiencies were identified, the MPO worked with its committees to determine which groups of projects would be assessed first as part of the scenario process. Assessments were iterative: each process resulted in finding new deficiencies that needed to be addressed. From there, new projects were identified and then assessed by using the evaluation criteria. The MPO went through this process three separate times over a six-month period in order to develop its preferred scenario of transportation improvements. This preferred scenario became the basis for the LRTP’s project list. The costs of the projects in the scenario were then compared to the amount of funds expected to be available—a process known as fiscal constraint—before the LRTP could finally be approved in May 2014. 13

Figure 3.1. LRTP process diagram. (Note that “performance measures” are synonymous with “evaluation criteria” and that Steps LRP-9 and LRP-11, in this case, were outside of the evaluation.) In addition, the second survey of MPO stakeholders (described in Task 2) also asked questions about the use of scenarios as part of the LRTP process, as this was the most challenging component for many of the stakeholders due to the fact that grouping projects was a new concept for the LRTP process. Task 4. Surveys of the General Public Surveys of the general public were conducted in the spring of 2013 and the spring of 2014. These surveys generally asked the public, as opposed to MPO stakeholders, which performance measures were most important and served to inform the results of the first three tasks. Semi- annually, CSR performs the Jefferson Area Community Survey (JACS). This omnibus survey asks a wide variety of questions, and CSR worked with the MPO to include its questions about evaluation criteria. This survey was implemented using CSR’s Computer Assisted Telephone Interviewing System. For the spring 2013 survey, CSR interviewed 1,013 individuals and had an overall survey response rate of 14.8% of qualified households. For the spring 2014 survey, CSR contacted 715 individuals and had an overall response rate of 17.0% of qualified households. Both surveys were focused on the six jurisdictions that make up the Thomas Jefferson Planning District, which the MPO is within. Compared with the surveys for Tasks 1 and 2, the questions for both implementations of JACS were more general and accessible, because they were for an audience 14

less familiar with the LRTP. For example, in the environmental areas, a JACS question was, “How important is the impact of the proposal on sensitive habitats, wetlands and areas near rivers or streams for you in deciding your support or opposition to any proposal?” By contrast, the initial MPO stakeholder survey had asked respondents about the importance of very specific measures such as “the distance of the project from sensitive habitats and riparian (stream) buffers” and the “lost acreage of wetlands.” In both JACS efforts, a triple-frame sample design was used. This sample design combines two landline-based contact methods and cell phone contact methods. Furthermore, both surveys weighted the results prior to analysis. This weighting was done to better reflect estimated distribution of phone service types in the region. The weighting also adjusted for gender, homeownership, and percentage of African-Americans, as reflected in the 2009 American Community Survey data for the region available from the U.S. Census Bureau. 15

CHAPTER 4 Results—The Most Influential Performance Measures As previously noted, the questions asked in the JACS and the initial stakeholder survey were not the same questions but were related in that both surveys sought to determine performance areas, or evaluation criteria, that were important to respondents. Figure 4.1 shows the evaluation criteria results from these initial surveys. Both groups ranked safety (JACS—Transportation Safety; stakeholder—Public Safety) as important when considering future transportation improvements. However, the rankings for congestion [JACS—Travel Time (Congestion); stakeholder—Passenger Mobility] were very different among the two groups, with the general public ranking congestion as the second most important evaluation criterion, while the MPO stakeholders ranked it as the fifth most important. For the remaining results, JACS response categories were related to stakeholder survey categories as bulleted below. These relationships are inferred based on the more detailed questions in the stakeholder survey. (For example, Question 2C of the JACS asked about the importance of impacts on “historical sites, parks and green space”; Questions 5a and 5b of the MPO stakeholder survey asked, respectively, about “historical and archaeological sites affected” and “acres of green space consumed,” both of which were categorized as “land use” impacts in the MPO stakeholder survey. Accordingly, the JACS response category, “Public Sites and Green Space” is aligned with the MPO stakeholder category “Land Use.”) • Schools, Minority and Low Income, Ease of Walking and Biking and Bus System (JACS): Social Justice and Community (stakeholder) • Air Quality and Sensitive Habitats (JACS): Environment (stakeholder) • Public Sites and Green Space (JACS): Land Use (stakeholder) • Budget (JACS): Economy (stakeholder) To review further results from these initial surveys, see Appendix A. 16

Figure 4.1. Results from the first round JACS (left) and first round stakeholder survey (right). Each chart in Figure 4.1 combines questions from its respective survey. In the JACS survey on the left, the total number of valid responses ranges from 665 responses to 747 responses across evaluation criteria. In the stakeholder survey on the right, the total number of valid responses was 44 responses for all evaluation criteria. Whereas the JACS had not differentiated between the more general performance measure areas and the more detailed performance measures within those areas, the initial MPO stakeholder survey did have such differentiation. Thus MPO stakeholders were asked about the importance of various areas as shown in Figure 4.1 (right) (e.g., land use versus the environment) and then, within each area, those respondents were asked about the importance of specific measures such as percentage change in pollutant emissions versus percentage change in greenhouse gases. As shown in Appendix A, there is greater variation in performance measures across different areas than there is within a given area. For example, consider the environment area, which 50% of respondents indicated as “extremely important” for transportation decisions; by comparison, for the economy area, this percentage was 18.2%. (When one considers respondents who indicated “extremely important” or “very important,” the environment area has a percentage of 84.1%, compared with 65.9% for the economy area.) Within each of these areas a greater percentage of respondents tended to rate the environmental performance measures as “extremely important” than was the case for the economic performance measures. Figure 4.1 shows modest variation within each area; for example, within the environmental area, almost 50% of MPO stakeholder respondents noted that “extent of waterways where pollutants would 17

exceed regulatory limits” was extremely important compared with 33% of MPO stakeholder respondents indicating that “lost acreage of wetlands” was extremely important (Figure 4.2). That said, generally the importance of the environmental measures (shown to the left of Figure 4.2) was higher than that of the economic measures (shown to the right of Figure 4.2). Figure 4.2. Percentage of MPO stakeholder respondents indicating that a measure was “extremely important.” Respondents were also asked the extent to which support from various groups would influence respondents’ support for or opposition to a particular transportation project. Eight groups were considered: elected officials at the state or local level, appointed officials, local government professional staff, organizations that advocate for environmental protection, organizations that advocate for economic development, neighborhood and community groups, local media (including newspapers, radio, television, local websites, or blogs), and major employers in the area. For each group, respondents indicated whether the group exerted a lot, a little, or no influence on respondents’ support for a given transportation project. There were only two groups where more than half of respondents indicated that the group would influence respondents’ support by “a lot”: organizations that advocate for environmental protection (where 63% of respondents indicated the group would have a lot of influence) and neighborhood and community groups (61%). By contrast, only about a third of respondents indicated that organizations that advocate for economic development (34%) or major employers in the area (32%) would exert a lot of influence. Roughly a quarter of respondents indicated that a lot of influence was held by local officials or elected officials. Figure 4.3 shows these percentages. (Based on an approximate statistical test, there is no significant difference among groups that are within 10 percentage points for this particular data set. This suggests, for example, that there is not a significant difference between “organizations that advocate for environmental protection” or “neighborhood and community groups”; however, there is a significant difference between the influence of these groups and the influence of “local government professional staff.”) 18

Figure 4.3. Percentage of MPO stakeholder respondents indicating that support from a group would influence respondents’ support or opposition for a particular transportation project “a lot.” Without individual respondent information, it is not possible to determine precisely if the proportions shown in Figure 4.3 are significantly different because p1 and p0 are not completely independent. An approximate test, however, can be applied based on the expression ( ) ( ) 41/1/ 0001 ppppZ −−= , where, for the sample of 41 respondents, a difference is declared if Z exceeds 1.96, p1 is the proportion of persons rating Group 1’s influence as “a lot,” and p0 is some fraction of interest, such as that proportion of persons rating Group 2’s influence as “a lot.” 0% 10% 20% 30% 40% 50% 60% 70% Organizations that advocate for environmental protection Neighborhood and community groups Local government professional staff Organizations that advocate for economic development Major employers in the area Elected officials Appointed officials Local media Pe rc en t o f R es po nd en ts No significant difference No significant difference No significant difference 19

CHAPTER 5 Results—The Influence of Performance Measures on Decision Making A second stakeholder survey focused on the degree to which evaluation criteria or performance measures can influence the support decision makers give to a scenario of projects. Beginning in February 2014, the University of Virginia’s Center for Survey Research (CSR) began implementation of this second survey. It aimed to describe how a specific grouping of transportation improvements, known as Scenario X, performed when analyzed through a series of seven specific performance measures. These were drawn from seven categories: • Environment • Land Use • Social Justice, Community, and Alternative Modes of Transportation • Economy • Public Safety • Passenger Mobility • Freight Mobility Survey respondents were provided numeric results specific to each performance measure for Scenario X, as well as a “no build” scenario. For example, the environment measure was “the amount of change in pollutants in stormwater.” Respondents could see that pollutants in storm water for the “no build” scenario amounted to 1,079.1 tons per year, while Scenario X resulted in 1,096.4 tons of pollutants per year, and that Scenario X would increase total tonnage of storm water pollutants per year by 1.6%. It should be noted that while the environmental measure showed an outcome that would be generally considered a worsening, the other six evaluation criteria comparisons showed either no change or an improvement of Scenario X over the “no build” scenario. Two of the survey questions for each measure asked respondents to make a direct comparison between the “no build” scenario and Scenario X. These questions asked respondents whether their positions on Scenario X would change if the value listed for Scenario X were smaller or larger than the original value. For example, these two questions were presented for the environmental measure: 1. The table [for the environmental measure] shows that this performance measure changes by 1.6% (17.3 more tons) for Scenario X. Suppose that it instead changed by 0.1% (10.3 more tons). Would that new information change your level of support? 2. Suppose this measure changed by 8.3% (89.6 more tons). Would that new information change your level of support? 20

Respondents were given the following options with which to respond: 1. Definitely 2. Probably 3. Probably Not 4. Definitely Not 5. Don’t Know/Not Sure The full results for this survey can be reviewed in Appendix B. The following sections in this chapter provide a general overview of outcomes for the influence of evaluation criteria. The first section describes the extent to which changes in performance measures influenced support for a scenario, and the second section examines reasons why this support varied by performance measure. Impacts of Changes in Performance Measures on Respondents’ Support for a Scenario For each performance measure, respondents were asked whether a change in the resulting value of the performance measure would influence their support for the transportation scenario. Table 5.1 suggests that about half (48%) of all respondents indicated that a change in the value of the performance measure would either “definitely” or “probably” influence their support for the scenario. Respondents’ tendencies to adjust their support varied with the performance measure and by whether the changed value was lower or higher than the original value shown in the survey for Scenario X. Variation between Values Higher than Scenario X versus Values Lower than Scenario X For each of the seven measures, a higher percentage of respondents indicated that the changed value would “definitely” or “probably” influence their support when the measure was higher than what was originally shown in the survey for Scenario X, and a lower percentage indicated a change in their support when the measure was lower than that originally shown for Scenario X. For example, when respondents considered that the environmental performance value was lower than what was presented for Scenario X, 36% of the respondents indicated that the changed value would “definitely” or “probably” influence their support for the scenario. By contrast, when the environmental performance measure value was higher than the original Scenario X value, 81% of respondents indicated that the measure would “definitely” or “probably” influence their support for the scenario (see Table 5.1). 21

Table 5.1. Percentage of Respondents Indicating That Changing the Value of the Performance Measure Would “Definitely” or “Probably” Influence Their Support for the Scenario Performance Measure Performance Measure Is Total Smaller Than for Scenario X Larger Than for Scenario X Environment 36% 81% 60% Land Use 35% 44% 40% Social Justice 38% 64% 51% Economy 44% 58% 51% Passenger Mobility 17% 28% 22% Freight Mobility 55% 57% 56% Safety 40% 67% 53% Average (all measures weighted equally) 37.94% 56.75% 47.58% Average (all responses weighted equally) 37.86% 57.08% 47.61% When considering all seven performance measures together, about half of the responses (48%) indicated that changing the value of the performance measure from its original Scenario X value would “definitely” or “probably” influence respondents’ support for a scenario. Just 38% of respondents, overall, indicated that a value smaller than the value presented for Scenario X would influence their support for the scenario. By contrast, 57% of respondents indicated that a value larger than the value presented for Scenario X would influence their support for the scenario. A paired t-test was used to determine the significance of these differences. The paired t- test calculates the average response for all the performance measures, giving them equal weight, and compares them. The paired t-test for this study indicates that the two overall percentages calculated (i.e., the “Smaller Than for Scenario X” and the “Larger Than for Scenario X,” where each performance measure carries equal weight) are significantly different (the test resulted in p = 0.02; generally, p-values of 0.05 or below are considered significant). Variation among Performance Measures For some of the performance measures, there was variation in the observed percentage of responses. This variation indicates that a change in the value of the measure would influence support for a scenario. For this analysis, a 95% “confidence interval” was calculated to more accurately compare the degree of influence for each measure. The meaning of this confidence interval is that if the survey were repeated many times, then 95% of the calculated confidence intervals would contain the “true” percentage (Montgomery 2001). For example, for passenger mobility, Figure 5.1 shows an observed percentage of 22%, with a 95% confidence interval of 12% to 33%. In theory, if this survey were conducted many times, then 95% of the confidence intervals calculated in this manner would contain the true percentage. Because this survey was only conducted once, another interpretation is that we are 95% confident that the true percentage 22

for passenger mobility is between 12% and 33%. The 95% “confidence interval” was calculated with the formula n/)pˆ1(pˆzpˆ −± , where z = 1.96, and n = 58 responses (see Figure 5.1). The 95% confidence intervals for five of the measures—social justice, economy, safety, freight mobility, and environment—are quite similar. For an example, consider the 65 responses for social justice (32 of which responded to a smaller value than for Scenario X and 33 of which responded to a larger value than for Scenario X). While the percentage of responses saying that a change in this measure would “definitely” or “probably” influence support for a scenario (51%) is nominally lower than the corresponding percentage for the environment (60%), the confidence intervals for these percentages show a large degree of overlap (Figure 5.1). [A rough test of significance suitable for comparing two different percentages—the normal approximation to the binomial—further suggests that there is not a significant difference between these five measures. That is, this test compares the percentages of 51% (observed for social justice) and 60% (observed for the environment) and, based on the number of respondents for each performance measure, yields a p-value of 0.30, which is greater than 0.05, and hence indicates that the percentages are not significantly different. The authors recognize that this is not a perfect test of significance, because these two percentages of 51% and 60% are not drawn from different (independent) samples of respondents but rather the same sample of respondents.] Figure 5.1. 95% confidence intervals for percentage of respondents who indicated that changing the value of the performance measure would “definitely” or “probably” influence their support for the scenario. (Example: 22% of responses indicated that changing a value of “passenger mobility” would definitely or probably influence their support for the scenario.) %,22pˆ = 23

By contrast, the confidence interval for the passenger mobility measure does not overlap with the intervals for the five measures mentioned above. Further, the percentage of responses indicating that a change in the value of the passenger mobility measure would influence their support for a scenario (22%) was significantly lower than that of the next lowest measure (land use, with a value of 40%). Figure 5.1 suggests three groupings in terms of measures’ influence on respondents’ support for a scenario: passenger mobility (lowest at 22%), land use (second lowest at 40%), and then the remaining five measures (means ranging from 51% to 60% with a mean of these means of 54%). That said, there is some variation in this latter grouping of five measures. For example, the rough test of significance mentioned previously—the normal approximation to the binomial—shows no difference between land use and social justice (p = 0.21) yet a significant difference between land use and the environment (p = 0.02). Thus it is conceivable that one could devise additional categories. However, as a visual trend, the three categories described here appear to be a reasonable interpretation. Explanation of Why Support Varied by Performance Measure The quantitative assessment presented above shows the extent to which changes in the outcome of a performance measure affect the support a respondent may have for Scenario X. There are several potential reasons for this variability by performance measure. The second stakeholder survey allowed respondents to include open-form responses about each of the seven performance measure questions, and these responses suggest three observations about the performance measures: 1. Respondents may have focused more on percentage differences than on absolute differences. 2. Percentage differences explain some, but not all, of the difference in performance measure influence. 3. Respondents may have needed additional interpretation for some of the measures. Each of these possibilities is described in more detail below. Percentage Differences versus Absolute Differences Each survey question regarding performance measures included introductory text that compared the “no build” scenario to Scenario X. This text included both the overall value of the performance measure and the percentage change between Scenario X and the “no build” scenario. Further, the questions that focused on the influence of the measure (an example of which is included in this chapter’s introduction) also included both the value and the percentage change. Despite providing both the absolute change and the percentage change, respondents seemed to associate only the size of the percentage change with the degree of change. In some 24

cases, a small percentage change in the performance measures between the smaller or larger option and Scenario X actually resulted in a relatively large absolute change between Scenario X and the smaller or larger option. The second and third columns of Table 5.2 repeat information from Table 5.1 (the percentage of respondents who indicated that the changing value of the performance measure would “definitely” or “probably” influence their support for the scenario). The remaining columns show the percentage differences from “no build” for Scenario X and the “smaller” and “larger” questions, where applicable. (For two of the measures, percentage change was not calculated because of the nature of the measure.) These results suggest that some respondents focused primarily on the percentage change. Table 5.2. Comparison of Responses Indicating Changed Support for the Scenario and the Percentage Change Values Presented in the Survey Questions Performance Measure “Definitely” or “Probably” Responses When Performance Measure Is Percentage Change from "No Build" Scenario Smaller Than Scenario X Larger Than Scenario X Scenario X Smaller Option Larger Option Environment 36% 81% 1.6% 0.1% 8.3% Land Use 35% 44% 2.5% 1.4% 3.6% Social Effects 38% 64% 19% 13.5% 24.5% Economy 44% 58% 1.0 C/B Ratio 0.75 C/B ratio 1.25 C/B ratio Passenger Mobility 17% 28% 0.4% 0.2% 0.6% Freight Mobility 55% 57% 1* No change 3* Safety 40% 67% 0.6% 0.1% 1.1% *Reduction of at-grade auto-rail crossing Indicates less success than Scenario X Indicates more success than Scenario X Note: C/B = cost/benefit. The open-ended responses also indicated that there may have been some preconceived opinions about the size of a percentage change that would be significant. For example, within the survey one of the respondents noted that “I would likely change my response to ‘definitely’ if the percentage change were 5% or higher. I believe that a change less than 5% won't pique much interest from the user.” Other open-ended responses also indicated that the percentage change between the smaller value, larger value, and Scenario X was not sufficient to shift the respondent’s decision. Of the total 86 open-ended comments recorded through the survey process, 26 of the comments focused on the relevance of the size of the percentage change or the nature of the percentage change statistic. These responses suggest that there was some difficulty in keeping the percentage change information in perspective. It should be noted that many respondents submitted the same comment across multiple questions, which is why the total number of comments (86) is larger than the total number of respondents (41). 25

Percentage Differences Explain Some, but Not All, of the Difference in Performance Measure Influence The environmental performance measure provides a good example of where the observed percentage change affected the respondent’s position regarding Scenario X. For example, 36% of the respondents said they would “definitely” or “probably” change their position with regard to Scenario X if there was just a 0.1% increase in the total tons of stormwater pollutants per year, an improvement upon the original Scenario X (which was shown as causing a 1.6% increase). In contrast, 81% of the respondents would “definitely” or “probably” change their position on Scenario X if the scenario showed an 8.3% increase in the total tons of stormwater pollutants per year: a worsening compared with Scenario X, and a much larger change when compared with the smaller option of 0.1%. For this measure especially, but also for some of the other measures, there was a dramatic difference between the percentage of respondents whose positions would be affected by a change in the measure’s outcome and the perceived degree of change in performance between Scenario X and the smaller or larger alternative. Figure 5.2 aims to portray the strength of this relationship. For Figure 5.2 the vertical axis shows the percentage of respondents who would “definitely” or “probably” be influenced by a change in a performance measure from the result originally presented for Scenario X. The horizontal axis shows the difference between the value for Scenario X and the values of the smaller and larger options (these values are represented in Table 5.2). For the purposes of this analysis all the values on the horizontal axis are represented as positive values. The term for the values on the horizontal axis is the absolute difference (in percent) between Scenario X and the smaller or larger option. If it were the case that percentage differences explained all or even most of the difference in the influence of performance measures on stakeholders’ support for the scenario, the points in Figure 5.1 would be tightly clustered near a diagonal line from the lower left side of the chart to the upper right side. While there is some evidence of such linearity (especially shown by the two results for the environment), it is far from definitive. Quantitatively, a correlation analysis of all the measures represented suggests that the difference from Scenario X explains about 39% of the difference in the percentage of respondents influenced by the performance measure. That is, between one-third and one-half of the variation in percentages shown in the second and third columns of Table 5.2 is explained by the difference between the result for Scenario X and the smaller or larger results computed from the right three columns of Table 5.2. 26

Figure 5.2. Percentage of respondents influenced by a measure as a function of the percentage change in the measure. Brown indicates less success than Scenario X; green indicates more success than Scenario X. Figure 5.2 excludes two measures that are somewhat anomalous in this regard: the economy measure, as it is a benefit–cost ratio, and the freight mobility measure, as its “smaller” option shows no change from the “no build” condition. Respondents May Have Needed Additional Interpretation for Some of the Measures One other relevant observation arose from the passenger mobility measure, which focused on shifting regional transportation trips from automotive trips to transit, bike, and pedestrian modes. For this measure (percentage of trips by auto), the smaller and larger values suggested (0.2% and 0.6%) were only a slight percentage change from the given value for Scenario X performance (0.4%). When considering the total shift in regional trip types, though, these small percentage changes result in a nontrivial number of trips shifting to alternative modes. It is worth noting that, for this measure, trips were shown as a percentage of trips by auto, not an overall number of trips, which may have confused respondents. Possibly because the percentage change was perceived as being very small, this measure had very little influence on respondents, with only 17% changing their position with the smaller 27

option, and 28% changing their position with the larger option. It appears that few respondents understood that the measure was assessing total daily trips, and that a small percentage change in auto trips corresponded to a substantial number of trips shifted to transit, bike, and pedestrian modes. Throughout the 2040 LRTP process, this measure was listed as a percentage but was represented to stakeholders as the total number of trips. It does not appear that this translated clearly within the survey. Reference Montgomery, D.C. Introduction to Statistical Quality Control, 4th Edition, John Wiley & Sons, Inc., Hoboken, N.J., 2001. 28

CHAPTER 6 Results—TCAPP (PlanWorks) Assessment As previously stated, the Charlottesville-Albemarle MPO’s implementation of the TCAPP process began in January 2013 and ended in May 2014. The process covered the long-range planning steps from the TCAPP Decision Guide listed in Table 6.1. Steps LRP-9 and LRP-11 were omitted from the process because they focused on air quality conformity analysis, which was not a part of the LRTP process because the Charlottesville-Albemarle MPO is located in an attainment area. Table 6.1. LRP Portion of the TCAPP (PlanWorks) Decision Guide STEP TITLE LRP-1 Approve Scope of LRTP Process LRP-2 Approve Vision and Goals LRP-3 Approve Evaluation Criteria, Methods and Measures LRP-4 Approve Transportation Deficiencies LRP-5 Approve Financial Assumptions LRP-6 Approve Strategies LRP-7 Approve Plan Scenarios LRP-8 Adopt Preferred Plan Scenario LRP-9 Adopt Finding of Conformity by MPOa LRP-10 Adopt LRTP by MPO LRP-11 Approve Conformity Analysisa a Omitted because they focused on air quality conformity analysis. Source: http://transportationforcommunities.com/shrpc01/framework_application_kdps/21/0. By January 2013, when the formal implementation of the TCAPP process began, several components of the LRTP were already under development, including the core goals of the plan, evaluation criteria, and a list of potential projects. The LRTP had been under development for about a year prior to the implementation of the TCAPP process. Before implementing TCAPP, the MPO had a strong handle on what decisions needed to be made in order to update the LRTP but had not yet developed a process for achieving these defined decisions. Elements of these key decisions such as goals and project lists had been brought to the MPO stakeholders, but these items had been crafted and reviewed in a somewhat haphazard fashion, which made it challenging for stakeholders and staff to appreciate the overall process and endgame for the LRTP. There were clearly elements of the planning process that needed to be accomplished, but the process for accomplishing those requirements was not clearly mapped. The TCAPP Decision Guide helped the MPO develop a process for its LRTP. While it should be noted that the TCAPP Decision Guide is not a “how-to,” step-by-step process for all MPOs to follow when completing a long-range transportation plan, it does provide the key 29

decisions. The guide implies following these steps successively, but does not mandate it, and in this MPO’s case, following each step successively was not an option because the MPO was unable to begin its fiscal-constraint process (LRP-5) until December 2013. The TCAPP guide outlines the decisions that need to be made to bring a transportation project to fruition, but it allows for flexibility. The following section outlines the MPO’s process for implementing the TCAPP Decision Guide. This section is structured around each of the steps in the guide. Each step includes a description of the step, the MPO’s experience in implementing the step, and the lessons learned from the implementation, including success and failures. LRP-1: Approve Scope of LRTP Process The first step in the TCAPP Decision Guide concerns the process issue that the MPO struggled with most in the development of the LRTP: to develop a process for developing the plan and to gain consensus that this process is appropriate. TCAPP Description The purpose and expected outcome for this first step is described in Table 6.2. Table 6.2. TCAPP Step 1—Purpose and Outcome Purpose Outcome To develop a common understanding and reach agreement on the LRTP process, including all information relevant to transportation, community, and the environment. This includes stakeholders to engage; roles and responsibilities; tools and data sources to be used; time frames; and public involvement plan. Documented agreement on the LRTP process and its elements. This agreement can be used as a foundation when starting the corridor planning and environmental review processes. Confirmation that the transportation process is in agreement with the larger community plans and programs. Source: http://transportationforcommunities.com/shrpc01/kdp_step/21/0/1. MPO Process When the MPO was pursuing this pilot test SHRP 2 funding in the fall of 2012, staff took that time to inform MPO stakeholders about the MPO’s intent to pursue this opportunity and about the MPO’s intent to use the TCAPP Decision Guide as the anchor points for the development of the LRTP. However, once the project was awarded and the actual implementation of TCAPP began, the MPO did not revisit the process directly with its stakeholders. While the MPO did outline the intended steps for the LRTP process, staff did not pursue consensus on the process, nor did staff directly relate the process to TCAPP. This communication failed because the MPO was still in the process of developing its first round survey for its stakeholders and was concerned that providing too much information to these stakeholders about TCAPP could affect the results of those surveys. In January 2013, the MPO was still developing these surveys with its project partners. As the implementation of 30

TCAPP began, the focus of the survey process was unclear. Given this uncertainty, MPO staff avoided discussing the topic out of concern that too much discussion about TCAPP could affect the survey results. Successes and Challenges The MPO stakeholders were supportive of the MPO’s pursuit of funding for this project in the fall of 2012. Assessing the LRTP process and developing clear decision-making anchor points for the process was something that stakeholders, especially those who had participated in the development of previous LRTPs, thought would be beneficial to the process. In general, the concept of implementing TCAPP to help anchor the LRTP was appreciated. The MPO chose to limit the amount of information provided to the MPO stakeholders about the implementation of TCAPP. There were concerns from MPO staff that providing too much information about TCAPP could affect the assessment process that the MPO was still developing. Because the MPO did not fully explain the implementation of TCAPP, there were many challenges later in the process that could have been avoided had TCAPP been explained more completely. Upon review, the MPO did not implement LRP-1 of the TCAPP guide to its fullest extent. The decision not to do so was related to the overall assessment process for TCAPP, but that decision resulted in subsequent confusion about the overall process. LRP-2: Approve Vision and Goals The second step in the TCAPP Decision Guide focuses on the development and approval of the vision and goals for the LRTP. The vision and goals element is typically focused on “How do you want the region’s transportation system to function in the next 20 years?” TCAPP Description The TCAPP Decision Guide describes the purpose and outcome for this second step (Table 6.3). Table 6.3. TCAPP Step 2—Purpose and Outcome Purpose Outcome To develop a common, comprehensive set of vision and goals for the planning area that incorporate the vision and goals from previous or existing plans, if applicable. Where no community vision and goals exist, transportation-specific vision and goals consistent with community values. Where a regional community vision and goals exist, transportation- specific goals for the planning area consistent with the regional vision and goals. Source: http://transportationforcommunities.com/shrpc01/kdp_step/21/0/2. MPO Process The development of the vision and goals for the MPO’s 2040 LRTP began about a year prior to the implementation of TCAPP. The Charlottesville-Albemarle MPO has put a tremendous 31

amount of effort into the development of LRTP goals during the previous two LRTPs. These goals, called the “Regional Mobility Goals,” have been key elements in the MPO’s previous LRTPs. With the update of the 2040 LRTP, MPO staff did not move away from these previously crafted goals. Staff worked with MPO stakeholders to refine and update these goals. While there was a considerable amount of restructuring, the momentum of the final goals and the vision they created remained similar to what was included in previous plans. The Regional Mobility Goals were redeveloped with MPO stakeholders for about a year but were not approved by the MPO stakeholders officially until January 2013. Key elements of the Regional Mobility Goals include development of a multimodal network, a cohesive relationship between land use and transportation planning, and a more structured and proactive method for pursuing transportation funding. Also, as part of all LRTPs, MPOs must consider the “Eight Planning Factors” outlined in federal transportation legislation. These factors focus on a wide variety of planning issues involving economic vitality, safety, security, accessibility, environment, connectivity, efficiency, and preservation of the existing system. The MPO’s LRTP was guided by both the Regional Mobility Goals and the Eight Planning Factors. Successes and Challenges The historical precedent for the MPO’s LRTP goals further cemented a regional transportation vision that has been crafted and developed for over 10 years. Having the LRTP process rooted in that vision helped unite MPO stakeholders on the transportation needs for the region. A firmly rooted vision did not avoid debate among stakeholders about which projects would most appropriately achieve MPO goals. It did, however, give all stakeholders the same ends to strive for. Because the MPO stakeholders had worked with these goals for some time, they were easily understandable to the MPO’s stakeholders, as well as to the general public. The MPO struggled to relate its goals and performance measures in the way that is envisioned in TCAPP. While the Regional Mobility Goals and Eight Planning Factors were both firmly rooted in the community’s planning practice, they were not developed anew with the implementation of TCAPP. Had the development of the goals coincided with the implementation of TCAPP, they might have fit the TCAPP decision structure more appropriately. The vision and goals component of the TCAPP guide relates directly to the third step in the guide, which considers evaluation criteria, measures, and methods. The evaluation criteria, which serve as the lens through which various projects are viewed and compared, are meant to be generated from the goals and visions that are developed in Step 2. Because the MPO’s goals and vision were developed as part of a non–TCAPP-related process, the goals are not structured in a way that allowed the MPO to easily and seamlessly develop related evaluation criteria. For example, the Regional Mobility Goals focused solely on the development of a multimodal transportation vision. Objectives from these goals include working with the Virginia Department of Transportation (VDOT) to create more flexible roadway designs for more balanced multimodal 32

performance and continuing to work on the development of a regional transit authority. While these objectives clearly contribute to the region’s overarching vision of a multimodal transportation system, they do not easily lend themselves to the development of performance measures that foster objective comparison between projects. LRP-3: Approve Evaluation Criteria, Methods and Measures The third step in the TCAPP Decision Guide focuses on the development of evaluation criteria and performance measures that allow LRTP stakeholders to compare project scenarios. The development of performance measures for assessing and benchmarking transportation improvements is becoming standard practice for transportation planning. TCAPP Description The TCAPP Decision Guide describes the purpose and outcome for this third step (Table 6.4). Table 6.4. TCAPP Step 3—Purpose and Outcomes Purpose Outcome To provide a reasonable and measurable basis for • The creation of scenarios that represent the identified strategies and • The comparison of scenarios in order to select the preferred scenario. Evaluation criteria and measures should include consideration of the identified goals and objectives for the plan and vision for the region. The specific criteria, methods, and requirements that will be used to compare strategies and scenarios so that the adopted plan will meet the approved goals for the planning area. Source: http://transportationforcommunities.com/shrpc01/kdp_step/21/0/3. MPO Process Prior to implementing TCAPP, the MPO felt it was important to use evaluation criteria in this iteration of the LRTP. Evaluation criteria that were under development focused mainly on the MPO’s Travel Demand Model: a tool that the MPO spent a year and half updating and calibrating, specifically for developing the LRTP. The MPO was also in the process of contributing to another project, which focused on the development of evaluation criteria on a community-wide scale. Through this project, the MPO developed several other transportation- specific evaluation criteria that included changes in travel time to work, changes in the number of crashes, and changes in access to transportation facilities (e.g., bus system) (http://www.tjpdc.org/livablecommunities/PerformMeasuresReport.pdf). These previously developed criteria were altered slightly to function as evaluation criteria for the MPO’s LRTP project scenarios. As the MPO began its implementation of TCAPP, staff researched potential evaluation criteria and performance measures by using the resources available through SHRP 2. A separate SHRP 2 project called Performance Measurement Framework for Highway Capacity Decision 33

Making was a key resource for the MPO in developing additional performance measures; especially those that focused on environmental and community impacts (http://shrp2webtool.camsys.com/Default.aspx). Using the Performance Measure Framework as a resource, the MPO was able to develop measures that considered stormwater runoff impacts, air quality impacts, existing land use impacts, and historical and archeological site impacts. While the MPO did not use the measures discussed in the framework exactly as they were described, the framework did provide inspiration and resources for how the MPO could create similar resources for its LRTP process. Through this development process the MPO created 16 distinct evaluation criteria that covered four key measurement areas (Table 6.5). These four areas are mobility, economy, environment, and community. Table 6.5. TCAPP Evaluation Criteria Used in the Charlottesville-Albemarle MPO’s LRTP Measure Description MOBILITY Congestion The total percentage of roads that will have a level of service E or F in 2040. Delay The total daily hours of delay that congestion will cause in the year 2040. Mode Share The percentage of trips across the four main travel modes; i.e., automotive, transit, bike, and walk for 2040. Vehicle Mobility The total systemwide vehicle miles traveled for 2040. Vehicle Crashes The total systemwide crashes per year for 2040. Bicycle Connectivity The total percentage of bikeable roads in the urban area. ECONOMY Access to Jobs The average travel time to work. Transit Accessibility The percentage of population and the percentage of employed individuals within the MPO with access to transit. ENVIRONMENT Habitat The aggregate impact of projects on natural resources and habitats within 500-foot buffer of project. Air Quality The percentage change in air quality gases and particulates in tons per year. Water Quality The percentage change in the amount of stormwater pollutants in tons per year. Floodplain The total acreage of floodplain within a 500-foot buffer of the projects. Historical/Archeological Sites The total number of historic or archeological sites within a 500-foot buffer of these projects. COMMUNITY Land Use The total number of land parcels within a 500-foot buffer of the potential projects by usage: residential, commercial/industrial, parks, education/religious/charitable, and agricultural/undeveloped. Environmental Justice and Title VI: Transit Access The total percentage of environmental justice or Title VI groups with access to transit: minorities, 65 and older, limited-English speaking, and household income of less than $25,000. Environmental Justice and Title VI: Impacts The total percentage of environmental justice or Title VI groups potentially impacted due to projects: minorities, individuals aged 65 and older, limited-English speaking, and household income of less than $25,000. 34

The 16 measures developed served as the lens through which all transportation scenarios were viewed and compared. This LRTP update was the first time the MPO explicitly utilized evaluation criteria in the development of its LRTP. Successes and Challenges The implementation of evaluation criteria as part of the LRTP process has been extremely useful in grounding the decision-making process in objective analysis. The MPO wanted to use evaluation as part of this process because previous LRTPs were not clearly grounded in technical assessment. In many cases, the discussion surrounding potential transportation improvements was driven more by political concerns than by regional benefits. The use of evaluation criteria allowed the discussion to focus more directly on the impacts of project scenarios. It also allowed the MPO stakeholders and the public to compare scenarios more objectively. As the MPO was developing evaluation criteria for the first time in its history, stakeholders made recommendations on elements that should be considered. While these contributions were useful, many of them were impossible to implement because of lack of data or time constraints for analysis. It was difficult to convey to the stakeholders that some suggestions simply could not be accomplished. Another major challenge for developing these criteria was finding data to complete the analysis. For example, reliable crash data were difficult to acquire and manipulate into a useable format for the evaluation process. Finally, as mentioned in the vision and goals section, the MPO struggled to relate its evaluation criteria to its regional goals and objectives. The goals for the LRTP have a historical precedent in our community and focus on vision but did not relate seamlessly to essential evaluation criteria. Unfortunately for this process, there is a disconnect between the 2040 LRTP’s goals and vision, and the evaluation criteria. LRP-4: Approve Transportation Deficiencies The fourth step in the TCAPP Decision Guide focuses on identifying transportation deficiencies or issues with the existing transportation system. All LRTPs must first assess existing problems with the regional transportation system before they identify future projects and solutions for improving the system. TCAPP Description The TCAPP Decision Guide describes the purpose and outcome for this fourth step (Table 6.6). Table 6.6. TCAPP Step 4—Purpose and Outcome Purpose Outcome To identify transportation deficiencies within the planning area that should be addressed in the LRTP. Transportation deficiencies are where the current or future system is expected to experience congestion, safety issues, lack of interconnectivity, A list of specific corridors, roads, and areas that are deficient or need improvement. 35

or other operational problems, as well as inadequate roadway capacity. Source: http://transportationforcommunities.com/shrpc01/kdp_step/21/0/4. MPO Process The MPO began identifying regional transportation deficiencies about six months prior to the implementation of TCAPP. For roads, the process focused on identifying future sites of major and minor congestion issues, using the MPO’s Travel Demand Model. The MPO defined congestion using traffic volume-to-roadway-capacity standards, which are identified for urban areas by the VDOT. The MPO used its travel demand model and these criteria to determine regional roadways that would have the biggest congestion issues in the year 2040. The MPO determined transit deficiencies based on access to transit. MPO staff assessed how much of the MPO’s population and employment, projected for the year 2040, were within a quarter-mile radius of a bus stop. The assessment allowed the MPO to identify areas that would have a significant population or employment density, but would lack transit access. For bicycle deficiencies, the MPO’s assessment process was based in identifying areas that lacked connectivity. MPO worked with local staff and the public to identify major regional barriers to bicycling. Then, the MPO used existing facilities and additional roadways deemed to be safe based on roadway speed, vehicular volume, and facility type to determine if and how these barriers could be navigated. The MPO used this data to create “bubbles of bikeability,” which rendered regional bike connectivity, allowing the MPO to focus on less connected areas. Pedestrian deficiencies were identified by using a similar analysis as was used for transit. The MPO looked at how much of the MPO’s 2040 population and employment would have access to pedestrian facilities, this time looking at a distance of only 200 feet. As with transit, this analysis allowed the MPO to determine how much of the region’s future population and employment would have reasonable density, but limited access to pedestrian facilities. Successes and Challenges The MPO stakeholders found it helpful to use the identification of deficiencies as a starting point for identifying which projects should be considered for inclusion in the 2040 LRTP. However, many stakeholders felt that the transportation deficiencies identified should have been more diverse. Assessing only congestion-related deficiencies for roads, and only accessibility-related issues for transit, for example, resulted in missed opportunities for addressing other issues. Furthermore, some stakeholders felt that the transportation deficiencies were not comparable across modes. The analysis for road congestion did show a considerable number of congestion- related issues in the region in 2040, while the transit analysis showed that a significant number of regional employment centers had access to transit. Some stakeholders felt that the analysis advocated focusing resources on road improvements rather than transit improvements. While the MPO developed its transportation deficiencies analysis before it began implementing TCAPP, the MPO was familiar with the TCAPP process, and used it as a resource. 36

LRP-4 mentions a variety of deficiencies (e.g., safety, connectivity, and operation issues) that may be identified, but it focuses primarily on the identification of roadway and corridor deficiencies. The MPO attempted to look at transportation deficiencies from a multimodal perspective to more closely parallel the Regional Mobility Goals, but there was still a perceived slant toward roadways. It was difficult to diversify this analysis, and the focus on roads frustrated many of the MPO’s constituents. LRP-5: Approve Financial Assumptions The fifth step in the TCAPP Decision Guide focuses on introducing the fiscal-constraint process into the long-range transportation planning process. All LRTPs must be fiscally constrained, meaning an MPO cannot plan to spend more funding than it is expected to receive over the life of its LRTP. TCAPP Description The TCAPP Decision Guide describes the purpose and outcome for this fifth step (Table 6.7). Table 6.7. TCAPP Step 5—Purpose and Outcome Purpose Outcome To reach agreement on potential revenue source, the restrictions and requirements for allocating revenue and the methodology for identifying costs so that scenarios can be fully considered. Approved project cost methodology and revenue sources for evaluation and comparison of scenarios. Source: http://transportationforcommunities.com/shrpc01/kdp_step/21/0/5. MPO Process The MPO addressed the two main components of LRP-5 at separate times in the LRTP process. The first component considers the overall expected funding for the region, specifically focusing on the sources of this funding and how much the region is expected to receive. The second component guides the development of cost estimates for projects under consideration. Regarding the first component: in the state of Virginia, MPOs must work with the VDOT to develop overall funding estimates for their LRTPs. This is a standard process for fiscal constraint. The MPO is bound to the VDOT’s schedule in gathering this information, which caused some delays in the LRTP fiscal-constraint process. In 2013, Virginia’s General Assembly voted on a complete overhaul of how the state gathers transportation revenue. Coupled with changes related to new federal transportation legislation, revenue reform resulted in the VDOT having to completely redevelop transportation funding forecasts for many MPOs across the state. Because of this redevelopment process the MPO did not receive its estimated LRTP funding forecast until December 2013. The MPO could not begin the fiscal-constraint process until that time. The second component of LRP-5 focused on the development of project cost estimates. Prior to the MPO’s implementation of TCAPP, it was clear that there were several major 37

roadway and transit improvements that the community wanted to see evaluated in the Long- Range Transportation Planning process. Every community has transportation project priorities that surface and gain momentum every few years regardless of their feasibility or cause of controversy. The Charlottesville-Albemarle MPO is no exception. Prior to the implementation of TCAPP, MPO stakeholders went through a process where they worked with MPO staff to develop a candidate list of potential improvements to be considered for the 2040 LRTP. This process resulted in a list of 21 projects: 10 road projects and 11 transit projects. After preliminary steps to narrow and consolidate this list, the MPO was left with a list of 15 projects: seven road projects and eight transit projects. The MPO developed cost estimates for these 15 projects. For the road projects, the MPO used VDOT’s Planning Level Estimation Spreadsheet, which was developed by VDOT to determine cost estimates early in the transportation planning process. The MPO worked with VDOT staff and local staff to ensure that the estimates were appropriate and reasonable based on the project descriptions. For transit improvements, the MPO had to build a cost-estimation spreadsheet from scratch. The MPO worked with local transit staff and staff from the Virginia Department of Rail and Public Transportation to develop this cost-estimation tool with data from local, state, and federal resources. The MPO also had to divide the transit-related cost estimates into three distinct components: operating costs, capital costs, and infrastructure costs (i.e., bus- only lanes). The estimation spreadsheet for transit improvements took three months to development and implement. Successes and Challenges As mentioned in the process section, the MPO was not in control of the development of its overall LRTP funding estimate. Therefore, the MPO was unable to fully consider fiscal constraint until after the approval of the 2040 LRTP preferred scenario (LRP-8). While this step was not completed in the order outlined in the TCAPP Decision Guide, it was completed in the same order as it had been with previous LRTPs; toward the end of the process, MPO stakeholders and members of the public did have some concerns about the timing of the fiscal- constraint process. Many stakeholders had concerns about the MPO developing an LRTP that it then could not afford. If the MPO had been in a position to develop the fiscal-constraint component earlier, as TCAPP suggests, these issues may have been avoided. LRP-6: Approve Strategies The sixth step in the TCAPP Decision Guide focuses on the development of strategies for addressing the transportation deficiencies identified through LRP-4 so as to develop a future transportation system that functions as efficiently as possible. TCAPP Description The TCAPP Decision Guide describes the purpose and outcome for this sixth step (Table 6.8). 38

Table 6.8: TCAPP Step 6—Purpose and Outcome Purpose Outcome To develop and evaluate groups of strategies relative to stated needs. A range of strategies to address transportation deficiencies and achieve vision and goals. Source: http://transportationforcommunities.com/shrpc01/kdp_step/21/0/6. MPO Process For the MPO’s implementation of TCAPP, Steps LRP-4, LRP-6, and LRP-7 (Approve Plan Scenarios) worked in tandem. First, the MPO assessed transportation deficiencies and developed strategies for addressing them. The MPO’s plan for implementing TCAPP was developed around three main steps: (1) determine transportation deficiencies (LRP-4); (2) develop strategies or projects to address said deficiencies (LRP-6); and (3) analyze the strategies using evaluation criteria (LRP-7). Once the evaluation criteria analysis was completed, the MPO worked with its stakeholders to identify additional transportation deficiencies and began another iteration of that process. The MPO implemented three rounds of this three-step process, lasting from April to October 2013. Round one of this scenario development and analysis process was meant to serve as an example of the process for the MPO stakeholders. For this first round only, MPO staff developed the scenarios of projects. Using the list of 15 projects that had been previously identified, MPO staff developed three different scenarios. One considered only the road projects; one considered only the transit projects; and one considered both road and transit projects combined. The reason these scenarios were structured in this manner was twofold: the first was to show how the scenario process worked, and the second was to illustrate the extremes of implementing only road projects, only transit projects, or a mixture of the two. Analyzing these three distinct options (road-only, transit-only, or multimodal) allowed the MPO stakeholders not only to learn about the analytical approach the MPO was taking to implement its LRTP, but also to gain some insight about project groupings. In Round 1, the multimodal options that considered both roadway and transit improvements ended up being the most valuable to the region. The stakeholders learned that the implementation of a multimodal scenario was the best option for the future regional transportation system. After this first round, the MPO completed Rounds 2 and 3 of the process. In these rounds, MPO stakeholders and the public developed the project scenarios that would be analyzed. This process was completed with the development of a draft preferred scenario in September 2013. The preferred scenario was developed based on the assessment of projects from the previous three scenarios. Throughout the process the scenarios tended to clarify which projects showed the most potential. With each scenario iteration, the project groupings became clearer. By the time the process led into the preferred scenario it was clear which projects had the most potential impact. 39

Successes and Challenges With the implementation of TCAPP, the MPO, for the first time in developing its LRTP, approached the development and selection of candidate projects by using a highly methodical approach. Using a process that MPO stakeholders could generally relate to resulted in a more thoughtful and technically grounded approach to the development of the LRTP. With previous plans, the process for choosing projects was more politically driven and based on assumed needs. The TCAPP process diffused some of the uncertainty surrounding those assumptions. In the TCAPP guide, LRP-6 and LRP-7 both encourage the development of strategies for addressing transportation deficiencies and creating project scenarios. This process meant analyzing and assessing multiple projects instead of considering each project individually. While many MPO stakeholders appreciated a more analytical approach, others balked at the use of project scenarios. Some stakeholders were confused as to why projects were being considered in groups, and some elected officials outside the MPO Policy Board felt that using project scenarios was misleading for developing regional transportation improvements. MPO staff did work to explain that the transportation network is a system and that it is important to consider improvements in a holistic manner. While many did begin to see the value in developing and analyzing project scenarios, there were still some who were not comfortable with the concept, even after the preferred scenario was selected. LRP-7: Approve Plan Scenarios All LRTPs require stakeholders to make choices and trade-offs when considering which projects should move forward in the LRTP process. LRP-7 is about reviewing those trade-offs and making those choices. The seventh step in the TCAPP Decision Guide compares the strategies or project scenarios developed in LRP-6. This focuses on providing objective comparisons between scenario options. TCAPP Description The TCAPP Decision Guide describes the purpose and outcome for this seventh step (Table 6.9). Table 6.9. TCAPP Step 7—Purpose and Outcome Purpose Outcome To identify plan scenarios for testing and comparison in order to select a preferred plan scenario for the region. The scenarios are designed to address the approved deficiencies. This begins the iterative analysis that is conducted for a full understanding of the trade-off decisions necessary to identify the preferred plan scenario. Scenarios should be identified in terms that can be easily understood by the decision makers, planning partners, and stakeholders. A list of feasible plan scenarios. Source: http://transportationforcommunities.com/shrpc01/kdp_step/21/0/7. 40

MPO Process As stated in the LRP-6 summary, LRP-7 is part of a three-step analytical progression that was the heart of the MPO’s long-range transportation planning process. This analysis focused on three main steps: (1) determine transportation deficiencies (LRP-4); (2) develop strategies or projects to address said deficiencies (LRP-6); and (3) analyze the strategies using performance measures (LRP-7). The MPO completed three separate rounds of scenario analysis. The analysis for the scenarios required a considerable number of calculations. As was mentioned in the evaluation criteria section (LRP-3), the MPO developed 16 evaluation criteria in four areas of consideration: mobility, economy, environment, and community values. These 16 measures resulted in each analysis having 16 different numbers for consideration. MPO staff wanted to provide its stakeholders with all the data used in analysis while making the results easily accessible. To do this, the MPO created scenario analysis summary charts that were color- coded based on how well the evaluation criteria showed movement toward or away from goals. All scenarios were compared to a “no build” base, which assessed the success of the 2040 regional transportation system if no new transportation improvements were made, beyond projects that are fully funded, or improvements that have been proffered from developers. Table 6.10 provides an example of this analysis. This table compares Scenario 1A with the “no build” scenario. While Scenario 1A shows an improvement in congestion, it shows significant impacts to the environment. The percentage change between the “no build” values and Scenario 1A’s values are also shown and color-coded. Table 6.10. LRTP Performance Measure Analysis Summary Chart for Scenario 1A PERFORMANCE MEASUREMENT BASE SCENARIO 1A Mobility Value Unit of Measure Value % Change Congestion (% of roads at LOS E or F) 14.1% % of Roads 12.6% 10.5% Congestion (hours of delay per day) 23,181.0 Hours 20,187.0 11.6% Mode Share (percentage of trips) 759,319 Trips/Day 759,334 0.0% Auto 88.1% Percentage of Trips 88.1% 0.1% Transit 2.5% Percentage of Trips 2.5% 0.1% Bike 2.7% Percentage of Trips 2.7% 0.2% Walk 6.7% Percentage of Trips 6.8% 0.9% Vehicle Mobility (vehicle miles traveled) 6,228,031.0 Miles/Day 6,145,450.8 0.6% Vehicle Crashes (crashes per year) 2,865.0 Crashes/Year 2,827.0 1.3% Bicycle Connectivity (% in largest area) 68.2% Percentage of Largest Area 73.4% 5.2% Environment Value Unit of Measure Value % Change Habitat 1,775.5 Ecological Score/Mile 1,786.9 -0.6% Air Quality (tons per year) 13,321.0 Tons/Year 13,211.0 0.8% Water Quality (% change in stormwater/water pollutants) 1,079.1 Tons/Year 1,168.3 -8.3% Floodplain (acres of 100 year floodplain affected) 99.1 Acres 120.2 -21.3% Historical (designated historic sites within 500 feet of projects) 1,141 # of Sites 1,171 -2.6% Archeological (designated archeological sites within 500 feet of projects) 264 # of Sites 299 -13.3% Note: LOS = level of service. 41

The goal of this table was to give an impression of the analysis while also providing the data for readers. The MPO also created a how-to guide for reading the table (Figure 6.1). Figure 6.1. LRTP how-to-read evaluation criteria summary memo. The use of scenarios and the evaluation of each scenario by using clear and accessible criteria resulted in the development of a draft preferred scenario approved in October 2013. Successes and Challenges As mentioned in the LRP-6 section, the use of a highly structured, analytical approach for assessing project scenarios resulted in more objective information for the MPO stakeholders, and, subsequently, more informed decision making. The process for reviewing and identifying necessary improvements based on transportation deficiencies, developing strategies to address those deficiencies, and evaluating those strategies resulted in the development of a project list based not on political desires but on regional need. As was mentioned in the LRP-5 section, the MPO began this process with a list of 15 improvements that had been regional favorites through several iterations of past regional LRTPs. When it was determined that some of these improvements might not bring about the transportation benefit that was hoped for, these projects were dropped in favor of new projects that seemed to provide the needed benefit. In some cases, projects that had never been considered moved forward in the scenario analysis, resulting in a preferred scenario that addressed future regional transportation more directly. As mentioned previously, the use of project scenarios confused many MPO stakeholders, and it took time to get participants acclimated. This was particularly challenging with the general public, as this technical approach for transportation planning is structured more for planners and 42

not for those outside of the profession. Throughout the process, members of the public would provide input, and they requested projects that had already been deemed unsuccessful by the MPO stakeholders. Furthermore, while this three-step process of deficiencies, strategies, and scenarios resulted in a stronger LRTP, to truly understand this process, a stakeholder had to be fully invested. Casual investment or occasional review often resulted in confusion. This proved to be the most problematic element when engaging the general public. While the analysis created a better LRTP, it may be that the planning process is now more challenging to access for participants outside of transportation planning. LRP-8: Adopt Preferred Plan Scenario The eighth step in the TCAPP Decision Guide discusses adoption of a preferred scenario or list of potential transportation improvements. The key purpose of all LRTPs is to outline a list of future transportation improvements that can be implemented to keep the region’s transportation system functioning efficiently. Transportation projects can take a long time to come to fruition. The 20-year outlook of the LRTP is the starting point for many transportation improvements. TCAPP Description The TCAPP Decision Guide describes the purpose and outcome for this eighth step (Table 6.11). Table 6.11. TCAPP Step 8—Purpose and Outcome Purpose Outcome To evaluate proposed scenarios in order to identify the locally preferred scenario that addresses the deficiencies while supporting the vision and goals. The evaluation of the plan scenarios will include the application of the approved evaluation criteria, methods, and measures. The preferred scenario and documentation of the evaluation of scenarios. Source: http://transportationforcommunities.com/shrpc01/kdp_step/21/0/8. MPO Process Based on the findings from the three rounds of scenario analysis, the draft preferred scenario was developed at the end of September 2013. The draft preferred scenario was evaluated like all previous scenarios, and the MPO held a public event solely devoted to this draft. At the end of November, MPO stakeholders approved the draft preferred scenario to move forward as the final scenario for the LRTP. Because fiscal constraint had not yet been completed, the decision called for all projects in the scenario to be included in the LRTP on either the fiscally constrained project list or on the visioning project list. 43

Successes and Challenges The preferred scenario was approved by the MPO’s decision-making body, the MPO Policy Board, with relatively little discussion. The almost six-month process of assessing scenarios resulted in the MPO stakeholders having a clear understanding of the projects that were in this scenario and a clear understanding that this particular group of projects achieved regional goals. The MPO stakeholders, who had been most involved in TCAPP’s implementation, understood each step in the decision-making process. When it came time to make that final decision, the MPO stakeholders were ready and fully understood the choice they were making. A tremendous amount of effort and guidance went into the development of the preferred scenario, but the scenario became threatened when new stakeholders came into the process who had not participated in the complete implementation of TCAPP. The preferred scenario was approved in November 2013. Following the approval of the scenario, the representation on the MPO Policy Board changed with local elections. This change in leadership resulted in some debate about the addition of new projects. Local staff also wanted to add improvements that they were hoping to implement quickly. MPO staff obliged by accommodating a few minor project requests based on these changes. Also, one of the major projects in the MPO that had been considered fully funded became uncertain because of recent elections and other factors. These political changes made the status of the preferred scenario, which had been a major achievement for the long-range transportation planning process, ambiguous. LRP-10: Adopt LRTP by MPO The 10th step in the TCAPP Decision Guide focuses on approval of the final long-range transportation report—specifically, reflecting public input on the report. All of the steps in the long-range planning section of the TCAPP Decision Guide lead up to this final step: approval of the plan document. TCAPP Description The TCAPP Decision Guide describes the purpose and outcome for this 10th step (Table 6.12). Table 6.12. TCAPP Step 10—Purpose and Outcome Purpose Outcome To review the draft LRTP that reflects the public comments for final adoption. Adopted LRTP. Source: http://transportationforcommunities.com/shrpc01/kdp_step/21/0/10. MPO Process The MPO’s Public Participation Plan requires that the LRTP go through two public hearings prior to its approval by the MPO Policy Board. In addition to hosting these hearings, the report is posted on the MPO’s website for a 60-day period prior to approval of the plan. Copies of the plan are made available at various locations around the community including the MPO’s offices, 44

offices for the City of Charlottesville and Albemarle County, regional public libraries, and the VDOT Residency Office. The MPO also writes a press release about the draft LRTP report, hoping that local media will promote the availability of the plan for review. Successes and Challenges The 2040 Long-Range Transportation Plan was approved by the MPO Policy Board on May 28, 2014. This approval served as the culmination of the 2-year LRTP process. Through the fiscal- constraint process, all of the projects identified in the preferred scenario were included on the LRTP’s fiscally constrained project list. Two projects that were considered outer year improvements were moved into a separate funding section called “preliminary engineering studies.” These projects, while considered community priorities in the LRTP process, will be reviewed further before the next update of the community’s LRTP. The approval of the plan coincided with a major transportation funding upheaval within the MPO. A very controversial bypass project, which was fully funded in both the MPO’s LRTP and Transportation Improvement Program, was deemed by the FHWA to no longer meet its original purpose and need. This determination made it very difficult for the project to proceed as intended. In response to this, Virginia’s Secretary of Transportation implemented the Route 29 solutions process. This process was an advisory process for developing transportation improvements that would help achieve congestion relief and mobility goals in the MPO’s Route 29 corridor. While the advisory panel included an MPO representative, it was not an MPO- driven process. The process was completed mid-May 2014. The state’s transportation decision- making body, the Commonwealth Transportation Board, will review the resulting projects for approval at its June 2014 meeting. Following this meeting, the MPO will likely update the 2040 LRTP document in order to accommodate these newly developed transportation improvements. This amendment process also requires the removal of the bypass project. The upcoming amendment process is not anticipated to greatly affect the overall project structure in the preferred scenario, but it does mean that the scenario could change somewhat if the amendments are approved. These changes would need to be accommodated and would not be subject to the MPO’s performance measurement process as was the case during the development of the 2040 LRTP. This alteration highlights a major dynamic in all community planning: even when using a rational planning approach, as was used in the MPO’s implementation of TCAPP, certain factors can alter the outcomes of the approach, and planning documents such as the 2040 LRTP must be flexible. 45

CHAPTER 7 Conclusions Nine conclusions are drawn. The first two pertain to how the individual steps of TCAPP are deployed, and the latter seven pertain to the use of performance measures. 1. Generally the TCAPP sequence of steps is logical. While it is not necessary that all LRP steps be followed, the order of the LRP steps matched what is ideally followed as part of LRTP development. In fact, in one instance the MPO had to deviate from this order: Step LRP-5 (Approve Financial Assumptions) could not be completed until after Step LRP-8 (Adopt Preferred Plan Scenario). Because of this deviation, MPO staff observed that many stakeholders were concerned the MPO might have developed a set of LRTP projects with costs exceeding available funds. 2. The use of scenarios offered both a benefit and a drawback for this case study. The benefit was a stronger LRTP based more on quantifiable needs than on assumed needs— in short, a more technically driven and less politically driven LRTP. However, as noted in the discussion under LRP-7, the disadvantage was that stakeholders who participated infrequently tended not to fully understand the planning process. By extension, the use of scenarios may impede engaging the general public in the planning process. 3. There appears to be at least an anecdotal association between the importance of performance areas and the influence of associated groups. A greater percentage of respondents noted that the “environment” was extremely important (50% of respondents) than respondents noting the “economy” was extremely important (18.2% of respondents). When asked about the influence of various groups on respondents’ support for a given initiative, 63% of all respondents indicated that organizations that advocate for the environment would have a lot of influence compared to 34% of all respondents who indicated that organizations that advocate for economic development would have a lot of influence. 4. There is substantial variability in the importance of different performance measurement areas. For example, based on the first survey of MPO stakeholders, consider the area that was generally most important (land use), which had 49% of respondents rate it as “extremely important” and 33% rate it as “very important.” By contrast, the least important area (freight mobility) saw only 4% rate it as “extremely important” with 38% rating it as “very important.” 46

5. There is less variability in the importance of different performance measures within each area. For example, based on the first survey of MPO stakeholders, consider the “economy” area. Within that area, the most important measure (“change in travel time to existing employment centers”) saw 69.8% of respondents rate it as “very” or “extremely” important. The three least important measures in that area (“project monetary costs that must be borne by localities,” “change in travel time to future employment centers,” and “travel time savings”) saw 58.1% of respondents rate them as “very” or “extremely” important. 6. Performance measures influenced decision makers. For example, from the second survey of MPO stakeholders, about half of responses (48%) indicated that changing the value of the performance measure from its expected value would definitely or probably influence respondents’ support for a scenario. 7. Performance measures had greater influence when their values were higher than expected rather than when their values were lower than expected. In the second stakeholder survey, a majority of respondents (57%) indicated that the performance measure would influence their support for a scenario when the measure was higher than expected. By contrast, a minority of respondents (38%) indicated that the measure would influence their support for a scenario when the measure was lower than expected. These percentages are significantly different and may suggest that respondents are more influenced by a scenario’s potential to deliver large benefits than its potential to fail to deliver promised benefits. 8. The influence of performance measures varies. The second survey showed that the influence of performance measures on respondents’ support for a scenario was unequal. The measure with the least amount of influence was passenger mobility: only 22% of responses indicated that this measure would “definitely” or “probably” influence support for a scenario, and the next lowest measure was land use (the corresponding percentage was 40%); these percentages are significantly different. The remaining five measures— social justice, economy, safety, freight mobility, and environment—were similar to one another, where between 51% and 60% of responses indicated that a change in the measure would influence respondents’ support for a scenario. 9. There are at least two possible reasons why the influence of performance measures on respondents’ support for a scenario was unequal. a. One possible reason, based on the second stakeholder survey, is that some measures show a greater percentage change than others when analyzed as part of a build scenario. For example, a hypothetical Scenario X that was examined herein could improve social effects by 19% but passenger mobility by only 0.4%, which may explain why more responses indicated that changing the social effects measure would influence project support than was the case if the passenger mobility measure changed. Some free response comments also supported this reasoning, with one comment being that a change of less than five percentage 47

points “won’t pique much interest.” Despite being provided with absolute changes, respondents continued to be interested more in percentage changes. b. A second possible reason, based on the first stakeholder survey, is that some performance areas are more important than others, as noted in Conclusion 4. For example, 40% of stakeholders noted that the area of safety was “extremely important,” and more than 80% noted that safety was that or “very important,” which may explain why a modest change in safety (from an expected improvement of 0.6% to a larger-than-expected improvement of 1.1%) would “definitely” or “probably” influence scenario support for most (67%) of respondents. 48

CHAPTER 8 Recommendations Six recommendations are offered. The first three pertain to how the individual steps in TCAPP are applied, and the latter three focus specifically on performance measure selection and interpretation. 1. Consider adding a multimodal focus to TCAPP. Specifically, in completing LRP-4 (Approve Transportation Deficiencies), the MPO found that TCAPP’s identification of types of deficiencies focused primarily on roadway and corridor issues, which conflicted with the MPO’s more multimodal regional goals. For example, this region identified transit deficiencies (based on large population or employment centers that were not within a quarter mile of a bus stop), pedestrian deficiencies (where such centers were not within 200 feet of pedestrian facilities), and bicycle deficiencies (based on areas that lacked bicycle connectivity, which was in turn based on vehicle speed and volume as well as roadway type). 2. Consider creating a cost-estimation spreadsheet for public transit or promoting existing resources. As part of LRP-5 (Approve Financial Assumptions), there was a need to develop cost estimates for candidate projects including roads and transit, but only road project estimates were available at the state level, and a transit estimation tool took several months to develop. This is likely to be the case in other states as well, so it could be beneficial for TCAPP to include a transit-specific estimation tool or links to existing tools. This spreadsheet could be part of TCAPP or could be a stand-alone research effort. 3. Add TCAPP deployment guidance indicating that Steps LRP-2 and LRP-3 should be developed in tandem or at least closely synchronized. Conceptually, LRP-2 (Approve Vision and Goals) precedes LRP-3 (Approve Evaluation Criteria, Methods and Measures). In practice, however, the meaning of some of the higher-level goals (in LRP- 2) may become clearer as specific performance measures are developed (in LRP-3). For example, consideration of alternative measures such as “bicycle connectivity” and “mode share” (in LRP-3) may help clarify for stakeholders a higher-level goal such as a “multimodal vision.” Thus, an MPO that is deploying TCAPP in the future may wish to iterate between LRP-2 and LRP-3. 4. Consider archiving the performance measures that users have found to be most useful. This case study focused on just one MPO, but it may be the case that as other regions use TCAPP, a consensus set of useful performance measures emerges. Thus while the results presented herein are specific to the Charlottesville-Albemarle MPO, an archive of performance measures from other locations might help determine if there are some measures that are generally more productive than others. For this case study in particular, the most important performance measure within each area, as defined by the initial stakeholder survey, is given in Table 8.1. The middle column gives the measure as rated 49

by survey respondents, and the right column gives the measure as quantified by the MPO. (For example, survey respondents indicated that the most important measure in the environmental area was “extent of waterways where pollutants would exceed regulatory limits.” The MPO was able to quantify that measure by calculating the “tons of pollutants in stormwater run-off per year.”) Table 8.1. Most Important Performance Measures in Each Area Performance Area Performance Measure From MPO Stakeholder Survey As Quantified by MPO Staff Environmental Extent of waterways where pollutants would exceed regulatory limits Tons of pollutants in stormwater runoff per year Land Use Impact on the connection of people to their jobs Average travel time to work in minutes Social Effects Number of persons for whom walking and biking are made more or less feasible because of the project Percentage of roads suitable for bicycling that form a connected network Safety Expected change in safety for motorists, bicyclists, and pedestrians Number of vehicle crashes per year Passenger Mobility Amount of change in passenger-miles traveled on non-auto modes Auto mode share (e.g., percentage of trips made by auto) Economic Benefit-cost ratio, where cost is monetary costs to government, and benefits are based on monetization of crash reductions and travel times Benefit-cost ratio, where cost is the cost to government, and benefits are based on the economic value of reduced crashes and improved travel time Freight Number of at-grade rail crossings Number of at-grade auto-rail crossings 5. Add TCAPP deployment guidance indicating that performance measure selection should also be based on measures whose numerical impact is large enough to be meaningful to stakeholders. For example, while the single most important area to stakeholders in this study was land use, only a minority of respondents were influenced by the measure chosen to represent land use, because no scenario could alter the measure by more than a few percentage points. As another example, a performance measure in an auto-oriented area that quantifies the percentage increase in walking, biking, and transit (individually or combined) may have a different influence than a performance measure that quantifies the percentage decrease in auto trips, because a given number of trips shifted from the auto mode to modes that have a low share of total trips will result in a relatively high percentage increase for non- auto modes but a relatively low percentage decrease for the auto mode. While the provision of absolute numbers may reduce the importance of percentages somewhat, this study suggests that respondents continue to look at percentage changes in performance measures. 6. Add TCAPP deployment guidance suggesting options for assisting stakeholders in interpreting performance measures. No matter how carefully measures are selected, they will rarely be directly comparable, and decision makers may benefit from additional context. For example, in the case of land use, an acknowledged expert adviser on land use 50

could provide a brief, high-level explanation of why no scenario could alter the measure by more than a few percentage points and could craft a context suggesting what level of change would be considered meaningful. Another option for adding context would be to provide decision makers with examples of how recent local transportation projects (with which they would presumably be familiar) would have affected the performance measure(s) under consideration. 51