National Academies Press: OpenBook

Quantifying the Benefits of Context Sensitive Solutions (2009)

Chapter: Chapter 3 - Research Approach

« Previous: Chapter 2 - Literature Synthesis
Page 11
Suggested Citation:"Chapter 3 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2009. Quantifying the Benefits of Context Sensitive Solutions. Washington, DC: The National Academies Press. doi: 10.17226/23012.
×
Page 11
Page 12
Suggested Citation:"Chapter 3 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2009. Quantifying the Benefits of Context Sensitive Solutions. Washington, DC: The National Academies Press. doi: 10.17226/23012.
×
Page 12
Page 13
Suggested Citation:"Chapter 3 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2009. Quantifying the Benefits of Context Sensitive Solutions. Washington, DC: The National Academies Press. doi: 10.17226/23012.
×
Page 13
Page 14
Suggested Citation:"Chapter 3 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2009. Quantifying the Benefits of Context Sensitive Solutions. Washington, DC: The National Academies Press. doi: 10.17226/23012.
×
Page 14
Page 15
Suggested Citation:"Chapter 3 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2009. Quantifying the Benefits of Context Sensitive Solutions. Washington, DC: The National Academies Press. doi: 10.17226/23012.
×
Page 15

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

11 This chapter presents the methodology followed for the completion of the research. The first section presents the devel- opment of the principles and associated benefits while the following section presents the case study selection process criteria and the data collection and analysis approach. Principles and Benefits A fundamental aspect of this research effort is the identifi- cation of CSS principles. As noted previously, several efforts have been completed to date that attempted to identify and document such principles. The team considered the project development process and identified associated actions that could be considered the cornerstone for the activities to be completed in each phase as a starting point. These actions include the identification of the appropriate project players including the team members to participate in project develop- ment and the stakeholders that could have an influence on the project, along with the definition of the process to be followed for the project development, the goals to be achieved through this process as a result of the process, and the actions required to assure that the project was a success. Using the team’s multi- disciplinary expertise a set of principles was developed during a half-day work session and presented in Table 1. The next step in this process involved the identification of potential benefits that could result from the application of each specific action principle. The benefits were grouped in two basic categories based on who receives the benefits, i.e., the agency or the users/community. It is important to distinguish benefits based on the potential beneficiaries, since some of these bene- fits are internal to the agency’s operations and will have no direct impact on users. This provides a distinct separation for the agency to determine those benefits that could be inter- nal to the agency as well as those that the users/community will receive. The list is provided in Table 2 (the first 11 are agency benefits and the last 11 are benefits primarily associated with users/community). At the same time, AASHTO had also developed a set of principles that were reviewed and considered in this research (27). As a general guideline for performance measuring, a principle should be concisely focused, self explanatory, and capable of conveying intended actions. Therefore, it was deemed appropriate to define a comprehensive framework that could be used by AASHTO in CSS language that includes action principles. In order to coordinate and combine this research with their work, a hierarchical structure was devel- oped, where the CSS Goal is defined as the very general statement of what a project should target followed by the CSS Core Strategies (the five AASHTO “principles”), the CSS Action Principles (the 15 principles presented in Table 1), and the fundamental CSS Benefits (a “select few” of the 22 benefits in Table 2). This new description concept is pre- sented in Table 3. This approach allows for a unified format where the core strategies (AASHTO’s broad “principles”) are first stated followed by the action principles needed to achieve them and concluding with the fundamental benefits to be measured to assure the proper implementation of the process. It should be noted that the fundamental bene- fits capture mostly those that are perceived by the users/ community and are considered essential in delivering a contextual solution. Case Study Selection Process The development of the performance measures in this research effort is based upon the identification of case studies and the collection of data to demonstrate the estimation of such benefits from applying the CSS principles. Therefore, candidate projects were identified for consideration as sources for performance measure evaluation. A large number of such candidate projects were initially developed based on a variety of sources and included more than 100 cases. The projects C H A P T E R 3 Research Approach

included in the preliminary list were derived from the follow- ing sources: • The 2005 and 2006 submissions for the AASHTO Center for Environmental National Context Sensitive Solutions Competition; • A 2002 report by Kentucky Transportation Center under contract with FHWA titled “Context Sensitive Design Case Study Documentation” (28); • Cases on the www.contextsensitivesolutions.org web site; • Cases submitted by the NCHRP panel members; and • Cases from Kentucky, Pennsylvania, and Florida submitted by team members. The next step involved the identification of candidate cases that could be useful in the development of performance mea- sures for CSS projects. Prior to their selection, the following criteria were established to allow for selecting cases that will be representative of a variety of conditions and contexts: • Addressing principles and benefits identified; • Providing for a geographical representation to assure con- sideration of terrain, climate, and other key factors; • Including a variety of development context within urban and rural areas; • Considering the current phase of the project; • Including a variety of transportation modes; and • Representing a range of project scopes, sizes, and types. Using those criteria, the team evaluated each project in the original list of 112 cases and identified the cases for potential consideration in Phase II. The team reviewed available docu- mentation to develop a basic understanding of each project, created a short summary for each project, and identified the principles used and benefits accrued from the application of CSS. A focus group approach was then undertaken to review all projects and collectively determine the appropriateness of each project. The focus group consisted of a transportation planner, roadway design engineer, safety expert, and project develop- ment expert. Each member presented the cases assigned for review and provided an assessment on whether the case is suitable for further consideration. Following a discussion and deliberation, the focus group reached a consensus on includ- ing 49 cases in Phase II for further consideration. Some cases were eliminated from the list because they demonstrated limited scope, inadequate coverage of CSS principles, minor stakeholder and public involvement, or few potential benefits. Several cases were considered marginal because they involved a limited number of principles or they were in geographic regions providing cases with more clearly defined and stronger CSS applications. Once the list of candidate projects was refined, representa- tives from the project’s transportation agencies were contacted to determine the level of cooperation that could be provided for data collection in Phase II. A telephone contact was initi- ated where the scope of the research was explained, the project was reviewed, and an assessment of the willingness to cooper- ate was made. As noted above, a short case description was pre- pared by each team member that included an assessment of the principles and benefits for each project. This description was provided to the contacted person for verification and evalua- tion of the accuracy of the information. This allowed for accu- rately determining the context of the project and the principles applied and benefits accrued. This process also provided a pre- liminary indication of the cooperation to be achieved during Phase II for the data collection. Data Collection The data to be collected for this effort has two different forms: (1) quantitative, based on data provided by the agencies 12 Table 1. CSS principles. 1. Use interdisciplinary teams. 2. Involve stakeholders. 3. Seek broad-based public involvement. 4. Use full range of communication strategies. 5. Achieve consensus on purpose and need. 6. Address alternatives and all modes. 7. Consider a safe facility for users and community. 8. Maintain environmental harmony. 9. Address community and social issues. 10. Address aesthetic treatments and enhancements. 11. Utilize full range of design choices. 12. Document project decisions. 13. Track and meet all commitments. 14. Use agency resources effectively. 15. Create a lasting value for the community. Table 2. CSS potential benefits. 1. Improved predictability of project delivery. 2. Improved project scoping and budgeting. 3. Improved long-term decisions and investments. 4. Improved environmental stewardship. 5. Optimized maintenance and operations. 6. Increased risk management and liability protection. 7. Improved stakeholder/public feedback. 8. Increased stakeholder/public participation, ownership, and trust. 9. Decreased costs for overall project delivery. 10. Decreased time for overall project delivery. 11. Increased partnering opportunities. 12. Minimized overall impact to human and natural environment. 13. Improved mobility for users. 14. Improved walkability and bikeability. 15. Improved safety (vehicles, pedestrians, and bikes). 16. Improved multi-modal options (including transit). 17. Improved community satisfaction. 18. Improved quality of life for community. 19. Improved speed management. 20. Design features appropriate to context. 21. Minimized construction-related disruption. 22. Improved opportunities for economic development.

on specific benefit metrics; and (2) semi-quantitative, based on survey responses from persons involved in each project. A dif- ferent set of tools is needed for each type of data to properly collect the required information. In general, a set of forms and tables to be completed by the agency was the main collection tool for the quantitative data and a web-based survey to be dis- tributed to the project team members and stakeholders was the data collection tool for the semi-quantitative data. The approach considered for collecting the data to be used in the analysis of quantitative data was to request all possible doc- uments and pertinent data from team members and for them to complete forms that would allow for their quantification. This approach was considered more appropriate than requiring team members to identify the pertinent information and com- plete the forms, since the latter could have resulted in a lower response rate. A list of potential data items to be used for col- lecting the required quantitative data was developed (Table 4). The various data items requested assist in understanding the conditions of each case and documenting the associated ele- ments including extent of public involvement, role of stake- holders, creation of an interdisciplinary project team, commu- nity impacts, environmental impacts, and application of design flexibility. Project team members identified which data were available and the answers were cross referenced to identify the person(s) who could be contacted for providing the required data. This approach allowed for identifying alternates in case the original request was not fulfilled. These data were requested from the appropriate team members with an initial e-mail sent indicating the required data items with a request for a transmit- tal in any available form (electronic or paper). A set of web-based surveys was developed for estimating the required semi-quantitative data. The goal of the surveys was to solicit the expert opinions of the project team and stake- holders on the level of satisfaction from the application of the CSS principles on the project and the interaction between the team and the stakeholders. For the project team survey, two versions were developed: one for the team leader/project manager and a second for the team members. Both surveys have a common section that solicits opinion and level of sat- isfaction from the application of the CSS principles and their associated benefits as well as the levels of interaction between the team and the stakeholders. Both surveys have an addi- tional section where the respondent addresses the availability level of the required data to complete the benefit quantification. This part of the survey was a critical element for two rea- sons. First, the answers of the team leader/project manager 13 Table 3. CSS principle driven and benefit justified. Primary Goal: Find a “best fit” transportation solution for the context that meets expectations of the transportation agency, stakeholders, and community. Core Strategies: Establish a shared stakeholder vision to provide a basis for decisions. Demonstrate a comprehensive understanding of context. Foster continuing communication and collaboration to achieve mutual success. Exercise flexibility and creativity to shape transportation solutions. Preserve and enhance community and natural environments. Action Principles: Use interdisciplinary teams. Involve stakeholders. Seek broad-based public involvement. Use full range of communication methods. Achieve consensus on purpose and need. Address alternatives and all modes. Consider a safe facility for users & community. Maintain environmental harmony. Address community & social issues. Address aesthetic treatments & enhancements. Utilize full range of design choices. Document project decisions. Track and meet all commitments. Use agency resources effectively. Create a lasting value for the community. Fundamental Benefits: Increased stakeholder/public participation, ownership and trust. Improved community satisfaction. Design features appropriate to context. Decreased costs for overall project delivery. Minimized overall impact to human and natural environment. Improved mobility for users. Improved safety (vehicles, pedestrians and bikes). Improved quality of life for community.

allowed for determining whether a project would be part of the final case study database. Second, the answers of the team members allowed for determining who should be contacted to obtain the specific data required for the possible quantifica- tion of the stated benefits. The survey for the project team members included a more targeted set of questions regarding data availability. It was determined that any requests for avail- able data will be better handled if it was distributed among the various project team members and not solely requested from the team leader/project manager. This way the person who may be able to provide the data was identified. These surveys were pilot-tested and refined appropriately, and a sample is provided in Appendix B. The survey for the stakeholders requests only opinions on the benefits of the project and their level of satisfaction with their interaction with the project team (Appendix B). Limit- ing the survey only to benefits was considered more appro- priate for the stakeholders, since they most likely could not relate to the CSS principles that the team may have applied. In addition, it was unlikely that stakeholders would have data related to the project, therefore further justifying omission of this section from the survey. Team leaders/project managers were requested to provide the names and contact information for the project team mem- bers and the stakeholders involved in the project. This pro- vided the required information to initiate the surveys to col- lect the opinion of the team members and the stakeholders’ satisfaction regarding the project. The results from the team leader/project manager surveys were used to select the final candidate cases based on the potential availability and cover- age of data, since the initial project screening was not suffi- ciently detailed due to the limited time available for identifying the potential case studies. All surveys included a set of questions that could be used to evaluate potential differences in the level of satisfaction between project team members and stakeholders to establish the magnitude of the Arnstein gap (which is the difference in the perception or satisfaction of application between the proj- ect team and the public). Data Analysis As noted above, the approach taken in this research for quantifying the CSS benefits is principle-driven. This implies that the benefits derived from application of CSS principles need to be determined and then quantified in an efficient and practical manner to allow for estimating the magnitude of these benefits. Therefore, the focal point becomes the identifi- cation of the most appropriate CSS performance measures as they are manifested through the application of CSS principles. In essence, the key questions become the following: • What were the CSS principles applied and resulting benefits? • What is the evidence for having achieved these benefits? • What are the real-world benefits? Some benefits yield quantitative metric results (comprised of numerical values that are standard units of measure using a ratio level integer scale with equal magnitude and a fixed zero point) measured in terms of dollars saved, time saved, or an increase or enhancement results in terms of acres or lineal feet. Many economic studies are available to provide 14 Table 4. Data list for quantitative elements. Stakeholder meetings documentation (date, agenda, attendance and minutes/summary). Public involvement meetings documentation (date, agenda, attendance and minutes/summary). Public involvement tools used (list and/or samples). Record of cost estimates and final costs (by phases). Scope change(s) documentation. Change order(s) documentation. Scheduled and actual completion dates (by phase). Memorandums of agreement with regulatory agencies or other stakeholders. Alternative and modal options analysis documentation. Design options analysis documentation (including design exceptions if applicable). NEPA related documentation (such as Executive Summary, FONSI, EA or EIS). Capacity analysis documentation. Crash data by severity and type. Design speed data. Operating speed data. Maintenance of traffic plans (construction phase). Construction phasing alternatives study. Maintenance/operating cost records (Before). Maintenance/operating cost records (After). Legal actions, decisions or findings documentation. Project decisions and commitments logs/records.

a means to estimate the dollar values of some of the bene- fits of interest including the value of preserving wetlands and minimizing disruption. Also, economists have devel- oped impact assessment tools that can also assign primary/ secondary benefits (in dollar terms) to society for some ben- efit types including environmental. Benefit can be deter- mined in some instances using principles of benefit-cost analysis such as the case of determining the value of timely completion. Many benefits can only be evidenced through semi- quantitative assessments which are, none-the-less, real ben- efit measures. The research team used semi-quantitative methods whenever possible to supplement the other metrics. These semi-quantitative metric methods derive numerical values from rating or ranking scales expressing opinion, atti- tude, or perception. These metrics are values expressed as inte- gers simply indicating a relative scale position with no absolute zero. Industry uses many such devices to determine customer satisfaction and preference. These methods have become sys- tematic and standardized to provide semi-quantitative mea- sures for before/after comparison, trend analysis, and inter- industry comparison. These same approaches can work for estimating CSS benefits and were employed here using basic statistics that establish maximums, minimums, and averages and to provide for cross-tabulation analysis. The research team intended to develop, test, and provide benchmarks (numerical ranges and/or averages) for many benefit metrics based on the selected series of case studies. 15

Next: Chapter 4 - Findings »
Quantifying the Benefits of Context Sensitive Solutions Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB’s National Cooperative Highway Research Program (NCHRP) Report 642: Quantifying the Benefits of Context Sensitive Solutions explores guidelines for quantifying the benefits of applying the principles of Context Sensitive Solutions to transportation projects.

Appendices A through E for NCHRP Report 642 are available online.

Appendix A: Literature Review Summaries

Appendix B: Team Member and Stakeholder Surveys

Appendix C: Rationale for Principle-Benefit Association

Appendix D: Documentation of Case Studies

Appendix E: Case Study Scores

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!