Estimating Highway Preconstruction Services Costs - Volume 2: Research Report (2016)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

55 C H A P T E R 5 5.1 Introduction This chapter provides an outline of Estimating Highway Preconstruction Services Costs, Volume 1: Guidebook (hereafter referred to as the guidebook). The purpose of the guidebook is to provide support to highway industry practitioners regard- ing the implementation of the PCS cost-estimating system and newly developed model. The NCHRP Project 15-51 literature review, case studies, and interviews revealed a number of issues, practices, and techniques beneficial to the implementation of PCS cost estimating for transportation projects. The research team held an intensive 2-day workshop to develop a structure and format for effectively organizing and communicating the information gathered to a wide audience of transportation agencies, designers, contractors, researchers, educators, and others within the transportation industry. The guidebook is delivered as part of Task 6; in conjunction with the submission of this final research report, it completes Phase 2 of the project. The guidebook describes effective practices and high-impact decision points for estimating the costs of preconstruction. The guidebook also explains tools and techniques for the practical implementation of typical projects. The first section of the guidebook is written for the upper management of agencies. It contains the business case and key management messages. The next sections of the guidebook explain the key principles of the PCS cost-estimating process during project development and the various approaches that can be applied at different stages of the process. The benefits of utilizing historical project cost data and the correct prepara- tion of the data for modeling are presented. This is followed by a descriptive introduction to each of the three data-driven estimating models. In addition to the sophisticated data- driven models, a functional-level estimating technique is also provided to assess department resource allocation and cost- ings for negotiation with external consultants. The imple- mentation and continuous improvement of all these models are discussed in Chapter 6. Project-specific considerations are discussed in Chapter 7, along with contract administration guidance. Finally, an appendix containing agency-specific PCS cost-estimating tools and effective practices found in the Task 2 case studies is included to furnish examples for DOT implementation. 5.2 Guidebook Outline The guidebook is intended to facilitate effective implemen- tation of PCS cost-estimating systems to help improve the state of the practice in delivering transportation projects. The guidebook focuses on practical methods, systems, and pro- tocols intended for immediate application by transportation professionals. The research team acknowledges that each agency uses dif- ferent terminology and project development processes. As a result, the content of the guidebook remains as general as pos- sible and gives readers direction on how to create databases and models based on their unique agency requirements. There is no set estimating model to be used. Agencies are at liberty to use any or all of the data-driven models, depending on which suit their data best. One of the key underlying assumptions developed from the research is that the ability to effectively implement the PCS cost model requires a dynamic, integrated team sharing a common vision of project success. An important compo- nent of this shared vision is an understanding of the inter- related nature of high-impact decisions made by the project team at critical points in the project development process. The guidebook clearly describes the step-by-step process to develop a top-down PCS cost-estimating model with three different data-driven approaches: • Multiple regression modeling, • Decision tree analysis, and • Artificial neural network modeling. Development of Guidebook

56 The guidebook also provides a complete guide on develop- ing a functional-level PCS cost-estimating model or a bottom- up model. The content of the chapters is summarized here: • Chapter 2: PCS Cost-Estimating Process. This chapter describes the overall PCS cost-estimating process that is aligned with a typical project development process. The need of three different approaches for estimating PCS costs is explained. The appropriate timing of application and the effectiveness of use are discussed. • Chapter 3: PCS Database Development and Management. This chapter discusses a database development and manage- ment process required for a successful implementation of data-driven PCS cost estimating. Some specific topics dis- cussed in this chapter include data collection and cleaning strategies, identification and evaluation of potential input variables, and development/optimization of PCS databases. • Chapter 4: Top-Down PCS Cost Estimating. This chapter explains the development process of top-down PCS cost- estimating models. Three data-driven methods—multiple regression, decision trees, and artificial neural networks— are discussed in detail. • Chapter 5: Functional-Level PCS Cost Estimating. This chapter discusses the development process of functional- level PCS cost-estimating models. It discusses the use of a work breakdown structure in developing a functional-level PCS cost-estimating model, discusses the feedback loop for continuous improvement, and also addresses issues in database creation, maintenance, and management. • Chapter 6: Implementing PCS Cost-Estimating Models. This chapter discusses important aspects related to the implementation of PCS cost-estimating models in practice, such as the interpretation of PCS cost estimates, incorpo- ration of this estimate into decision-making procedures, tracking of PCS costs throughout the project develop- ment process, capturing lessons learned from the use of the framework described in the guidebook and specific models developed by using it, and continuous improvement pro- cedures to optimize the performance of these models. • Chapter 7: Project-Specific PCS Estimating Issues and Con- tract Administration Guidance. Finally, this chapter dis- cusses project-specific and contract administration issues associated with the development and use of PCS cost- estimating models. This chapter covers project monitoring strategies, actions required under potential scope changes, how to use the PCS cost estimate to identify and quantify scope creep, and some aspects related to the use of in-house versus external designers/consultants. Definitions specific to preconstruction services are included in Appendix A, and effective practices, methods, and tools information from the case studies are included in Appendix C. 5.3 Guidebook Vetting 5.3.1 Background The guidebook is oriented toward decision making and a process view of preconstruction project management. It is meant to support decision making by the project team and does not provide a prescriptive recipe for how to use the PCS cost model’s output. Task 5’s objective was to test the draft PCS cost-estimating guidebook to ensure that the materials are applicable for practice. The research team conducted a vetting workshop of the guidebook with two DOTs that are interested in inves- tigating the use of the PCS cost models. For this study, the research group approached Iowa DOT and Montana DOT. These agencies were selected because they had both featured in case studies for this project, the research team was familiar with their practices, and real PCS data from each could be used for exercises. Iowa DOT currently does not estimate the PCS cost for a project; however, the agency is looking at ways to introduce a process to do this. The lack of a formalized system or famil- iarity with PCS cost estimating made the agency a great can- didate to pilot test the guidebook as it tested the guidebook’s ability to communicate concepts at an introductory level. MDT estimates PCS costs for all projects using a standard percentage of construction costs. It also records PCS hours; however, it does not use this collected historical data to esti- mate future projects’ PCS costs. This agency provided a good environment to assess whether the business case of the guide- book models was compelling enough to warrant implemen- tation of more rigorous data-driven models. The PCS cost data collected from the two agencies during the case studies also allowed the team to customize workshop exercises and examples for each agency. The vetting workshop took place over 2 days and involved upper management, design, construction, and contracting per- sonnel from the selected DOTs. Each DOT was also at liberty to invite representatives from local consulting firms; however, despite invitations, none participated in the workshops. The first day of each workshop consisted of a mixed program of presentations of guidebook material and interactive exer- cises to apply the concepts of the guide in practice. Table 5.1 illustrates the day’s agenda. DOT staff typically involved in preconstruction project tasks from different offices and departments participated. Par- ticipants were divided into groups for the day and worked together to complete each of the exercises. To provide a genu- ine experience, laptop computers were provided with loaded software that allowed the groups to physically develop linear regression, decision tree, and artificial neural network models. Historical PCS cost data collected from the DOTs was curated into a small database to use in the examples.

57 A preliminary filed review form from a real highway recon- struction project was also used in an exercise on functional- level estimating. Participants were asked to assign activity hours to all the PCS activities required for the project given the information provided within the form. These hours were then aggregated and assigned an average labor rate to pro- duce an early estimate. The second day of the workshop involved a focus group with DOT preconstruction personnel and higher manage- ment. This meeting provided a platform to assess the previ- ous day’s results and separate the feedback that applies only to the given DOT and that which had broader application. Although this step was not required in the RFP, the research team believed it was vital to developing an implementable guide book. This step has been extremely valuable on past guidebook efforts by research team members. 5.3.2 Measuring Results The purpose of the workshops was to evaluate how effectively the guidebook communicated PCS cost-estimating practices and whether the research could be implemented by highway agencies using the resources provided within it. The workshops also aimed to collect feedback detailing recommendations for changes to the draft guidebook content. To fulfill these goals, a number of research methods were employed. These included: • Surveys, • Observation, and • Focus groups. Using three methods to evaluate the performance of the guidebook allowed the research team to “detect recurrent patterns or consistent relationships” (Abowitz and Toole 2010) within the feedback received. Observations were made during Day 1 of the workshop. Of particular interest to the research team was participants’ engagement with the material provided. Specifically, the observers were looking to see if groups were able to relate the new estimating concepts to their prior estimating expe- rience and find value in the information provided. Obser- vations were also made during the exercises to see how well the guidebook aided groups in completing tasks. If the group had to ask additional questions, this was noted as it implied that the guide was not comprehensive enough in that particular area. A survey was conducted at the end of Day 1 to measure participants’ views on the guidebook and the concepts pre- sented within it. The survey was developed following prin- ciples of Taylor-Powell and Renner (2009) and aimed to quantify perceived changes in motivation, knowledge, and estimating skills as a result of guidebook. At the end of Day 1, participants completed the survey, and an open discussion was held about the merits and problems of the guidebook. “Focus groups are carefully planned discussions stim- ulated within a predefined group environment to obtain perceptions about a defined area of interest in a permissive, nonjudgmental environment” (Yu et al. 2006). Day 2 involved a structured discussion about the strengths and weaknesses of the guidebook and the realities of implementing its con- cepts within an agency. It was important for the research team to steer discussion to issues relating to the guidebook’s applicability to all highway agencies and limit focus on just its application to the DOT participating in the vetting. The final stage of the vetting process was to combine the feedback provided and address any issues within the guide- book to improve its quality. This was an iterative process. Feedback for the Iowa DOT vetting was used to revise the guidebook before it was provided to the Montana DOT for the second vetting. A summary of the feedback received from the two workshops is detailed in the following sections. 5.3.2.1 Iowa DOT Vetting The first draft of the revamped guidebook was sent to Iowa DOT two weeks prior to the workshop to allow participants an opportunity to read it and bring any questions they had to the vetting. The research team conducted the workshop, which had 13 DOT participants Observations made throughout the day were that the par- ticipants were very engaged and receptive to the concepts presented. The final survey confirmed this with very positive feedback. Of the survey respondents, 75% reported that they were “quite a bit” or “a great deal” more aware of the overall PCS cost-estimating process as a result of the guidebook. All participants reported a greater understanding of top-down data-driven models and how to develop a database. The focus group held the day after the workshop was very positive; there was a general consensus that the guidebook contained DAY 1 Welcome and introduction Introduction to PCS cost estimating Break Traditional PCS estimate exercise Discuss guidebook estimating models Lunch Demonstration of software for model development Estimate exercise using guidebook models Discuss functional-level estimating Functional-level estimating exercise Feedback/discussion Table 5.1. Workshop agenda of Day 1.

58 practical approaches that would significantly improve the preconstruction phase of a project. The major findings of this vetting were: • The guidebook is repetitive in some sections; this redun- dancy should be removed. • While the guidebook contains many great figures, there is still a lot of text, using call-out boxes to draw out the impor- tant information would make it more readable. • The business case in Chapter 1 should be aimed at higher management as they are the people who will make a deci- sion on whether these practices are implemented. • Appendix D needs to be written less technically. • The guidebook needs to better explain that implementing these estimating techniques cannot be achieved by one per- son and that sufficient up-front resources will need to be committed by the DOT to maintain databases and develop models. As a result of this feedback, the guidebook was reorganized, with Chapter 3 dedicated to database development and man- agement to better illustrate the up-front investment required by DOTs to implement data-driven estimating processes. Redundant text was removed, and call-out boxes were intro- duced. Chapter 1 was revised, and Appendix D was modified. 5.3.2.2 Montana DOT Vetting An updated draft of the guidebook was then provided to MDT two weeks ahead of its vetting. This workshop was led by Dr. Gransberg, Dr. Jeong, and one graduate student. There were 21 participants from a variety of districts and departments for Day 1. Again participants were very engaged and there was lively discussion throughout the day. Groups worked diligently on the exercises provided and successfully applied guidebook concepts to their solutions. Survey responses at the end of the day recorded 80% of participants being “a great deal” or “quite a bit” more aware of the overall PCS cost-estimating process. All participants whose duties involved PCS estimat- ing indicated that they would consider implementing con- cepts presented in the guidebook, especially using top-down estimating models. The focus group held the following day was very encouraging. The group believed the research was timely but would require a change in agency culture to work effectively. Staff will need to place more accountability on the PCS estimating phase. The major comments and findings from this vetting are as follows. Strengths of Guidebook • Good organization–very methodical. • It steps through process beginning to end very well. • Easy to read. • Thorough coverage of all the functions (survey, enviro, roadway, etc.). Improvements • Paragraph is very vague and could use a little more explanation. • Could clarify that PCS can be whatever an agency wants– just preliminary engineering or can split out ROW. Comments • This research fits well with MAP-21 requirements. • There is an increasing need for cost accountability from FHWA; these tools will help justify resource allocation. Training • Participants felt that a workshop (or consulting session) would help implement the tools of the guidebook. There were many comments that the workshop really helped the guidebook make more sense. • A consulting session would be good to help an agency establish a database and analyze what factors are the most important inputs (what data to collect).