Managing Capital Costs of Major Federally Funded Public Transportation Projects: Research Results Digest (2006)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Research Results Digest 78 September 2006 INTRODUCTION This digest contains recommendations for strategies, tools, and techniques to bet- ter manage major transit capital projects over $100 million. Information is presented on various estimation, project-management, and cost-containment approaches that were applied successfully to 14 case studies. The results, which indicate which tools and tech- niques contributed to more effective project management, will be useful to transit sys- tems and communities endeavoring to plan or implement major transit investments. The contractor’s final report for this project is published as TCRP Web- Only Document 31 (www4.trb.org/trb/ onlinepubs.nsf). RESEARCH OBJECTIVES The underestimation of capital costs for major public transportation projects around the country has raised public scrutiny of the industry’s ability to estimate, manage, and contain such costs. The goal of this research project, therefore, was to identify and sug- gest strategies, tools, and techniques to bet- ter estimate, manage, and contain the capi- tal costs and schedules of major transit projects over $100 million. Of particular in- terest was the ability to estimate at the planning and engineering developmental stages—with some desired degree of ac- curacy—the resulting as-built costs for major federally funded public transporta- tion projects. Recent research has docu- mented this issue as a concern noted across other major transportation modes in the United States and abroad. LITERATURE SEARCH AND DATA COLLECTION In order to assess the scope of the proj- ect cost underestimation or overrun problem and to identify various strategies and tech- niques used in the control of cost escalation, the researchers conducted a comprehensive literature search and a thorough review of available project documentation. The ob- jective of the literature search was twofold. First, it documented the current state of the industry in responding to the challenges of cost increases in all of the transportation modes. Second, it identified sources of data to support the cost and schedule analysis in order to size the extent of the cost issues. The final report contains the summary of the literature search along with findings from the case-study examinations. Project-Selection Process An initial set of over 30 candidate proj- ects was reduced to 28 projects conforming MANAGING CAPITAL COSTS OF MAJOR FEDERALLY FUNDED PUBLIC TRANSPORTATION PROJECTS This digest summarizes the findings from TCRP Project G-7, “Managing Capital Costs of Major Federally Funded Transportation Projects,” conducted by Booz Allen Hamilton, McLean, Virginia. Subject Areas: IA Planning and Administration, VI Public Transit Responsible Senior Program Officer: Dianne Schwager TRANSIT COOPERATIVE RESEARCH PROGRAM Sponsored by the Federal Transit Administration

to the basic study requirements—fixed guideway transit projects in excess of $100 million with suffi- cient information to support the cost and schedule analysis. Cost and schedule information was col- lected from each of these 28 projects to establish the extent of the cost and schedule trends. The research team initially formulated a set of hypotheses on the reasons for overruns in total costs for major transit capital projects. From the set of 28 projects, the researchers’ hypothesis-testing pro- cess suggested a number of priority projects for more in-depth analysis. The TCRP project panel se- lected a subset of 14 projects from this prioritized list for more detailed study of the strategies, tools, and techniques used to estimate, manage, and control capital costs. The selection for the 14 case studies was made on the basis of identifying a representative number of projects with varying capital cost experi- ences and approaches. RESEARCH APPROACH The research team documented cost and project- definition information used to estimate the costs and the project estimation, management and cost- containment approaches used to manage these cost estimates throughout the course of the project- development process. The team developed a data- base structure to guide (a) the collection of project capital cost data and (b) the approach for analyzing cost changes by major cost drivers through each phase of project development. The major cost driv- ers considered in this analysis were initial inflation adjustment; scope changes (including unit cost and quantity); and schedule changes (including the in- flationary impact of project delays). The phases of project development included the following: • Alternatives Analysis (AA)/Draft Environ- mental Impact Statement (DEIS); • Preliminary Engineering (PE)/Final Environ- mental Impact Statement (FEIS); • Final Design (FD); and • Construction/Operations. The research team then examined nine of these projects in more detail to determine the successful strategies, tools, and techniques used to manage bet- ter the capital costs of these projects. The final report identifies these projects, their cost and schedule estimates as they progressed through the project- development process, and their project-definition characteristics throughout this process. FINDINGS AND CONCLUSIONS The project examined the strategies, tools, and techniques to better estimate, contain, and manage capital costs based, in part, on the experiences of the case study projects. The literature review and case studies built a foundation for the analysis. The ana- lytical structure was shaped by the hypotheses. The extent of the cost and schedule increases encountered in major transit projects has been defined through the project analyses. The related factors and causes be- hind cost escalation have been identified through the case study analyses. The combination of all of these efforts helped to shape the following conclusions to this research. Project Definition Project definition entails the conceptualization of the alternatives and the refinement of this project def- inition through the course of the project-development process. The inception and evolution of a project can have a large impact on the capital costs. In particular, the level of design is an important factor affecting the uncertainty of the capital costs and the subsequent variation in the estimates. Clear cost priorities, established early in project development, are important to cost and schedule per- formance. These priorities should be reflected in the initial evaluation of alternatives. Establishing clear budget and schedule constraints early in the project- development process helped contain scope creep and identify reasonable project-development sched- ules. However, some flexibility with respect to scope and schedule should be maintained in the project- development process in order to adapt to the more unique project conditions identified throughout the development process. This flexibility combined with appropriate budgetary targets and reasonable devel- opmental schedules formed the successful factors in project definition. Several of the case studies point out that some of the most difficult risks to capital costs and schedule are (a) time to achieve political consensus and (b) ac- quisition of private property. As part of the project- development process, it is important to manage public expectations and communicate the tradeoffs between scope, cost, and schedule in order to control scope 2

creep. Outreach to community and businesses is im- portant to minimize project redefinition and maxi- mize support. It is also crucial because of the influ- ence of the political process in defining and funding major transit projects. Outreach can be achieved through a transparent alternatives analysis process and clear communication of the project refinement in the engineering process, and its effect upon the capi- tal cost estimate. In summary, while engineering issues were encountered, these were controllable. The larger impacts that were both unexpected and less controllable were the stakeholder, third-party, and real estate acquisition issues and their impacts upon the project definition. Other project-definition strategies that contrib- uted to the control of cost and schedule were value engineering and design-to-budget. Value engineering activities at each phase of project development helped to control project costs by refining the design in con- sideration of project cost factors. A design-to-budget approach begins design with a fixed budget in mind. This strategy also appeared to contribute to better cost and schedule control and cost-containment results. Thus, the project-definition strategies that con- tributed the most success to the project-definition process were a transparent development process with extensive stakeholder input, a reasonable project- development schedule that reflects sufficient time for stakeholder outreach, a value engineering exer- cise at each stage that reconsiders the definition re- sults to that point, and a design-to-budget approach that maintains budgetary considerations within each stage of project development. Procurement Within the sample set of projects, the reasons for selecting a particular method of procurement varied from past experience with the chosen method, state and local procurement regulations, and interest in achieving a compressed developmental schedule. Design-build and a variant, construction manage- ment at risk offer some potential cost and schedule savings opportunities by (a) consolidating design and construction management with the construction contractor and (b) reducing agency procurement and project management requirements. However, these savings can be offset by increased project definition to more clearly establish the project expectations be- fore final design is complete. The construction man- agement at risk approach also offers similar consol- idation of construction and installation contracts without the inclusion of the final design to gain some of the schedule compression and procurement re- ductions, and without some of the project-definition concerns of design-build. Many of these more recent approaches to project delivery are still under refine- ment, but seem to offer measurable benefits to the completion of projects within planned schedules and closer to initial cost estimates. The researchers found that refinement of these approaches may take addi- tional time. Other procurement techniques include prequalifi- cation of contractors and industry review of contract documents. Prequalification helps to ensure quality and past performance of the contractors. The industry review helps to improve the contracting terms and project management approach within the project cost objectives. Incentives and penalties may also offer some advantages to the project-development process, but concerns were raised about the impacts of these measures. The benefits of these procurement incen- tives were highlighted, but could not be demonstrated through these higher-level cost results. Project Organization It was found that having a common goal be- tween the owner and contractors of building a high- quality and safe project on time and within budget supported better project outcomes for all parties. The approaches to roles and responsibilities to achieve success were varied. However, the organizational approaches listed below appeared to be commonly followed: • Develop a good and amicable working rela- tionship with other third-party organizations and the contractor through partnering; • Select and maintain the right staff with dedi- cation to the project; • Limit distractions and identify priorities; • Emphasize leadership attention, involvement, and support including senior staff from all owner and contractor participants, along with ongoing outreach with stakeholder represen- tatives; and • Partner with contractors and stakeholders to develop a common sense of project ownership. These approaches can be achieved through eq- uitable sharing of the responsibilities and credit for project successes. 3

Capital Cost Estimation Cost and schedule estimation are important func- tions during all phases of the development cycle. The importance of good, early cost and schedule esti- mates is particularly important, since they often set internal and external expectations. At the same time, more cost and schedule uncertainties occur during the planning phases of a project. Thus, the cost esti- mator is faced with trying to estimate capital costs without sufficient project definition. Most projects used a deterministic cost-estimating method. This approach synthesizes hundreds or even thousands of assumptions into one estimate for cost. The deterministic method uses many sources and data points that can account for varying under- lying assumptions. As the project develops, it is quite difficult to maintain control over these many inter- related project characteristics. Change in individual aspects can substantially affect the cost estimate; yet consistent management of these various project char- acteristics and maintenance of these impacts on the cost estimate is illusive. One of the techniques most mentioned for better cost estimation was the development and mainte- nance of historical bid estimates on a unit basis. This source of cost information was noted as a good com- parative source for these actual as-built costs. These bid estimates were used to compare with the cost build-up estimates developed at the engineering level to ensure full inclusion of all project requirements and use of reasonable unit cost estimates. A probabilistic cost and schedule estimation ap- proach integrated with a detailed risk assessment process may better convey the assumptions in- volved in the cost estimate and the risks reflected in the cost estimate. This approach was developed as an improvement to the deterministic cost-estimation approach to focus better on (a) the key risk items and (b) the project unknowns that affect the project capital cost. Project Management Project management controls include contract mechanisms for schedule, quality, claims, testing, change orders, subcontracting, progress payment, and closeout. They are also defined by the roles and responsibilities of the owner and contractor, with the aim of ensuring the successful implementation of the project. Project management controls are thought to be influenced by the following: • Nature of the project (e.g., technology, com- plexity); • Type of contract (e.g., ownership, financing); • Size and scope of the project; • Experience of owner agency staff; and • Project setting (e.g., new start project or ex- tension of an existing line). The functions of project management control in- clude project budget and schedule control, change order and claims management, quality assurance and quality control (QA/QC), and risk management. Project Budget and Schedule Control Project budget and schedule control are functions performed by the management, scheduling, and ac- counting systems. The related functions include ver- ifying budget to actual costs, level of detail and sepa- ration in work breakdown structure, consistency with schedule and progress reporting, and the require- ments included in subcontractors’ contract terms and clauses. Project schedule management also considers contractor and agency interface, response capabili- ties, and incentives and disincentives including im- plications for payment. These budget and schedule control functions have become increasingly important to the process as the computer systems capabilities have increased. More control is exercised through these systems and they are providing increased visibility into these issues. Reporting the contractor’s cost and schedule perfor- mance within an electronic format on a monthly basis with the major milestones identical to those in the master schedule provides essential oversight of the budget within a detailed task basis. This process has been reported to be contributing to improved cost- containment performance when focused on the risk issues in a timely basis. However, as evidenced by these results, the schedule control and payment strategy did not deter- mine the success of the project in maintaining either. The contractor results were slightly better when con- tractors were assigned responsibility, but only nomi- nally. The actual schedule control and payment strate- gies appeared to have better performance when the payment approach was tied to specific schedule and cost outcomes. Whether the agency or the contractor, or a shared relationship was used, the outcome was more determined by the quality of the approach rather than the specific approach to schedule and cost control. 4

Change Order and Claims Management Scope changes should be defined early, estimated, negotiated, and settled in an expeditious manner. Ideally, the project has estimation staff based onsite along with regular support from the engineering and procurement staff for claims negotiations. Design costs can be controlled by releasing the change order for design only with review milestones, if possible. In addition, clear definition of the review process with timely responses was viewed as essential. Most agencies reported that, as a first step, claims were initially addressed through bilateral negotia- tions. Most projects had contractual provisions for an alternative dispute resolution board or the like, but only two projects used it. The five projects that used alternative dispute resolution or had provisions for it appeared to exhibit better average cost variance due to scope or schedule changes. Timeliness of responses to proposed changes is essential. A project may also have an expedited re- view process available in case schedule adherence is threatened. An attempt was made by some projects to resolve all claims at the lowest level possible un- less there was disagreement. Empowering the field staff while maintaining adequate financial authoriza- tion is important for fast resolution of issues arising during construction. Quality Assurance/Quality Control Addressing quality issues at an early stage helped to avoid unnecessary complications. Several innova- tive quality approaches were identified, including application of a QA/QC manual, a just-in-time train- ing program, preparatory phase inspections, an inde- pendent testing program, and use of specific metrics to monitor and recognize a contractor’s effort on quality. Risk Management Risk management entails (a) the comprehensive identification, assessment, and mitigation of risks and responsibilities to the parties involved at early stages of project development and (b) the subsequent monitoring of these identified risks throughout the project development process. The assignment of the risks to the contractor or agency (or shared responsi- bilities) and the subsequent division of risk manage- ment roles and responsibilities between the owner or agency and the contractor is a key consideration. Risk mitigation is aided through the assignment of individual risks to the party best able to manage it. Also, assignment of blind risks to the contractor does little to minimize cost risk and much to increase bid premiums. Recent federal policies have begun re- quiring detailed risk assessments for major transit projects. Moreover, relating the risk assessment process to the development of specific line-item con- struction contingencies may be an important step in the development of the risk assessment process. The process to combine these aspects of risk identifica- tion, mitigation, and management is being refined through much testing and experimentation within the projects. The expectation is that a consolidated process will evolve that considers this entire risk management process within the cost containment objective. STRATEGIES, TOOLS, AND TECHNIQUES The following conclusions address the five ques- tions originally raised in the research plan. These potential improvement strategies draw from the case studies examined as part of this research. Where pos- sible, supporting statements have been included from the case studies and the literature review. 1. What can local project sponsors do to esti- mate costs and schedules reliably at the alternatives analysis, preliminary engineer- ing, final design, and construction phases? • Use Reasonable Starting Assumptions— In the early phases, it is imperative that sponsors establish a realistic project scope and schedule based on actual needs and known constraints (e.g., budget, deadlines, regulations). These estimates often set initial expectations and form public opinion. A better-defined scope and schedule can help minimize uncertainties during estimation. When uncertainties are unavoidable, it is imperative to communicate the assumptions and limitations of the estimates in a way that is easily understood. These uncertainties could also be accounted through targeted contingencies. The means to accomplish this are to sup- port continued research into the actual costs and schedules of these projects. Early risk assessments could also provide a method to 5

achieve this. Thus, continued research into the actual schedules and as-built costs will help to ensure that realistic schedules can be used, as-built unit costs are available, and all of the typically required project scope items are identified for project planners and engi- neers. Also, these data should be accessible and updated periodically. • Improve Estimation Quality—The case studies show that estimates can be im- proved in all phases of development using formal estimation manuals and through re- views or validations that are comprehen- sive and independent. Cost-estimation data, or as-builts, from previous projects may be shared or pooled to improve over- all data quality and reduce uncertainty as noted above. The cost-estimation staff should consistently represent the disci- plines of the wider project team, including senior staff, and use formal cost-estimating models or systems. • Increase Estimation Transparency—A process that involves the public and is clear to all stakeholders at all phases is more likely to produce reliable estimates. Docu- mentation on estimate assumptions and methodologies that are available to the stakeholders and public may attract wider support and less risk of future change later in the project development process. More- over, the integrity of the cost estimation can be assured by a system of checks and balances through the use of independent cost estimators, risk assessment processes, independent value engineering processes, and formal cost-estimate validation pro- cesses. Also, an incentive structure can be developed and implemented to encourage agencies to have more accurate project es- timates; and possibly accelerate the project review process. 2. What can local project sponsors do to con- tain project costs and schedules at each phase? • Optimize Project Parameters—It is im- portant for all project stakeholders to recog- nize the tradeoffs and interrelationships be- tween scope, schedule, and quality to cost. During the alternatives analysis and prelim- inary engineering phases, several optimiza- tion techniques have proven to be effective at increasing the project’s overall “value proposition” by containing costs while not significantly compromising scope, schedule, quality, or the anticipated benefits. Among these techniques are design-to-budget, value planning, value engineering, and risk assess- ment approaches. • Apply a Broad Range of Project Manage- ment Controls—During implementation, the scope, schedule, and QA/QC controls used by project managers varied but usually included each of the following elements: • Problem Resolution—The quality test- ing process needs to focus on resolution of the root cause rather than solely on the symptoms. • Efficient Processing—Changes should be identified early, estimated, negotiated, and settled in an expeditious, predefined process. The change order and claims management process should be clear and fair to all parties. • Appropriate Incentive Structure— Incentives and penalties can be applied to payment mechanisms for schedule management or change order processes to empower staff with appropriate au- thority for change control. These incen- tives need to be crafted carefully to focus the effect on the quality outcome and limit the contractor payment without measurable progress. • Create an Effective Organization—The project organization must ensure that re- sponsibilities and risks are placed on the parties that can best affect them. It must also ensure that there is an appropriate level of technical capacity. The effective organization may be achieved through partnering and a dedication to continuous open communications between the owner (including Board of Directors), contrac- tors, consultants, and FTA representa- tives. Integrating the project team in both design and construction can also enhance project ownership. These concepts have shown to be beneficial in several case studies. 6

3. What can local project sponsors do to com- plete projects within their estimated costs and schedules? • Select the Appropriate Delivery Method—The selection of the appropriate procurement and delivery method is im- portant to the success of the project devel- opment. This selection is best based on the desired level of responsibility, risk, con- trol, expertise, and scope flexibility. With the many procurement approaches avail- able, this assessment of the project charac- teristics and agency culture can form the basis to this decision. The delivery method may impact cost and schedule by affecting the required level of project management, oversight, and construction management. Public-private partnerships are a recent positive trend. • Recognize the Tradeoffs in Contracting— A single, larger contract may decrease competition and increase construction prices in the local market. Multiple smaller contracts increase competition, but also in- crease the number of potentially costly project interfaces and management com- plexity. The right balance must be struck to ensure that projects are segmented ap- propriately and to take advantage of dif- ferent delivery methods, technical capabil- ities, project characteristics, and local market conditions. 4. What changes outside of the direct control of project sponsors could foster the reliable estimation of project costs and schedules? • Improving Estimation Consistency— Standard procedures for estimation in the industry may help minimize the inconsis- tencies observed in estimates. One exam- ple is the standard use of midpoint of con- struction, year of estimate (YOE) dollars for reporting capital cost estimates with re- alistic schedule assumptions. A consistent and appropriate basis for cost estimation can also help lessen the problem of incom- patible estimates between projects and the impact of missed project elements and low unit costs. The omission of some relevant project elements and their reasonably ex- pected unit costs, particularly in the early phases of project estimation, has been a contributing factor to the negative public perception of large cost overruns. Building a cost and schedule database of as-built costs using a consistent capital cost database structure could contribute to improving cost- estimation consistency. • Communicating Uncertainties and Limi- tations in Estimates—Some uncertainty cannot be avoided, and estimates must re- flect all relevant project risks. Deterministic cost estimates with general contingency fac- tors are a crude way of recognizing the ex- pected variability. A detailed, probabilistic risk assessment is becoming a more com- mon and accepted method of identifying risk factors and quantifying their potential impact on project development. The recent federal requirement of a formal risk assess- ment process could improve both the relia- bility and public confidence in project cost estimates. A consistent process that links these identified project uncertainties to a fo- cused risk management process could be one improvement opportunity. This would formalize the risk process into identification, quantification, allocation, mitigation, and management. One example of a formal risk assessment approach used in Washington State is the Cost Estimate Validation Pro- cess (CEVP)™. The subsequent develop- ment of construction phase cost contingen- cies related to the remaining risk items may be the appropriate direction for increasing the reliability of the project cost and sched- ule estimates. 5. What changes outside of the direct control of project sponsors could help contain the costs and schedules of major public trans- portation projects? • More Effective Public Outreach and Stakeholder Engagement—In general, projects are recognizing that early and con- stant communications with stakeholders maximize public support and build political 7

consensus. However, this effort could be made more effective by enhancing the pub- lic understanding of the basic tradeoffs among project scope, schedule, quality, and costs. Although these tradeoffs are some- times dependent on specific project condi- tions, it may be more effective to establish and promote a common framework for communicating project constraints to all stakeholders. In this manner, project spon- sors can more clearly explain to stakehold- ers the reasons for any cost variability through the phases of project development. The charts developed for the case studies and presented in Appendix C of the final report are examples of such a framework. (See examples in Attachments A and B of this digest.) This framework can also in- clude presentation materials regarding the modal options, the more specific project characteristics of station layouts, vehicle and systems options, and the challenges of third-party expectations from the project. With increased availability of as-built cost and schedule databases, the cost impacts of these options can better guide the public debate on these options. • Improved Funding Mechanisms—As federal funding for new projects has de- clined on a percentage basis, local, state, and private funding and public-private partner- ships are becoming more common in transit project development. This trend may help to better control project costs and schedules because it introduces greater accountability at the local level. However, funding mecha- nisms can be improved by adding flexibility in order to meet the needs of new project delivery mechanisms developed to supple- ment the decline of federal funding. Fund- ing needs are greater for the design-build project delivery options to meet the specific delivery payment schedules and the com- bined sizes of the contracts. FUTURE RESEARCH This section presents several ideas for continued research efforts into the important area of cost esti- mation, management, and control, along with ad- vancement of certain ideas identified throughout this research. Continued Study Research Ideas for continued study research efforts in- clude the following: • Expand the sample size for the project cost and schedule analysis to build a more statisti- cally reliable basis, • Complete more case studies into the more successful tools and techniques used to con- trol cost and schedule, and • Develop an outreach workshop of interested project development experts to contribute to the identification and validation of tools and techniques and their application process. Suggested Transit Research Ideas for transit research efforts include the following: • Continue to refine the consistent database structure for all capital cost and schedule re- porting requirements throughout the project- development process; • Develop the guidance necessary to require that all project cost and schedule reporting conform with that structure; • Develop a uniform project element structure for this project data and provide element defin- itions for its context within the cost-estimating and historical bid cost databases; • Develop a “basis of estimate” document from this project element structure; • Continue and enhance the documentation of the as-built cost and schedule databases; • Expand the analysis capability of these data- bases to examine project cost risk items and values; • Increase the development of the risk assess- ment process, as follows: • Analyze technical analysis process improvements, • Capture schedule risks, • Better link the risk items to the project bud- getary line items, 8

• Examine the optimal roles and responsibil- ities within the process, • Consider the scheduling of the assessment efforts to coincide and better support the project development process, and • Examine the potential application of the risk estimates as construction cost contingencies; • Review current project management oversight guidance procedures to look for opportunities to emphasize cost and schedule management within the process; and • Examine the potential application of the cost validation processes, such as the CEVP™, to the transit project development process. 9