Contingency Factors to Account for Risk in Early Construction Cost Estimates for Transportation Infrastructure Projects (2022)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

30 Guidelines on the Historical Data-Driven Approach for Estimating Construction Contingencies The rapid development and application of digital technologies in the last two decades have transformed DOTs into information technology-driven and digital data-rich organizations. Many DOTs have accumulated a significant amount of digital data across different divisions and offices. Project cost estimates and cost performance data are some of the core digital datasets that DOTs have collected and stored. A historical data-driven construction contingency estimation will allow cost estimators to have confidence in their contingency numbers. The CDF of con- struction contingency estimates, which is the main output of a historical data-driven approach, will specifically serve as a back-check tool for validating construction contingency estimates from the risk-driven approach described in Chapter 5. This chapter provides general guidelines for performing a construction contingency estima- tion using historical cost data available in DOTs. An overall process along with step-by-step work tasks is described for easy implementation. A case study using a DOT’s cost data demonstrates how the proposed approach works in a real setting, the results it produces, and how to utilize outputs in estimating construction contingency. 6.1 Overall Process Figure 13 shows the overall process of the data-driven construction contingency estimating approach. This process is designed to allow cost estimators to determine construction contin- gencies by major project characteristics such as project type, size, and location. 6.1.1 Historical Data Collection—Data Availability The quantitative approach starts with gathering historical cost data for each of the project development phases. Since this approach determines construction contingencies by analyzing cost variation patterns throughout the project development process, DOTs need to collect and provide historical cost data from the early planning stage to the end of the project. In particular, the following datasets are required: (a) early cost estimates in the scoping phase, (b) engineer’s estimates in the project design and PS&E development phase, and (c) final construction costs at project close-out. Figure 14 displays a general project development process for a typical design– bid–build project and the three cost data collection points required for the approach. An early estimate is one that is developed based on preliminary project plans in the project planning and scoping phase. The early cost estimates must be broken down to base estimate and contingency at a minimum. A base estimate is the sum of major work item costs and allowance. Typically, a DOT’s office of project planning and development maintains the early C H A P T E R 6

Guidelines on the Historical Data-Driven Approach for Estimating Construction Contingencies 31 cost estimate data. The engineer’s estimate, which is developed based on a complete project plan, specification, and highly detailed drawings, is the final cost estimate before advertising a project. The engineer’s estimate has a complete list of pay items with their quantities and historical unit prices. The contract office can typically provide engineer’s estimate data. Final construction cost data include pay item-level cost data along with adjusted payment amounts resulting from change orders and claims during the construction phase. The final construction cost data are typically available in a contract administration system such as AASHTOWare SiteManager. Table 7 shows the ideal data attributes for the data-driven method. 6.1.2 Data Cleaning The data collected from different offices in Step 1 are combined to create a master database. Ideally, historical projects must have all cost data attributes mentioned in the previous step. Figure 13. An overall process for quantitative construction contingency estimating. Figure 14. Three types of cost data for the quantitative approach.

32 Contingency Factors to Account for Risk in Early Construction Cost Estimates for Transportation Infrastructure Projects However, data cleaning is necessary to eliminate (a) historical projects that may have missing data points and (b) unnecessary data attributes available in the raw dataset. The data cleaning process filters out projects that have undergone scope changes between the scoping and final design phases, since scope change typically requires a new baseline cost estimate and thus is not covered by construction cost contingency. An attribute indicating any scope change in the database may facilitate the process of eliminating those projects. Otherwise, the agency must establish a data cleaning guideline to properly eliminate those scope change projects by comparing early estimates and engineer’s estimates. Data preprocessing is done to reorganize raw historical cost data into a proper format for direct comparison of project costs in different project phases. As a project progresses from planning to design, pay items become finalized. Pay item quantities and unit prices may change to reflect the final design and project conditions. Also during this process, a pay item may change to a different pay item, or it may be broken down into multiple pay items under the same work category. Grouping pay items into a small number of work activities might be necessary for equivalent cost comparison. 6.1.3 Major Project Factors to Be Considered Step 3-1: Calculate the Difference between the Base Cost Estimate and the Final Construction Cost In this step, the difference between the base cost estimate from the scoping phase and the final construction cost at the end of construction is measured for each project. This difference can be expressed as a ratio of the final construction cost at the end of the project to the base cost estimate in the scoping phase, as shown in the formula that follows. Note that the base cost estimate does not include the construction cost contingency estimated at the scoping phase. ( ) = −    ×RFB % Final construction cost at the end of project Base cost estimate at the scoping phase 1 100 In which RFB is the Ratio of the Final construction cost to the Base cost estimate. The RFB indicates the percentage of the base estimate that should have been used to allocate the construction contingency amount at the scoping phase to fully cover any construction cost changes during the design and construction stages for the particular project. Step 3-2: Determine Project Characteristics Major project characteristics such as project type, location, complexity, and size may signifi- cantly influence the construction contingency amount since they may pose different types of Cost type Ideal Data Attributes Early estimate Base estimate (major work items + allowance) Construction contingency Engineer’s estimate The entire list of pay items with their quantities and unit prices Final construction cost The entire list of actual pay items with their quantities and unit prices Change orders Table 7. Ideal data attributes for the data-driven method.

Guidelines on the Historical Data-Driven Approach for Estimating Construction Contingencies 33 risks and uncertainties. The master database of historical projects can be reorganized by these project characteristics. Figure 15 shows an example of the Florida Department of Transportation (FDOT)’s project classification using project type and project size. In this example, highway projects are categorized into eight project types (roadway new construction, reconstruction, resurfacing, rehabilitation, relocation, bridge new construction, replacement, and rehabilitation) and three project sizes (large, middle, and small). Since DOTs have established their own project classification systems that consider the common project characteristics in their states and business practices, it is suggested that each agency adopt its own standard project classification system. However, a large number of project types may lead to a small number of projects of each classification in the database; this may lead to unreliable modeling results. In this case, a logical grouping of similar project types to a smaller number of classifications is suggested. Step 3-3: ANOVA Testing for Project Characteristics In this step, the analysis of variance (ANOVA) test is performed to determine whether different project characteristics may lead to statistically different patterns of construction contingency amounts. If determined to be different, those characteristics must be kept in the master data- base and different distribution curves of construction contingencies must be developed and applied for a particular project type. The ANOVA test can be performed using statistical software (e.g., JMP, Minitab, or IBM SPSS) or spreadsheet software (e.g., Microsoft Excel). For example, new construction projects typically involve more risks than simple repair projects owing to their high project complexity, thus leading to a higher amount of construc- tion contingency than that of a simple repair project. The ANOVA test can statistically verify whether this assumption is true or false based upon historical cost performance data; it can also tell whether or not a different distribution curve of construction contingency should be developed. The ANOVA test is a statistical hypothesis-testing technique used to test the equality of two or more populations (Cuevas et al. 2004). The ANOVA test uses the p-value to determine whether the differences between some of the means are statistically meaningful. In a hypothesis test, p-values are used to help decide whether to reject the null hypothesis. The smaller the p-values, the more likely it becomes to reject the null hypothesis. The null hypothesis of the ANOVA test is that there is no difference among different group means. The alternative hypothesis is that at least one group mean is different from the rest. The p-value, which is less than a statistically sig- nificant level, can serve as solid evidence against the null hypothesis. For example, if a significant Figure 15. Categorization of project characteristics.

34 Contingency Factors to Account for Risk in Early Construction Cost Estimates for Transportation Infrastructure Projects level is 0.05 and the p-value is less than 0.05, the null can be rejected and the alternative can be accepted because there is less than a 5% probability that the null hypothesis is correct. If the p-values of ANOVA tests for two characteristics are less than 0.05, these characteristics are considered significant factors in developing different probability distribution curves of construction contingency for the factors. 6.1.4 Contingency Range Estimation Step 4-1: Probability Distribution Fitting In this step, a distribution fitting is applied to determine the most appropriate distribution curve that can explain the distribution of RFBs of historical projects for project characteristics, including project type, size, or location (Liu and Liu 2011; Van Dyke et al. 2017). Distribution fitting is the selection of statistical distribution curves that best fit an observed dataset. These probability distribution curves can quantitatively model uncertainty associated with construction contingency amounts based on historical data. As a common method of distribution fitting, the goodness-of-fit test is a statistical hypoth- esis test for selecting a particular distribution that best fits with the observed data (Huang and Prokhorov 2014). This test can (a) summarize the deviations between the observed values and the specified types of statistical distributions and (b) determine the most fitted distribution with the slightest deviations (Aslam 2021). The goodness-of-fit test typically provides p-values for various distribution curves. Specifically, the p-value is used to measure the confidence of evidence against the hypothesis that the data do not follow the distribution. A significance level of 0.05 can be used as a cutoff value to reject this hypothesis. A significance level of 0.05 indicates a 5% risk of concluding that the observed data do not follow the distribution. If the p-value of a specific distribution is less than 0.05, it means that the RFBs do not follow the distribution. On the other hand, a p-value of over 0.05 indicates there is no evidence that the data do not follow the distribution. Probability distribution fitting calculates p-values for six different PDFs, as shown in Figure 16. Distribution fitting provides p-values for all candidate distribution curves. The largest p-value serves as an indicator for identifying the most fitted distribution curve for observed data. Multiple statistical systems that can perform goodness-of-fit tests include R statistics, SAS, IBM SPSS, JASP, and Minitab. For the example that follows, the Minitab statistics package was used. Step 4-2: Determine Distribution Parameters This step determines distribution parameters for developing distribution curves for projects in different categories of project characteristics. Distribution parameters refer to descriptive measures that serve as the inputs for determining the distribution curve’s shape, shift, and skewness. Each distribution curve is defined by multiple specific parameters, and the combi- nation of parameter values creates a unique distribution curve. Table 8 presents parameters for candidate distribution curves. Most statistical systems also determine parameters correspond- ing to distribution curves and can visualize the curves with measured parameter values. In this process, an agency can develop its unique distribution curves of construction contingencies that are derived from its historical projects’ cost performance data. • Mean (μ): shifts distribution curves left or right on the horizontal axis. • Standard deviation (σ): determines the width or spread of the curve. The greater the standard deviation, the greater the stretching. The value less than one is to compress the curve. • Shape (β): allows distribution to take on a variety of shapes, depending on the value.

Guidelines on the Historical Data-Driven Approach for Estimating Construction Contingencies 35 Figure 16. Candidate curves for probability distribution fitting. Distribution Parameter 1 Parameter 2 Parameter 3 Normal mean (μ) standard deviation (σ) N/A Lognormal mean (μ) standard deviation (σ) N/A Exponential mean (μ) N/A N/A 2-parameter Exponential mean (μ) threshold (γ) N/A Weibull shape (β) scale (η) N/A 3-parameter Weibull shape (β) scale (η) threshold (γ) Gamma shape (β) scale (η) N/A 3-parameter Gamma shape (β) scale (η) threshold (γ) Loglogistic shape (β) scale (η) N/A 3-parameter Loglogistic shape (β) scale (η) threshold (γ) Note: N/A = not applicable Table 8. Parameters for candidate distribution curves.

36 Contingency Factors to Account for Risk in Early Construction Cost Estimates for Transportation Infrastructure Projects Cumulative probability Figure 17. Convert PDF to CDF. • Scale (η): affects the variability of the curve. As the scale is increased, a curve stretches further right and the height decreases. • Threshold (γ): describes the shift of the curve. A negative value shifts the curve to the left, and a positive value shifts it to the right. Step 4-3: Develop PDFs and Convert PDFs to CDFs The probability distribution curves can be easily developed with the parameters determined in Step 4-2 (see Figure 17-left). The PDFs from the goodness-of-fit test can be converted into CDFs (see Figure 17-right) for a more straightforward interpretation of model outcomes. CDFs can give a cumulative probability less than or equal to a specific value. The area under the PDF curve below the specific RFB (35%) is equal to the y-axis value of the RFB on the CDF. Thus the CDF can be used to straightforwardly identify the cumulative probability of RFB; this can serve as the certainty information related to construction contingency estimates. Step 5: Application to Real Projects Once a PDF and a CDF are developed for each project type, an estimator can use them to directly estimate a construction cost contingency using the base estimate of the project in the scoping phase. For example, using the CDF in Figure 17, an agency can achieve the certainty of 75% (i.e., 75% cumulative probability or 0.7484 on the y-axis) if 35% of the base estimate is allocated for the construction contingency amount (see the value of 35% on the x-axis). This information implies that based on the historical projects’ cost performance, there is a 75% probability that 35% of the base estimate will be sufficient to cover additional construction costs and deliver the project on budget. 6.2 Case Study This section provides an in-depth case study of when the overall process described in section 6.1 is applied to a DOT’s historical data. The intent of this case study is to demonstrate the entire process with real data, show practical issues and concerns that DOTs may encounter when they want to implement a historical data-driven construction contingency estimation approach, and explain how those issues can be handled effectively. This case study also offers a

Guidelines on the Historical Data-Driven Approach for Estimating Construction Contingencies 37 practical idea on how the results can be compared with the results from the project risk-driven contingency estimating approach described in Chapter 5 for a more defensible and reliable construction contingency estimation. 6.2.1 Data Collection The master database that was built for the case study consists of a DOT’s actual cost data, including: (a) early estimates, (b) engineer’s estimates, and (c) final construction costs (see Table 9). The early estimate dataset includes base estimates that are broken down to major pay items, maintenance of traffic (MOT), mobilization, and contingency. Major pay items represent work items that are essential to the project. As parts of allowance, MOT and mobilization are estimated as lump sum items due to the lack of information for detailed estimating at the scoping phase. Contingency is typically estimated by the DOT as a predetermined percentage of the base estimate. The engineer’s estimate dataset includes the entire list of pay items determined with detailed design and project plans and standard specifications for each project. Additionally, the DOT has a unique item as part of the engineer’s estimate, which is original contingency. The original contingency represents an available fund allocated to cover potential cost increases resulting from non-scope-related changes during construction. The final construction cost dataset includes the entire list of pay items with their actual payment amounts and any adjusted costs resulting from change orders. “Actual contingency” refers to the actual amount spent from the original estimated contingency. 6.2.2 Data Cleaning and Data Preprocessing The DOT master database built for the case study includes a large amount of raw cost data from more than 2,200 historical projects. Data cleaning was performed in order to (a) eliminate historical projects that were missing data points—resulting in the elimination of 980 projects, and (b) remove unnecessary and redundant data attributes. Related DataTypes Descriptions Early estimates in the scoping phase Base estimate Major pay items Cost estimate for major pay items Maintenance of traffic Cost estimate for maintenance of traffic Mobilization Cost estimate for mobilization Contingency Contingency estimate Engineer’s estimates in the final design phase Pay items Cost estimate for detailed pay items Original contingency Contingency estimate Final construction costs at the end of the project Pay items Actual paid amounts of pay items Change order Adjusted amounts due to change orders during construction Actual contingency Actual amounts paid out of original contingency Table 9. Descriptions of historical cost data.

38 Contingency Factors to Account for Risk in Early Construction Cost Estimates for Transportation Infrastructure Projects Through this data cleaning, historical projects that had undergone scope changes during the project design phase were also identified. Projects that had significant cost increases greater than their original contingencies or significant cost decreases from the base cost estimate were considered to have experienced scope changes and thus were eliminated from the master database. As a result, 739 historical projects were used for the case study. 6.2.3 Data Analysis and Project Characterization Project Characterization Since the DOT’s master database includes data attributes for project types (as shown in Table 10), its historical projects were first categorized according to the given project types. However, roadway replacement and bridge new construction projects had a small number of data points (15 roadway replacement projects and only 6 bridge new construction projects). Those two project types were therefore eliminated for further consideration. As a result, projects for the case study were categorized into six groups: (a) roadway new construction, (b) roadway reconstruction, (c) roadway resurfacing, (d) roadway rehabilitation, (e) bridge replacement, and (f) bridge rehabilitation. ANOVA Testing The ANOVA test was performed to evaluate whether different project types had statistically different patterns of construction contingencies. Table 11 shows the ANOVA test results. The p-value is the primary variable for interpreting the test results. Statistically, the p-value measures the probability that an observed difference could have occurred just by random chance. If the p-value is below 0.05, which is a typical threshold significance level, there is sufficient evidence that not all group averages are equal. The p-value (0.038) in Table 11 confirms that the average Project Types Number of Projects Roadway new construction 112 Roadway reconstruction 76 Roadway resurfacing 338 Roadway rehabilitation 90 Roadway replacement 15 Bridge new construction 6 Bridge replacement 50 Bridge rehabilitation 52 Total 739 Cases Sum of Squares Df Mean Square F p-value Type 0.553 5 0.111 2.386 0.038 Residuals 16.369 353 0.046 Table 10. Project types included in the case study. Table 11. Summary of ANOVA test results.

Guidelines on the Historical Data-Driven Approach for Estimating Construction Contingencies 39 RFBs for project types are not the same. Thus, the result justifies that the probability distribution curves of construction contingency for six project types need to be developed. 6.2.4 Model Development Model Probability Distribution Fitting Results The goodness-of-fit test results for six project types are provided in Appendix E. The case study used 0.05 as a threshold value for determining whether the distribution of RFBs followed a specific curve. The following are specific findings from the distribution fitting process: • For project types with a small number of historical projects, including bridge replacement and bridge rehabilitation, more than one distribution curve has p-values greater than 0.05. • Roadway new construction and roadway resurfacing, with the largest number of historical projects, have only one or two distribution curves with a p-value of over 0.05. • The 3-parameter Weibull is the only distribution curve with p-values greater than 0.05 for all project types. Because the analysis indicates that larger data sizes may give more reliable results with greater precision, the 3-parameter Weibull was selected for a standard distribution curve. Determination of Distribution Parameters A 3-parameter Weibull distribution is determined by the shape, scale, and threshold param- eters. The shape parameter represents how RFBs are distributed. For example, a shape value of 3 produces a distribution plot similar to a normal standard curve. A low shape value (i.e., 1 or 2) gives a right-skewed curve, and a high shape value (i.e., 8 or 9) provides a left-skewed curve. The parameter of the scale represents the position of the Weibull curve. If the scale increases, the distribution is stretched out to the right and its height decreases. Conversely, a reduced shape parameter pushes the distribution toward the left with an increase in distribution height. A threshold is a parameter for the shift of the distribution away from zero. A positive threshold shifts the distribution to the right, and a negative threshold shifts it to the left. As defined in this report, six project types have three specific parameters for Weibull distribution developments. Table 12 presents the three parameters and PDFs for six project types. 6.2.5 Apply Model to Projects Figure 18 shows the CDFs for six project types, and Table 13 shows the cumulative prob- abilities (or certainty level) when the RFB is 10%, 30%, and 50%. For example, a cumulative probability of 0.67 with 30% RFB indicates that if the contingency estimate is 30% of the base estimate, about 67% of the previous roadway resurfacing projects would have been completed within the estimated budget. This statement can be rephrased as follows: estimators can have 67% certainty if they set the construction contingency estimate to 30% of the base estimate. Appendix F includes the table representing certainty levels for various RFBs for six project types. Roadway new construction projects have the lowest cumulative probabilities, meaning that they may involve more complexity and unexpected risks than other project types. Roadway resurfacing projects have the highest cumulative probabilities when RFBs are 30% and 50%, indicating that these projects show lower uncertainty than other project types. This result may come from the fact that roadway resurfacing projects require the least major work activities. Furthermore, since roadway resurfacing is the most frequent project type, estimators may have experience and professional knowledge to help manage uncertainties associated with them.

40 Contingency Factors to Account for Risk in Early Construction Cost Estimates for Transportation Infrastructure Projects Roadway New Construction Roadway Reconstruction Roadway Resurfacing Roadway Rehabilitation Bridge Replacement Bridge Rehabilitation 1.478 Shape 1.720 Shape 1.527 Shape 21.433 Scale 23.90 Scale 25.032 Scale 8.348 Threshold 3.51 Threshold 2.89 Threshold 1.239 Shape 1.235 Shape 1.446 Shape 23.156 Scale 23.812 Scale 22.798 Scale 3.82 Threshold 5.12 Threshold 5.52 Threshold Table 12. Probability density functions (PDFs) by project type.

Guidelines on the Historical Data-Driven Approach for Estimating Construction Contingencies 41 Leverage Model with the Historical Data-Driven Approach Results Figure  19 shows how the CDF can be used with the results of the risk-driven approach described in Chapter 5. • ① in Figure 19 shows a construction contingency amount calculated from the qualitative approach and converted to the percentage of the base estimate. This percentage corresponds to the RFB developed by the quantitative approach. • ② in Figure 19 shows the CDF curve that provides the cumulative probability information for construction contingency. The cumulative probability indicates the certainty level of the contingency percentage so that the agency can perform a reality check to ensure that the con- struction contingency calculated from the risk-driven approach is within a reasonable range based upon the historical projects’ cost performance data. Figure 18. CDFs for six project types. Types RFB 10% 30% 50% Roadway new construction 0.022 0.638 0.93 Roadway reconstruction 0.177 0.688 0.905 Roadway resurfacing 0.101 0.697 0.966 Roadway rehabilitation 0.132 0.652 0.888 Bridge replacement 0.136 0.677 0.928 Bridge rehabilitation 0.091 0.670 0.927 Note: Bolded probabilities indicate the largest values for RFBs of 10%, 30%, and 50%. Table 13. Cumulative probabilities when RFB is 10%, 30%, and 50%.

42 Contingency Factors to Account for Risk in Early Construction Cost Estimates for Transportation Infrastructure Projects Figure 19. Leverage model outcomes with the result of historical data-driven approach.