Improving Mid-Term, Intermediate, and Long-Range Cost Forecasting: Guidebook for State Transportation Agencies (2020)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

8 Framework for Selecting a Cost Forecasting Approach 2.1 Introduction The framework for selecting a cost forecasting approach presented in this chapter is intended to serve as a map to guide planners and estimators through different sets of guide- lines and tools according to the unique set of requirements, preferences, and constraints of each state transportation agency (STA) in terms of data quality and availability, information technology (IT) and staff capabilities, the intended cost forecasting time horizon, and risk tolerance. The five-module framework is intended to be the first stop for any STA interested in implementing the guidance and tools described in this guidebook. The flow chart in Figure 2-1 shows the role of each module as part of the overall cost forecasting process. In summary, the planning team starts with Module 1, which assists STAs with the selection of a suitable cost indexing alternative that meets the needs and capabilities of the agency. The agency could opt for the use of a standard inflation rate without the analysis of a cost index. In that case, the planning team would be referred to Module 2, which is a compilation of annual inflation cost indexes that showed an effective performance for the three case studies conducted under NCHRP 10-101 (Rueda-Benavides et al. 2020). If an index-based forecast- ing process is selected in Module 1, the framework would direct STAs to Module 3, 4, or 5, depending on whether the intended cost forecast corresponds to a midterm, intermediate, or long-range forecast, respectively. 2.2 Module 1: Cost Index Selection Module 1 is illustrated in Table 2-1. This is the first step in the process of selecting a cost forecasting approach. This module is intended to guide the planning team in the selection of the cost indexing alternative that best fits its needs, preferences, and capabilities. Module 1 details the requirements and implications associated with the cost forecasting approaches listed in Table 2-2. This table also shows the four major aspects considered to assess the implications and requirements of each indexing alternative. The primary conclusion drawn from Module 1 is that the higher the level of resources and efforts invested in the improvement of cost forecasting procedures, the higher the effectiveness of the cost forecasting achieved by the STA. However, that statement is only valid if cost forecasting resources and efforts are properly and thoughtfully invested following the guidelines presented in this document. 2.3 Module 2: Standard Inflation Rate Selection This module is summarized in Table 2-3. The module provides low, medium, and high standard annual inflation rates for midterm, intermediate, and long-range forecasting processes for both asphalt and concrete paving activities. Those standard inflation rates are provided with C H A P T E R 2

Framework for Selecting a Cost Forecasting Approach 9 their associated forecasting error ranges. Module 2 is intended for agencies that decide not to calculate applicable inflation rates from the internal assessment of a cost index but rely instead on other STAs’ experiences. The three levels of magnitude for construction market inflation (low, medium, high) were identified on the basis of the three case studies conducted under NCHRP 10-101 (Rueda- Benavides et al. 2020). They were considered in Module 2 to facilitate more effective forecasting outputs for STAs on the low and high ends of the spectrum. The use of medium annual inflation rates could be considered by agencies that do not have reliable information from which to infer the level of magnitude of upcoming inflation rates. However, it should be noted that the forecasting error ranges in Table 2-3 are applicable under the assumption that the agency would appropriately place itself in one of the inflation magnitude categories. An error in doing so would increase the level of uncertainty and thus widen the forecasting inflation rate. To illustrate, for an intermediate-range forecast of concrete paving activities, if the agency considers that the inflation rate is going to be high for this forecasting period, Module 2 would suggest a compound inflation rate of 4%. On the basis of the forecasting error ranges in Table 2-3, the agency could expect, with a 90% confidence level, a ±30% forecasting error. It should be noted that this error is associated with the forecast value. For example, if the forecast estimate obtained for a given concrete paving program with this inflation rate is $10 million, the agency could expect, with a 90% confidence level, to have an actual program cost between $7 and $13 million. Although Module 2 constitutes an improvement in the quality of guidance for users of standard inflation rates, a better cost forecasting performance can still be achieved by calculating annual inflation rates through more formal quantitative procedures that use the guidelines provided in this guidebook. Formal quantitative approaches allow a more effective calculation of annual inflation rates on a case-by-case basis and result in a considerable reduction of cost forecasting uncertainty. Module 4. Intermediate-Range Forecasting Method Selection Standard or Index-Based Inflation Rate? Forecasting Time Horizon Index-Based Module 1. Cost Index Selection Module 2. Standard Inflation Rate Selection Standard Module 5. Long-Range Forecasting Method Selection Module 3. Mid-Term Forecasting Method Selection Intermediate-Range Horizon Long-Range Horizon Mid-Term Horizon Figure 2-1. Overall framework for selecting a cost forecasting approach.

M ai nt en an ce De ve lo pm en t Data Requirements & Considerations High In-House Multilevel Construction Cost Index (MCCI) In-House Construction Cost Index (Traditional CCI) External Construction Cost Index Standard Inflation Rate (No CCI) Effectiveness in Tracking Market Price Changes Medium Low N/A High Effectiveness in Addressing Project-Specific Requirements and Geographic Conditions Medium Medium-Low Low Data Management Effort for Development, Maintenance, & Implementation • Reasonably constant stream of in-house historical bid data from a large set of representative basket of pay items • Capable of handling certain level of missing data • Constant stream of pricing data for a small set of representative basket of pay items • Limited ability to handle missing data • Reliable CCI source with a record of consistency in terms of quality of index and timely publication of updates • Limited ability to handle missing data N/A High Medium Low None Staff and IT Efforts for Development & Maintenance High Medium Low Very Low Staff and IT Requirements and Considerations • In-house or outsourced advanced mathematical and statistical skills • Above-average computer hardware specifications and processing capacity • Spreadsheet applications are sufficient • STA’s good data management practices facilitate MCCI development efforts • In-house or outsourced average mathematical and statistical skills • Average computer hardware specifications and processing capacity • Spreadsheet applications are sufficient • STA’s good data management practices facilitate MCCI development efforts • A staff member or office/ group should bear the responsibility of tracking updates on the CCI by the external entity • In-house or outsourced average mathematical and statistical skills • Average computer hardware specifications and processing capacity for maintenance • Spreadsheet applications are sufficient, but STAs are encouraged to develop customized IT applications to automate MCCI maintenance • STA’s good data management practices facilitate MCCI maintenance efforts • A staff member or office/ group should bear the responsibility of checking on a regular basis with the entity suggesting the adopted standard inflation rate (if any) for possible changes in this recommendation N/A Table 2-1. Module 1: Cost index selection. Cost Forecasting Approaches Assessment Factors • In-house MCCI (see Chapter 3) • In-house traditional CCI (see Chapter 3) • External traditional CCI (see Chapter 3) • No CCI (standard inflation rate) (see Section 2.3) • Effectiveness in tracking market price changes • Effectiveness in addressing project-specific requirements and geographic considerations • Required data management efforts for development, maintenance, and implementation • Staff and IT requirements for development and maintenance Note: MCCI = multilevel construction cost index; CCI = construction cost index. Table 2-2. Module 1: Cost index selection—cost forecasting approaches and assessment factors.

Framework for Selecting a Cost Forecasting Approach 11 2.4 Modules 3 to 5: Midterm, Intermediate-Range, and Long-Range Forecasting Method Selection If the agency decides to use an index-based inflation rate, the next step in the framework would be the definition of the intended forecasting time horizon—midterm (3 to 5 years), intermediate-range (up to 15 years), or long-range (more than 15 years). Modules 3, 4, and 5 illustrate the process of selecting the methodology for midterm, intermediate, and long-range forecasts, respectively. These modules are illustrated in Tables 2-4 to 2-6. These three modules use a flowchart to guide the planning team in the selection of the fore- casting method that would produce effective inflation rates from the cost indexing approach selected in Module 1. In the case of midterm and intermediate-range forecasts, the process begins with a visual inspection of the selected cost index. At least 10 years of index values are plotted, and the resulted time series is then inspected to determine whether the most recent data show an abnormal market behavior, such as a deflation situation or an apparent devia- tion from the regular long-term market trend. These abnormal behaviors are usually referred to as “market corrections.” If that is the case, the planning team, using its knowledge of the agency, the state history, and the local economy, analyzes whether there are any reasons to believe that these market corrections will continue during the intended forecasting period. If so, Modules 3 and 4 would indicate that a more suitable approach could be a regression analysis following the guidelines provided in Chapter 4 and with a lookback period covering only the period with the abnormal market behavior. NCHRP 10-101 (Rueda-Benavides et al. 2020) found that the appropriate use of regression analysis to model market correction in midterm and intermediate-range cost forecasting could narrow the forecasting errors ranges in Table 2-4 by 60% and 40%, respectively. When regression analysis is being used to project market corrections into the future, it is important to consider that most corrections last less than 5 years (according to observations by Rueda-Benavides et al. 2020). Thus, the projection of an observed 3-year downward trend for more than 2 years into the future would anticipate an unlikely scenario. Likewise, the usual length of abnormal market conditions makes it inappropriate to forecast construction costs with 5 years of historical bid data or less. A 5-year lookback period could contain a downward trend that could be projected into the future over a midterm, intermediate, or long-range forecasting period, also representing scenarios that are unlikely. To prevent this issue, this report suggests the use of a lookback period of at least 10 years for both midterm and Note: CL = confidence level. Low Medium High Low Medium High Low Medium High 2% 3%–4% 5% 2% 3%–4% 5% 3% 4% 5% @50% CL @70% CL @90% CL 2% 3% 4% 2% 3% 4% 2% 3% 4% @50% CL @70% CL @90% CL Recommended Rates Expected Forecasting Errors ASPHALT PAVING ±15% ±15% −20% to 30% ±20% ±20% −25% to 35% ±30% ±30% −30% to 45% ±15% Recommended Rates CONCRETE PAVING Expected Forecasting Errors Forecasting Time Horizon Mid-Term (3–5 years) Intermediate-Range (up to 15 years) Long-Range (20 Years) Type of Inflation Rate Annual Simple Annual Compounded Annual Compounded Inflation Expected Level ±35% −30% to 35% −35% to 40% −40% to 50% −25% to 30% −30% to 40% −40% to 45% ±25% Table 2-3. Module 2: Standard inflation rate selection.

12 Improving Mid-Term, Intermediate, and Long-Range Cost Forecasting: Guidebook for State Transportation Agencies intermediate-range forecasts, and between 15 and 20 years (20 years ideally) of historical bid data for long-range forecasting procedures. As mentioned in Section 1.5, simple annual infla- tion rates could be used for midterm forecasts, while compound rates are more appropriate for intermediate and long-range time horizons. If no recent abnormal behavior is identified, or if it is not expected to continue along the intended forecasting period, the moving forecasting error (MFE) method would be more appropriate. The MFE method, which is explained in detail in Chapter 4, is an innovative data- driven cost forecasting approach proposed by NCHRP 10-101 (Rueda-Benavides et al. 2020). This methodology proved to be effective at producing forecast values from the analysis of historical bid data. The MFE method is designed to produce forecasting outputs in the form of risk-based forecasting timelines, such as the one shown in Figure 1-3. All forecasting error ranges in Tables 2-4 to 2-6 are associated with the use of the MFE approach. Those tables show different sets of error ranges for the two different types of work under consideration as well as for two different sources of market data: the innovative MCCI approach or traditional CCIs. A comparison of the latter two reveals a clear reduction in cost Midterm Forecasting Method Selection (with selected cost index) Minimum CCI lookback period 10 Years Type of inflation rate Annual Simple Selection of forecasting approach Expected Forecasting Errors (%) In-House MCCI Confidence level 50 70 90 Asphalt paving ±12 ±20 ±30 Concrete paving ±10 ±15 ±25 In-House or External CCI (traditional CCI) Confidence level 50 70 90 Asphalt paving −20 to 15 −25 to 20 −35 to 30 Concrete paving −15 to 25 −20 to 30 −30 to 40 Table 2-4. Module 3: Midterm forecasting method selection.

Framework for Selecting a Cost Forecasting Approach 13 forecasting uncertainty if an MCCI is used instead of a traditional CCI. Likewise, a comparison between MCCI forecasting error ranges in Tables 2-4 to 2-6 and those in Table 2-3 shows the considerable reduction in cost forecasting uncertainty that an STA could achieve by implementing the proposed MCCI and MFE methodologies. The MCCI forecasting error ranges in Tables 2-4 to 2-6 actually correspond to worst-case scenarios found in the case study results on both ends of each range. An STA would likely obtain narrower forecasting error ranges after the actual application of the MFE method with its own data. Results obtained from the application of traditional CCIs are not as promising as those provided by the MCCI. However, those also represent the worst scenarios across all case studies. A number of cases in NCHRP 10-101 showed a narrower error range with traditional CCIs than those shown in Table 2-3 (Rueda-Benavides et al. 2020). A refined and more reliable forecasting error range would also be obtained after applying the actual MFE method to the intended CCI. Intermediate-Range Forecasting Method Selection (with selected cost index) Minimum CCI lookback period 10 Years Type of inflation rate Annual Compounded Selection of forecasting approach Expected Forecasting Errors (%) In-House MCCI Confidence level 50 70 90 Asphalt paving −10 to 15 −15 to 20 −25 to 30 Concrete paving ±10 ±15 ±25 In-House or External CCI (traditional CCI) Confidence level 50 70 90 Asphalt paving −30 to 15 −35 to 20 −45 to 30 Concrete paving −15 to 50 −20 to 55 −30 to 65 Table 2-5. Module 4: Intermediate-range forecasting method selection.

14 Improving Mid-Term, Intermediate, and Long-Range Cost Forecasting: Guidebook for State Transportation Agencies The main difference between the proposed MFE method and regression analysis techniques lies in their assumptions of risk. The MFE can be classified as a more conservative, or risk- averse, approach, since it produces a cost forecasting output that combines results from several forecasting scenarios created within the available data. However, regression models are the result of a single configuration of the available data, making them more appropriate for risk-seekers who decide to rely on a single scenario and thereby underestimate cost forecasting uncertainties. That is the classification used in Module 5 (risk-seeking versus risk-averse) to help STAs decide between the MFE method and regression analysis techniques. In the case of intermediate-range forecasts, and in view of evident market corrections, Module 4 suggests the assessment of both MFE and regression analysis (linear and exponential), if possible. The final inflation rate selection would be made after reviewing the different MFE and regression analysis outputs. Long-Range Forecasting Method Selection (with selected cost index) Minimum CCI lookback period 15 to 20 Years Type of inflation rate Annual Compounded Selection of forecasting approach Expected Forecasting Errors (%) In-House MCCI Confidence level 50 70 90 Asphalt paving ±10 ±15 ±25 Concrete paving ±10 ±15 ±25 In-House or External CCI (traditional CCI) Confidence level 50 70 90 Asphalt paving −40 to 10 −45 to 15 −50 to 20 Concrete paving −15 to 55 −20 to 60 −25 to 75 Table 2-6. Module 5: Long-range forecasting method selection.