Improving Mid-Term, Intermediate, and Long-Range Cost Forecasting: Guidebook for State Transportation Agencies (2020)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

37 Index-Based Cost Forecasting Approaches 4.1 Introduction This chapter provides guidance on quantitative cost forecasting methodologies that can be used to generate effective inflation rates from the suitable cost indexes identified by the protocol explained in Chapter 3. Chapter 2 has already explained the implications associated with the methodologies addressed in this chapter as well as the circumstances under which they would be more appropriate. This chapter mainly covers some technical aspects of those methodologies. 4.2 Linear and Exponential Regression Linear and exponential regression analyses are common mathematical methods for assessing simple and compound inflation trends, respectively. Linear regression forecasting models assume a linear relationship between the length of forecasting periods and recorded construc- tion prices, usually represented by index values. Statistical software packages for conducting regression analysis usually produce linear regression models defined as shown in Equation 4-1. This equation does not immediately provide a simple annual inflation rate, but one can be easily calculated, as shown in Equation 4-2. Eq. 4-1 simple annual inflation rate Eq. 4-2 y a bx b a = + = where y = forecast index value, x = intended forecasting time horizon, a = slope of linear function, and b = current index value. Similarly, Equation 4-3 shows a typical exponential regression output yielded by statistical software packages. As occurs with linear regression analysis, the outputs of statistical software packages do not typically provide annual compound inflation rates, but Equation 4-4 shows how to calculate them from an exponential equation. Eq. 4-3 fixed annual compound inflation rate 1 Eq. 4-4 y ae e bx b = = − C H A P T E R 4

38 Improving Mid-Term, Intermediate, and Long-Range Cost Forecasting: Guidebook for State Transportation Agencies where y = forecast index value, x = intended forecasting time horizon, a and b = constants, and e = exponential constant. Figures 4-1 and 4-2 show the simple and compound annual inflation rates obtained from a linear and an exponential regression model, respectively, for the scope-based asphalt paving construction cost index (CCI) developed for the Minnesota DOT in Section 3.6.4. A visual comparison of these models and their R2 values suggests that an exponential equation would be more suitable for modeling the Minnesota DOT’s long-term market trends. The same conclu- sion was obtained from all case studies for both asphalt and concrete paving activities. 4.3 Moving Forecasting Error The moving forecasting error (MFE) methodology is a novel cost forecasting approach proposed by NCHRP 10-101 (Rueda-Benavides et al. 2020). It proved to be effective in gener- ating scope-based inflation rates and in quantifying cost forecasting uncertainty in the form of 80 130 180 230 280 330 6/29/1999 3/25/2002 12/19/2004 9/15/2007 6/11/2010 3/7/2013 12/2/2015 8/28/2018 In de x Va lu e Date Simple Annual Inflation Rate = 10.21% R2 = 0.863 Figure 4-1. Minnesota DOT asphalt paving CCI—linear regression model. 80 130 180 230 280 330 6/29/1999 3/25/2002 12/19/2004 9/15/2007 6/11/2010 3/7/2013 12/2/2015 8/28/2018 In de x Va lu e Date Compounded Annual Inflation Rate = 5.4% R2 = 0.877 Figure 4-2. Minnesota DOT asphalt paving CCI—exponential regression model.

Index-Based Cost Forecasting Approaches 39 risk-based forecasting outputs. This is an iterative process designed to maximize the value of the available data. For instance, a state transportation agency (STA) with only 20 years of historical bid data with which to produce long-range forecasts would be forced to rely on this single 20-year data set to predict the market behavior during the next 20 years. It seems to be common knowledge that not all 20-year periods would show the same market trends. Thus, in an ideal world, the agency would have access to multiple 20-year periods to consider multiple possible scenarios. Unfor- tunately, that is not the case for most STAs. The MFE method recognizes that there are several 3-, 5-, 10-, and 15-year periods within the available data and takes advantage of those smaller data partitions to better infer long-range market conditions. The proposed MFE methodology is applied to any cost index through the following six-step process: 4.3.1 Moving Forecasting Error: Step 1 Apply the protocol for the comparative suitability analysis of cost indexing alternatives explained in Chapter 3 to identify the most suitable cost index for the region under consid- eration. That index would be assumed to effectively represent its respective local construction market. It should be noted that if a better market representative were found for that region, that would become the most suitable index. 4.3.2 Moving Forecasting Error: Step 2 Use a 4% annual compound inflation rate to project the first index value into the future. This step forecasts an index value rather than a cost estimate in dollars. For each forecasting period in the available data, in 6-month increments, calculate and record the percentage difference between the forecast index value and the actual index value. Each of those differences is a fore- casting error measure. At the end of this process, a 20-year data set would produce 39 forecasting error measures for different forecasting periods (i.e., 0.5; 1; 1.5; 2; . . . 18.5; 19; 19.5 years). Since the first known index value is calculated after the first 6 months of data (at 0.5 years), a 20-year data set would not allow the calculation of a forecasting error for a 20-year forecast. The longest possible forecast would be over 19.5 years. On the basis of Equation 1-3 (Section 1.5), the forecasting error at each forecasting period would be calculated as follows (Equation 4-5): FE 1 1 100% Eq. 4-5 0 0 I I i I i t t t t ( ) ( ) = − × + × + × where FEt = forecasting error over t years, I0 = first known index value, It = known index value at time t, and i = compound inflation rate. 4.3.3 Moving Forecasting Error: Step 3 Repeat Step 2, but this time forecasting the second known index value (at 1 year). This second iteration would generate 38 forecasting errors with a 20-year data set (one less than at Step 1), with a maximum forecasting time horizon of 19 years. Continue repeating this process in 6-month intervals until forecasting the second-to-last known index value, always calculating and recording forecasting errors for the different forecasting periods.

40 Improving Mid-Term, Intermediate, and Long-Range Cost Forecasting: Guidebook for State Transportation Agencies 4.3.4 Moving Forecasting Error: Step 4 At the end of Step 3, the STA would have several forecasting error measures for different forecasting periods. With a 20-year data set, the agency would have 39, 38, 37, . . . 3, 2, and 1 forecasting error for forecasts of 0.5, 1, 1.5, . . . 18.5, 19, and 19.5 years, respectively. The number of recorded forecasting errors decreases as the forecasting time horizon increases. With a single measured error for a 19.5-year forecast, it is not possible to make reliable conclusions about potential market scenarios associated with long-range forecasts. Instead of relying on this single long-range forecasting error, the proposed MFE method takes advantage of the more reliable assessments conducted for shorter forecasting periods, identifies trends, and extrapolates those trends to the long-range forecasting zone. This is done by first calculating an average forecasting error for each forecasting time horizon (e.g., the average of the 39 recorded forecasting errors for 0.5 years), and then plotting all average forecasting errors as shown in Figure 4-3. This figure illustrates the average forecasting errors obtained when the MFE method was applied on a scope-based asphalt paving CCI developed for the Colorado DOT. Each point in this figure corresponds to the average forecasting error calculated for each forecasting period from 0.5 to 19.5 years. The negative sign in the average forecasting errors means that actual market values tended to be lower than those obtained with the inflation rate under consideration. Figure 4-3 shows how the more reliable average errors for shorter forecasting periods define a strong trend that can be projected to long-range periods. Similar outputs were obtained from the application of the MFE method in 11 case study regions. In most cases, with a 20-year data set, points tend to start falling off the trend after about 15 years, when average values start to be calculated with fewer than 10 observed forecasting errors. On the basis of that observation, the proposed MFE method ignores all values calculated with fewer than 10 observations and uses regression analysis to project the remaining values into the future to estimate expected errors for long-range forecasts. That is how the linear projection in Figure 4-3 was created. 4.3.5 Moving Forecasting Error: Step 5 The MFE method not only facilitated a better assessment of average forecasting errors but also allowed the projection of percentiles around average values to establish error ranges at 50%, 70%, and 90% confidence levels. Figure 4-4 shows the same linear trend of average Forecasting Time Horizon (Years) Av er ag e Fo re ca st in g Er ro r 4% A nn ua l I nf la tio n R at e Figure 4-3. Example of MFE output: Average forecasting errors for Colorado DOT’s asphalt paving activities with a 4% compound annual inflation rate.

Index-Based Cost Forecasting Approaches 41 errors from Figure 4-3, but this time with its respective confidence intervals. On the basis of this figure, the Colorado DOT could reasonably assume, with a 90% confidence level, that any 15-year asphalt paving cost forecast estimated in this region with a 4% compound inflation rate would offer a forecasting error between +12% and −27%. Confidence levels are defined by assuming that forecasting errors at each forecasting period follow a normal distribution. Thus, the confidence bands in Figure 4-4 are calculated from 50%, 70%, and 90% confidence intervals from those normal distributions at each forecasting time horizon. For example, the upper 90% limit in Figure 4-4 is the result of a regression model developed with the upper 90% confidence intervals of all forecasting time periods from 0.5 to 15 years. Average errors calculated with less than 10 observations are also discarded. 4.3.6 Moving Forecasting Error: Step 6 MFE cost forecasting errors, like those shown in Figure 4-4, can now be used to create a risk-based forecasting timeline (see Section 1.6) of forecasting factors. Forecasting factors are unitless values that form a generic risk-based forecasting timeline that can be used to estimate the intended inflation rate. Those generic outputs can also be translated into dollars to easily obtain a risk-based forecasting timeline for any current-dollar estimate. Figure 4-5 shows the risk-based forecasting timeline for the forecasting factors estimated from the forecasting errors in Figure 4-4. Figure 4-5 also shows the 4% compounded projection, which was actually used as a reference to develop the risk-based output. Table 4-1 details the process to calculate the forecasting factors for the first 5 years in Figure 4-5 by applying the forecasting errors in Figure 4-4 to the 4% annually compounded projection of a forecasting factor of 1. The risk-based forecasting timeline in Figure 4-5 is the result of plotting and connecting the data points from Rows C and J to O in Table 4-1. The unitless values in this figure can be easily transformed into dollar values to generate a risk-based forecasting output just by multiplying all forecasting factors by the given current-dollar estimate. For example, if the current-dollar estimate for a given asphalt paving program in Colorado is $10 million, the multiplication of this value by each of the forecasting factors in Figure 4-4 would generate the risk-based forecasting timeline shown in Figure 4-5. This figure is actually a scaled version of Figure 4-4. With Figure 4-6, the Colorado DOT could conclude that the expected average value for this program in 15 years would be around $15.8 million. With a 70% confidence level, the Colorado DOT could expect this value to be between $14 and $18 million, approximately. Forecasting Time Horizon (Years) Av er ag e Fo re ca st in g Er ro r 4% A nn ua l I nf la tio n R at e Figure 4-4. Example of MFE output: Average forecasting errors with confidence intervals for Colorado DOT’s asphalt paving activities with a 4% compound annual inflation rate.

42 Improving Mid-Term, Intermediate, and Long-Range Cost Forecasting: Guidebook for State Transportation Agencies Figure 4-5. Example of MFE output: Risk-based forecasting timeline for forecasting factors. Row Year (A) 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 A Forecast of FF =1 4% CAIR (B = 1.04^A) 1.02 1.04 1.06 1.08 1.1 1.12 1.15 1.17 1.19 1.22 B Average FE 0% –1% –1% –2% –2% –3% –3% –4% –4% –4% C Average FF(B * [1 + C]) 1.02 1.03 1.05 1.06 1.08 1.09 1.11 1.13 1.14 1.16 D 50% CL FE 25th PCTL –9% –9% –10% –10% –10% –11% –11% –11% –12% –12% E 75th PCTL 11% 10% 10% 9% 8% 8% 7% 7% 6% 6% F 70% CL FE 15th PCTL –13% –13% –14% –14% –14% –15% –15% –15% –16% –16% G 85th PCTL 17% 17% 16% 15% 15% 14% 14% 13% 12% 12% H 90% CL FE 5th PCTL –19% –19% –20% –20% –20% –21% –21% –21% –21% –22% I 95th PCTL 31% 30% 29% 28% 28% 27% 26% 25% 25% 24% J 50% CL FF 25th PCTL (B * [1 + D]) 0.93 0.94 0.96 0.97 0.99 1 1.02 1.04 1.05 1.07 K 75th PCTL(B * [1 + E]) 1.13 1.14 1.16 1.18 1.2 1.21 1.23 1.25 1.27 1.29 L 70% CL FF 15th PCTL (B * [1 + F]) 0.89 0.9 0.92 0.93 0.95 0.96 0.97 0.99 1.01 1.02 M 85th PCTL(B * [1 + G]) 1.2 1.21 1.23 1.25 1.27 1.29 1.3 1.32 1.34 1.36 N 90% CL FF 5th PCTL (B * [1 + H]) 0.83 0.84 0.85 0.87 0.88 0.89 0.91 0.92 0.94 0.95 O 95th PCTL(B * [1 + I]) 1.33 1.35 1.37 1.39 1.41 1.43 1.45 1.47 1.49 1.51 Note: FF = forecasting factor; CAIR = compound annual inflation rate; FE = forecasting error; CL = confidence level; PCTL = percentile. Table 4-1. Example of forecasting factors calculation.

Index-Based Cost Forecasting Approaches 43 Instead of directly producing risk-based forecasting timelines from the calculated forecasting factors, the Colorado DOT could also use Figure 4-4 to estimate an annual inflation rate for asphalt paving activities in the region under consideration. This inflation rate could be shared with other estimators across the region to facilitate cost forecasts without the need of sharing a spreadsheet with all forecasting factors. Assuming that the target inflation rate is intended to match the average trend in Figure 4-4, the Colorado DOT could perform a simple statistical analysis (probably by using Equation 4-4) to find that the average trend in Figure 4-4 would be matched by a 3.1% annual compound inflation rate, as shown in Figure 4-7. All case study results presented in this section to illustrate the use of the proposed MFE methodology were obtained by using an arbitrary 4% compound annual inflation rate as a $5,000,000 $10,000,000 $15,000,000 $20,000,000 $25,000,000 $30,000,000 $35,000,000 Fo re ca st ed C os t E sti m at e ($ ) Forecasting Time Horizon (Years) 90% Confidence Level70% Confidence Level50% Confidence LevelAverage Forecast 300 5 10 15 20 25 Figure 4-6. Example of MFE output: Risk-based forecasting timeline for $10 million program. 0.5 1 1.5 2 2.5 3 3.5 Fo re ca st ed C os t E sti m at e ($ ) Forecasting Time Horizon (Years) 90% Confidence Level 70% Confidence Level 50% Confidence Level 3.1% Compounded Projection 300 5 10 15 20 25 Figure 4-7. Example of MFE output: Risk-based forecasting timeline with applicable inflation rate.

44 Improving Mid-Term, Intermediate, and Long-Range Cost Forecasting: Guidebook for State Transportation Agencies point of reference. In a perfect world, any arbitrary inflation rate (even simple inflation rates) would yield the same results shown in Figures 4-4 and 4-5, but that is not the case. The use of regression analysis to approximate trends in market average error makes outputs from different inflation rates slightly different. The results from this study suggest that the use of reasonable inflation rates as a point of reference produces more accurate results, which is the reason that motivated the selection of the suggested rate (4%). However, future research efforts will further investigate this matter to determine whether a different input would offer better forecasting effectiveness. Even though the calculation processes presented in this section—or in this guidebook— may look complicated or highly technical, these are easy quantitative tasks when performed with the assistance of data processing technologies and tools available today, such as the Cost Forecasting Tool Kit that accompanies and is described in the following chapter.