Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
77 C H A P T E R 8 This chapter provides recommendations on how to evaluate and manage the uncertainty inherent in all DDFS forecasts. It is intended for both users and preparers. There is an element of uncertainty associated with all forecasts, and DDFS forecasts are no exception. This chapter describes the factors that generate forecast uncertainty, methods of evaluating the uncertainty, and methods of managing the uncertainty. 8.1 Sources of Uncertainty Exhibit 8.1 diagrams many, but not all, of the factors that generate uncertainty in DDFS forecasts. In most instances, DDFS forecasts are directly related to annual forecasts, and any uncertainties associated with the annual forecasts are carried over to the DDFS forecasts. Sources of uncertainty in annual and DDFS forecasts can be categorized in several broad categories: ⢠Forecast inputs such as projections of economic growth or fuel and other costs ⢠Forecast assumptions on industry changes, such as airfares, type of air service, and competitive factors ⢠Airport operator policy or infrastructure decisions ⢠Airline business and marketing decisions ⢠Forecast modeling factors ⢠Disruptive events, such as the September 11, 2001, terrorist attacks Most airport forecasters rely on secondary sources for projections of future economic growth or fuel prices, either from private economic firms or local and national government sources. Uncertainty is associated with these economic forecasts, even in the short-term, as forecasting organizations tend to have more difficulty predicting the peaks and valleys of business cycles than forecasting long-term growth. Aviation industry forecasting assumptions are another source of uncertainty. Airline costs are mostly a function of fuel, labor, aircraft maintenance, and aircraft acquisition costs, but how much of these costs are passed on to passengers in the form of airfares depends on the degree of competition among the airlines, or lack thereof. Airport operator and FAA policy and infrastructure decisions will affect an airportâs ability to accommodate demand. Many of the political, environmental, and financial resolutions required to advance large capacity projects at airports are inherently difficult to predict. Accurate forecasts of airline factorsâsuch as the degree of hubbing, fleet plans, and other air service elementsâare largely dependent on correctly assessing future airline behavior. How to Address Risk and Uncertainty in DDFSs
78 Guidebook for Preparing and Using Airport Design Day Flight Schedules Uncertainty will remain even if the DDFS preparer obtains full airline input and cooperation, as airline business plans contain their own uncertainty and the useful life of the business plan tends to be much shorter than the 20-year planning horizon of a typical master plan and is often affected by the strategies of competing airlines. Many annual forecasts are based on one or more quantitative forecasting equations, which convert inputs such as income and fare levels into passenger forecasts. The coefficients associ- ated with these equations are normally estimated using regression analysis, which is a statistical method of finding the best fit between forecast drivers (e.g., income, fares) and forecast output (e.g., passengers). These equations are subject to uncertainty because the coefficients are esti- mated using a small sample size. Also, a key variable may be omitted from the equation or an inappropriate variable may be incorrectly included. Historical measures of both economic data and airport activity data may be inaccurate. If so, these inaccuracies will be carried forward into any forecasts from which they were developed. Competion Aviaon Industry Factors Annual Forecasts Incomplete/ Inaccurate Data Forecast Model Parameters Economy Airline Travel Costs Fuel Labor DDFS Disrupve Events Airport/FAA Policy and Infrastructure Decisions Qualitave Data or Assumption Quantave Data Input Modeling Factors Output Legend Airline Business Decisions Exhibit 8.1. Sources of uncertainty.
How to Address Risk and Uncertainty in DDFSs 79 Even when accurate, relevant data are not always available on a timely basis, and the outdated information can result in inaccurate forecasts. Finally, unforeseen events, such as the September 11, 2001, attacks or the Severe Acute Respi- ratory Syndrome outbreak, or at a more local level, airline exit from or entry into a market, are major sources of uncertainty. ACRP Report 76: Addressing Uncertainty about Future Airport Activity Levels in Airport Deci- sion Making http://onlinepubs.trb.org/onlinepubs/acrp/acrp_rpt_076.pdf provides comprehen- sive guidance on ways to incorporate risk and uncertainty into annual activity forecasts. 8.2 Evaluation of Uncertainty Statistical methods for quantifying uncertainty are described in detail in ACRP Report 76 http://onlinepubs.trb.org/onlinepubs/acrp/acrp_rpt_076.pdf. In general, they provide a method for estimating how an actual future value of a forecast metricâe.g., passengersâis likely to deviate from the predicted value. By measuring historical variations in activity from the long-term average, the most likely distributions of activity around the long-term average can be estimated and applied to forecast values. These distributions are often described as confidence intervals. A confidence interval represents the probability that an actual activity level will fall within a specified forecast range. For example, if 85 peak hour operations are forecast, and the 90 percent confidence interval encompasses plus or minus five operations, it means that there is a 90 percent chance that actual peak hour operations will be between 80 and 90 operations. Another way of describing the confidence interval is that there is at least a 95 percent chance that peak hour operations will number 80 or more (5 percent chance that peak hour operations will be less than 80), and a 5 percent chance that peak hour operations will number more than 90 (95 percent chance that they will be less than 90). More detail on confidence intervals is pre- sented in ACRP Report 76: http://onlinepubs.trb.org/onlinepubs/acrp/acrp_rpt_076.pdf and in Appendix C of this guidebook. A critical question when evaluating uncertainty is how much of the uncertainty is directly attributable to the DDFS and how much is attributable to the annual forecasts upon which the DDFS is based. The analysis in Appendix C suggests that the majority of the uncertainty associated with DDFS forecasts results from the annual forecasts and, if the confidence intervals associated with the annual forecasts are accurately estimated, those same confidence intervals can be applied to the DDFS. 8.3 Management of Uncertainty This section describes three methods of incorporating uncertainty into DDFS forecasts to better manage the planning results. DDFSs are currently used mostly to assess requirements and effects under baseline forecast conditions and few attempts have been made to assess the effects of forecasting risk and uncertainty on DDFSs and resultant planning recommendations. The primary reason for this is the level of effort involved in either preparing alternative DDFSs or manipulating existing DDFSs to represent the potential range of outcomes. Three methods of assessing DDFS uncertainty are listed below. Ad Hoc Adjustments: Some DDFS elements, such as aircraft arrival and departure times, can be randomly adjusted to test the sensitivity of planning outcomes. This approach can be applied to airfield simulation models, gate requirement models, or terminal facility requirements that are dependent on peak passenger flows. The random adjustments should be tied to a probability distribution based on historical data if confidence levels are to be associated with the results. A confidence interval repre- sents the prob- ability that an actual activity level will fall within a speci- fied forecast range.
80 Guidebook for Preparing and Using Airport Design Day Flight Schedules Forecast Scenarios: Forecast scenarios can range from simple high and low scenarios to more complex scenarios involving potential airline changes, including air service, peaking/ distribution characteristics, and bankruptcies and mergers. Preparing a DDFS for each forecast scenario can generate a wealth of detail. However, preparing alternative DDFSs to match each forecast scenario can also be cost prohibitive. Incorporating Uncertainty to Aggregate DDFS Results: Much airport planning is based on aggregate DDFS results, including measures of airfield capacity, gate requirements, and peak passenger flows. When planning is based on aggregate DDFS results, establishing confidence intervals becomes a simpler process. Appendix B provides some default confidence intervals for various DDFS elements for large, medium, small, and non-hub airports. Note that these confidence intervals can only be applied to evaluate DDFS uncertainty, and do not include uncertainty associated with the annual forecasts, which can be much greater. Table 8.1 provides an example of how confidence intervals can be applied to curbside require- ments. In this medium-hub airport example, the calculated required curbside length was assumed to be 1,500 feet. As the curbside length requirement is based on the 60-minute peak, the confidence interval is based on the variation in numbers of peak hour O&D passengers for medium-hub airports. The table indicates that there is a 90 percent (95% minus 5%) degree of confidence that the requirement will be more than 1,369 feet and less than 1,631 feet, once uncertainty regarding the future peak hour percentage of O&D passengers is taken into account. Monte Carlo Analysis ACRP Report 76: Addressing Uncertainty about Future Airport Activity Levels in Airport Decision Making http://onlinepubs.trb.org/onlinepubs/acrp/acrp_rpt_076.pdf describes Monte Carlo analysis in detail and how to use Monte Carlo simulations to generate probability distributions for annual forecast outputs. In that report, probability distributions are identified for forecast Source: Appendix D Calculated Requirement (length in feet) 98% 95% 90% 75% 50% 25% 10% 5% 2% Large Hubs 92% 93% 95% 97% 100% 103% 105% 107% 108% Medium Hubs 90% 91% 93% 96% 100% 104% 107% 109% 110% Small Hubs 85% 87% 90% 95% 100% 105% 110% 113% 115% Non-Hubs 77% 81% 85% 92% 100% 108% 115% 119% 123% Curbside Requirement at each Confidence Level Large Hubs 3,000 2,754 2,795 2,841 2,917 3,000 3,083 3,159 3,205 3,246 Medium Hubs 1,500 1,343 1,369 1,398 1,447 1,500 1,553 1,602 1,631 1,657 Small Hubs 1,000 850 875 903 949 1,000 1,051 1,097 1,125 1,150 Non-Hubs 500 384 403 425 461 500 539 575 597 616 Note: Based on peak 60 minute O&D passengers. Variation in Curbside Length Requirements by Confidence Interval 90 percent confidence interval Table 8.1. Example of confidence intervals for curbside requirements by airport size.
How to Address Risk and Uncertainty in DDFSs 81 input factors (income, airfares, etc.) and forecast parameters (income and fare elasticities). Every Monte Carlo iteration involves generating a forecast with the inputs and parameters randomly calculated based on the probability distributions identified for each input and parameter. The process is repeated multiple times to generate a distribution of forecast outcomes that can be aggregated to provide a probability distribution for the activity forecast that incorporates all of the probability distributions associated with the inputs and parameters. Monte Carlo analysis can be applied to DDFS outputs in two ways, independently or in combination with annual forecasts. Table 8.2 provides an example of the inputs and param- eters that would be involved in each type of Monte Carlo analysis. Note that the input box under the independent DDFS analysis case is empty. It is empty because DDFSs are derived from annual forecasts and, in this particular example, the variability of annual forecasts is not addressed. As noted earlier, Appendix D provides confidence intervals for DDFS critical factors. Once developed, these confidence intervals can be used to run Monte Carlo models of DDFS metrics (average delay, peak period passengers, etc.) without requiring a new DDFS for each Monte Carlo simulation. This method does not offer the same degree of precision that would result from multiple DDFS and simulation runs, but it is more cost-effective and practical. Appendix C provides examples of independent or combined Monte Carlo simulations. It is not realistic to ignore the variability associated with annual forecasts in most real world planning. Monte Carlo analysis can be applied more comprehensively in a combined analysis by assembling the probability distributions associated with annual forecast inputs and parameters, and then adding probability distributions associated with peak period percentages and load fac- tors. The combined annual and peak period probability distributions can be used to generate more comprehensive probability distributions for facility requirements that incorporate both annual and peak period variability. A Monte Carlo approach to DDFS uncertainty would be less resource-intensive than develop- ing separate DDFS scenarios. Nevertheless, the effort is not trivial. Time is required to quantify the linkages between the annual forecast inputs and parameters and the DDFS-related param- eters and each associated probability distribution. The analysis provided in Appendix C suggests that the majority of uncertainty is in the annual forecasts rather than the DDFS forecasts. Therefore, if resources are limited, it is recommended that confidence intervals be developed for the annual forecasts and then applied to the facility planning requirements resulting from the DDFS analysis. In some instances, airport forecasts and plans are tied to activity levels instead of specific forecast horizon years. This approach mitigates some but not all of the uncertainty associated with annual forecasts. For example, if total passengers are used to define a specific activity level, forecast elements such as aircraft operations and peak hour passengers may still be off. Conse- quently, there will still be some uncertainty associated with the DDFS and an independent DDFS Monte Carlo analysis may be warranted. Type of Monte Carlo Analysis Type of Inputs Type of Parameters 1. Independent DDFS Monte Carlo Analysis Peak hour percentage, peak load factor, O&D raÂo 2. Combined DDFS and Annual Forecasts Monte Carlo Analysis Income, employment, populaÂon, airfares Income elasÂcity, fare elasÂcity, peak hour percentage, peak load factor, peak O&D raÂo Table 8.2. Types of Monte Carlo analysis as applied to DDFSs.
82 Guidebook for Preparing and Using Airport Design Day Flight Schedules Risk Registers ACRP Report 76: Addressing Uncertainty about Future Airport Activity Levels in Airport Deci- sion Making http://onlinepubs.trb.org/onlinepubs/acrp/acrp_rpt_076.pdf also identifies risk registers as an effective way of identifying and quantifying risk and uncertainty in airport activ- ity forecasting. Risk registers are especially useful in addressing low-frequency, high-magnitude risks that are difficult to define using probability distributions. Risk register factors are grouped within two general categories. The first category, risk iden- tification, includes: ⢠Risk ID ⢠Risk name and brief description ⢠Risk status: active, dormant, or retired ⢠Risk category ⢠Date the risk was first identified The second category, risk evaluation, considers: ⢠Probability of occurrence ⢠Description of the impact ⢠Metrics affected, such as passengers or aircraft operations ⢠Magnitude of impact, which can defined as a single variable or a probability distribution ⢠Duration of impact ⢠Recovery Table 8.3 provides an example of a risk register focused on risks that are more likely to apply to DDFSs than to annual forecasts. The first example in the table relates to the potential of an increase in connecting banks at an airline hub. This type of change occurs at the discretion of the hubbing airline, and airport operators are generally given short notice. Nevertheless, an increase in the number of connecting banks can reduce peak hour activity and lower the requirements for most terminal facilities, including gates. However, an increase in the number of connecting banks may result in one or more banks occurring between 10 p.m. and 7 a.m., which is defined as nighttime. Therefore, this contingency would also present a risk related to an airportâs noise program. Another example is irregular operations, which are most often the result of adverse weather conditions and, therefore, provide relatively short notice to airport operators. Aircraft are unable to depart and the demand for gate and hardstand facilities increases as a result. Passengers are likewise unable to depart and, therefore, the demand for concessions, restrooms, and other airport facilities greatly increases. The probability or likelihood of a certain risk category occurring during the planning period is a matter of judgment. For example, over a 20-year planning period, it is almost certain that irregular operations will occur at some point. On the other hand, technological developments, such as supersonic aircraft, are considerably less likely to occur. One of the key features of a risk register is that it can be easily updated. Therefore, as information relevant to the probability of an occurrence becomes available, it can be readily incorporated into the risk register. Also, the risk register can be used to prepare contingency plans should any of the events in the listed risk category occur. Appendix A shows one of the ways in which risk and uncertainty can be incorporated into a DDFS. Please click to access Section A.12 Dealing with Uncertainty. Risk registers are especially useful for addressing low- frequency, high- magnitude risks that are difficult to define using probability distributions.
Risk Identification Risk Evaluation Magnitude of Impacts Risk ID Risk Category Status Threat or Opportunity Probability/ Likelihood Description of Impact Impact on Low Mid High Expected Duration Expected Recovery 1 Airline Strategy Increase in number of connecting banks 30% Increase in number of connecting banks resulting in passengers and operations spread more evenly throughout the day with reduced peaks and increased nighttime operations Aircraft Operations, Passengers X Medium to Long term Uncertain 2 Airline Strategy Decrease in aircraft turnaround time 40% Decrease in gate requirements; reduced ability to recover from disrupted operations Aircraft Operations, Passengers X Long term None 3 Technology Supersonic aircraft 5% Change in international flight times/windows, U.S. CBP requirements International Aircraft Operations, Passengers X Long term None 4 Airport Facilities Runway Reconstruction 50% Reduced capacity; change in throughput, reduced peak activity Aircraft Operations, Passengers X Medium term Full 5 Irregular Operations Disruption in Schedule 99% Delay in Operations Aircraft Operations, Passengers X Short-term Full CBP = U.S. Customs and Border Protection Table 8.3. Example DDFS risk register.
