A Guidebook for Airport Winter Operations (2015)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

100 The unpredictability of winter events and their potential to impact airport operations will at times challenge even the best winter operations plans. As a result, recurring evaluation of the operational procedures, strategies, and tactics outlined within the plans will facilitate adapt- ing to changing conditions and finding opportunities for improvement. Chapter 6 identified the need for measuring performance of airport winter operations. Performance evaluations are important means to engage SICC representatives and other stakeholders. They can facilitate a common understanding of current problem areas, performance limitations, opportunities for improvement, and planned operational changes among all stakeholders. In short, regular perfor- mance evaluations enable effective expectations management. This chapter presents a structured and systematic winter operations performance evaluation process that relies upon documented performance measurement data. It will enable factually supported decision making on winter operations strategies, tactics, and procedures: those that are meeting established performance targets and those that are falling short. 12.1 Conduct Performance Evaluations Integrate performance evaluations as part of regular SICC meetings (e.g., after each event, weekly, etc.). The evaluation process is similar whether it follows a significant winter event or occurs at the conclusion of a winter season. However, post-season evaluations offer the addi- tional opportunity to evaluate performance trends and the effectiveness of operational changes implemented prior to and during the season. Performance evaluations can include a qualitative and quantitative component. Both components can be merged into a single effort, but the need for supporting data may limit the value of a quantitative evaluation if relevant event data are not compiled and available. Similarly, quantitative evaluations conducted early in the season to identify seasonal performance trends may not offer the desired insight. These limitations should be considered when scheduling performance evaluation meetings. 12.1.1 Qualitative Evaluation A qualitative evaluation of winter operations performance should cover the preceding winter event(s) that occurred since the last SICC meeting. During a post-event evaluation conducted with the SICC, elicit the committee’s opinions and perspectives on the recent performance. This qualitative assessment will enable a quick overview of the winter event and should identify if any airport departments or stakeholders were negatively impacted and, if so, in which operational areas. Example questions that can be asked as part of a qualitative evaluation include: • What worked well? • What did not work? C H A P T E R 1 2 Winter Operations Performance Evaluation

Winter Operations Performance Evaluation 101 • Did the event exceed target winter-event threshold conditions? • Where can we improve? • Who needs to be involved? Conduct a follow-on, data-supported, quantitative evaluation as part of an end-of-season review, or, if an event raises perfor- mance concerns. 12.1.2 Quantitative Evaluation A quantitative performance evaluation differs from a qualita- tive evaluation in that documented performance measurements are reviewed and compared to performance targets to validate specific performance shortfalls, quantify their magnitude, and fully understand their impact. A performance shortfall is the dif- ference between a performance target and an actual performance measurement (provided the actual value did not exceed the target value), as illustrated in Figure 12-1. If no concerns were raised during a qualitative evaluation, there may be little need for the full SICC to review event performance in greater detail. However, the airport should still review available data for trends that indicate the potential for future performance shortfalls. Example ques- tions that can be asked as part of a quantitative evaluation include: • Were measures and APIs sufficiently documented? • How do APIs compare to established performance targets? • How do APIs compare to data from the current and past seasons? • Were there any problematic performance areas despite existing performance targets being met? • Are performance targets still appropriate? • Were there performance shortfalls that require further investigation? 12.2 Examine Significant Performance Shortfalls Upon completing a post-event or end-of-season performance evaluation, potentially signifi- cant performance shortfalls may warrant further investigation. To understand when an investi- gation is necessary, establish significance criteria giving consideration to the actual or potential impact of the shortfall (e.g., near miss, incident, airport closure, etc.); similar prior shortfalls, if any; likelihood of recurrence; and the effort and resources required to conduct an investiga- tion. Consider conducting an investigation when there is a recurring winter operations perfor- mance shortfall, or when a significant procedural, mechanical, or human factor-related failure occurs. Example incidents that may justify investigation include an aircraft go-around caused by a missed runway reopening time, an equipment runway incursion, or a snowbank with the potential to damage a taxiing aircraft. Investigating insignificant issues, while well-intentioned, may tie up staff resources, impede progress on understanding incidents with far greater potential BEST PRACTICE—Post-Season Evaluation to Kick Off Pre-Season Planning A post-season meeting of the SICC with airport stakeholders is likely the most important meeting of the winter season. Many airports consider the post-season meeting as the start of the planning process for the following winter season. The last post-event meeting and/or critique should not be confused with the post-season review meeting. It is important to have access to minutes from all post- event meetings from the snow season for the post- season review. Procedural changes are often dic- tated by lessons learned during individual snow or ice events. A comprehensive post-season review of procedures becomes the basis for revisions to the SICP and updates of related procedural documents. Changes are incorporated into the appropriate documents with publication scheduled for late summer or early fall, as appropriate, to support annual, recurrent winter operations training. Figure 12-1. Illustration of a performance shortfall.

102 A Guidebook for Airport Winter Operations to impact the winter operations, and potentially drive investigation teams to cut short the inves- tigation process. The overarching reason to investigate performance shortfalls is to seek out the root cause(s) and implement a permanent fix or a series of permanent fixes (rather than short-term, quick fixes that only address symptoms and not the root cause) for shortfalls where the risk and con- sequences of recurrence are unacceptable. For this to occur, personnel assigned to investigate a performance shortfall must fully understand the nature of the concern prior to initiating the investigation. Failure to fully understand the shortfall may result in an ineffective response and continued recurrence. 12.2.1 Performance Shortfall Investigation A performance shortfall investigation is the process of identifying, collecting, and assembling those facts and circumstances that contributed to the shortfall. This step must occur before a root cause(s) is determined. To investigate a specific incident, designate an investigation team leader with the authority to make assignments and having the backing of airport management. The team should consist of representatives of each stakeholder group directly affected by or involved with the problem. The team should investigate organizational areas and functions, as summa- rized in Table 12-1. For investigations of complex issues, a written summary of the investigation and relevant facts should be prepared. Considerations for conducting an investigation include the following: • Align the time and effort to be spent on the investigation with the severity or consequences of the shortfall. • Perform the investigation in collaboration with staff who are subject matter experts. This will prevent an investigator’s lack of expertise from limiting the investigation or leading to incom- plete or faulty conclusions. • Encourage differing perspectives to avoid “group think,” preconceived outcomes, and unex- plored contributing factors. • Gather facts rather than opinions, judgments, or interpretations of facts. • Seek to identify acts of omission or commission by personnel, equipment, or processes that led to the shortfall. Areas Functions Management responsibility Organization or person who determines the course of action for a process, who owns the process, and who is accountable for the quality of the process. Procedures Documented or prescribed methods of accomplishing processes. Controls Checks or restraints that are designed into a process to ensure that a desired result is achieved. Process management Measures or information assessments to identify, analyze, and document potential problems with a process. Interfaces Interactions between independent processes. Source: DOT/FAA/Aviation Regulation-03/70 – Continuing Analysis and Surveillance System Description and Models (19, p. 9). Table 12-1. Organizational areas and functions for consideration in a performance shortfall investigation.

Winter Operations Performance Evaluation 103 • Seek out potential hidden or latent contributing factors rather than focusing on just the most apparent factor(s). • Ask the following straightforward questions to begin the collection of facts: – What happened? – When did it happen? – What was the sequence of events? – What was affected? – What should have or not have happened? – What conditions led to the problem? – What actions could have prevented the event? – Were there unusual circumstances? – What records are available? – What policies and procedures were applicable? • Seek relevant data sources including operating logs, observations, photos, staff interviews, and other records. • For safety issues or issues that developed over an extended period of time, classify facts along a timeline leading up to the occurrence and identification of the shortfall. 12.2.2 Root Cause Determination The process for determining a root cause begins after the facts about the circumstances that led to a performance shortfall have been assembled. A root cause can be defined as “the most basic cause (or causes) that can reasonably be identified that management has control to fix and, when fixed, will prevent (or significantly reduce the likelihood of) the recurrence of an issue” (20). There are numerous methods for determining root cause, each of varying complex- ity and effectiveness. Select a method that best meets organizational needs and limitations. It is important that the process lead to reasonably identifiable causes of an incident that are specific, controllable, and correctable so that effective corrective measures can be identified and put into place (21). It is also important to emphasize that for most incidents the root cause is likely a series of causal events collectively contributing to the incident, not a singular cause. Considerations for determining root causes include the following: • With the investigation facts compiled, ask the following questions to begin identifying poten- tial root causes: – Why did this happen? – What are the causes? – How do the causes relate to the shortfall? – How were these causes identified? – What data or evidence points to these causes? • Avoid determining an event as a root cause (e.g., “the friction tester got stuck in a snow drift” is an event, not a root cause). • Avoid confusing symptoms with root causes (e.g., “failure to follow the procedure” is a symp- tom of something else, not a root cause). • Avoid assigning individual blame or identifying human error as a root cause. Root cause will likely involve systems or process failures, in addition to human factors. • Consider individual factors when evaluating human factors, such as: – Lack of knowledge, – Lack of skill, and – Existing distractions. • Consider influencing factors when evaluating human factors, such as: – Consecutive hours worked, – Time constraints,

104 A Guidebook for Airport Winter Operations – Change in procedures, – Management of change, – Lack of training materials, and – Competing priorities. • Separate root causes from those secondary causal factors which, alone, would not have caused the shortfall. • Focus on the fewest number of and most impactful root causes that, if corrected, will prevent or significantly reduce the likelihood of shortfall recurrence. • Define each root cause in a clear and concise problem statement, without extraneous information. • Review facts uncovered during the investigation that did not directly contribute to the short- fall but are still relevant. They may indicate a previously unknown problem or potential prob- lem requiring a separate corrective action. 12.2.3 Corrective Action Identification Corrective actions are the reactive steps necessary to quickly mitigate an identified shortfall. They may not, how- ever, prevent it from recurring. Corrective action consider- ations include the following: • Define new or additional steps that can correct the shortfall. • Ask the following questions to evaluate proposed correc- tive actions: – Will the corrective action resolve the shortfall? – What staff and resources are required to implement the corrective action? – How quickly can the corrective action be implemented? • If the shortfall investigation reveals that it is not an isolated incident, or is systemic (i.e., department-wide), identify corrective actions that will correct the shortfall(s) for all affected areas. • Define each corrective action in a clear, concise, and assignable statement of action(s). 12.2.4 Preventive Action Identification Preventive actions are proactive steps intended to prevent a shortfall from recurring. While some corrective actions can also be preventive actions, preventive actions typically differ in that they focus on addressing root causes. Preventive action considerations include the following: • Determine if a preventive action is required for the shortfall. A preventive action may not be required for every shortfall. • Emphasize preventive actions for recurring shortfalls. • Associate each preventive action with a specific root cause. • Ask the following questions to evaluate proposed preventive actions: – How does the preventive action relate to the root cause? – If this preventive action had been in place, would the shortfall have occurred? – Does the preventive action address identified human factors? – What staff and resources are required to implement the preventive action? – How quickly can the preventive action be implemented? BEST PRACTICE—Non-punitive Reporting Procedures Accident and incident investigations are enhanced by instituting non-punitive reporting procedures. Future accidents are often prevented by forthright and honest reporting of circumstances leading up to an incident. Multiple airport operators discouraged immediate termination of an employee for a run- way incursion or for a vehicle accident, except in the case of willful misconduct or prior history. Airports noted the significant investment in operator train- ing and the loss of a valuable resource associated with unnecessarily rigid disciplinary action.

Winter Operations Performance Evaluation 105 • If the shortfall investigation reveals that the shortfall is not an isolated incident, or is systemic (i.e., department-wide), identify preventive actions that will prevent the root cause(s) for all affected areas. • Define each preventive action in a clear and concise statement of action(s). 12.3 Identify Performance Improvement Opportunities Performance evaluations, significant performance shortfall investigations, and observations from airport personnel or SICC members may reveal a broad range of opportunities to improve a winter operation program. An improvement opportunity is simply another way to look at a performance shortfall, as illustrated in Figure 12-2. This is where the true value of performance management becomes evident. The collaboration with stakeholders that led to the creation of performance goals, objectives, and associated performance targets will facilitate consistency of performance expectations. If those expectations are not met due to a performance shortfall, the opportunity for improvement will be evident and should be supported by the airport and its stakeholders. A lack of viable improvement alternatives may require revision of the performance target to a preferred and achievable level. When resource constraints prevent the complete elimi- nation of a shortfall, seek improvement through phased implementation of a series of actions as resources allow. Interim performance targets should be set to align with the expected outcome of each action (see Chapter 7). This clarity offered through performance management enables optimal investment in winter operations. Investment can occur at the strategic, tactical, or pro- cedural level, as described below. 12.3.1 Strategy Opportunities Opportunities to improve winter operations strategies can represent larger scale shifts in the execution of a winter operations program. Therefore, it is anticipated that this level of change will occur infrequently. Example strategy changes may include planning the incorporation of a performance measurement system, the incorporation of circuit routes for SRE, and the transi- tion to dedicated runway and taxiway snow teams. The need for strategy change may occur when the associated goal is changed or a new goal is established (see Chapter 6). A strategy change may also be necessary when the performance limitations of an existing strategy are reached and all feasible opportunities to further performance improvement are exhausted. Strategy changes will, inevitably, result in new or cascading changes to tactics and supporting procedures. 12.3.2 Tactic Opportunities The most frequent opportunities for winter operations program improvement are expected to occur at the tactical and procedural level. During a winter event, winter operations person- nel must be equipped with an array of tactics to employ as conditions dictate. Because tactics represent the “tools” in an airport’s winter operations toolbox, more and better tools will better enable effective implementation of specific strategies, achievement associated objectives, and attainment of overall goals. There are numerous potential opportunities to alter or improve Figure 12-2. Illustration of an improvement opportunity.

106 A Guidebook for Airport Winter Operations tactics. The reader is directed back to Chapters 8 through 11 for further consideration of alterna- tive winter operations practices in use within the industry. 12.3.3 Procedure Opportunities As the strategies and tactics that make up a winter operations program change or evolve, maintain written procedures that clearly describe how to effectively execute them. Procedures facilitate consistency in the execution of an activity under a fixed set of conditions so that the outputs and outcomes remain predictable. Procedure revisions may also be required when the root cause of a performance shortfall was determined to be an unanticipated set of circum- stances not considered or addressed in the procedure. Depending upon the activities covered by a procedure, experienced personnel assigned responsibility for implementing revised and new procedures should review and provide input on the new content. These personnel represent the subject matter experts. All personnel assigned responsibility for implementing revised and new procedures must be made aware of and be trained on the final revisions and new procedures. Sig- nificant changes in operations should be rolled out as part of pre-season planning to maximize opportunities to get personnel familiar and comfortable with new requirements. If personnel are provided access to written procedures in hardcopy format, careful consideration should be given to implementing a document control system so that outdated versions of procedures can be replaced and are not inadvertently used.