Airport Safety Risk Management Panel Activities and Outcomes (2016)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

3 Beginning in the mid-1990s, safety began to be viewed from a systemic perspective involving organizational, human, and technical factors. Since then, the aviation industry has taken steps to implement a systematic approach to manage safety, and both the FAA and International Civil Aviation Organization (ICAO) have established requirements and plans to implement Safety Management Systems (SMSs) for air traffic services, airlines, airports, and aviation service providers. The FAA, including its Office of Airports (ARP), is already implementing its SMS and has established a process to screen project plans or changes proposed by airports listed in the National Plan of Integrated Airport Systems. This screening process evaluates the need to convene a panel and conduct a safety assessment (SA). According to the FAA Office of Airports Safety Management (SMS)—Desk Reference (2012), if a panel is deemed necessary, the airport will be notified by the FAA to coordinate, acquire a facilitator, and arrange a Safety Risk Management (SRM) panel for the SA. Although airport operators are becoming more familiar with key aspects of the SRM processes and some Part 139 airports are conducting internal airport-led Safety Risk Assessments (SRAs), important questions remain. These questions are related particularly to the facilitation and outcomes of SRAs; the differences between safety assessments led by the FAA (SAs) and those led by the air- port (SRAs); decision making for risk mitigation actions; and the processes to implement and ensure that risk mitigation actions are in place. This report complements ACRP Report 131: A Guidebook for Safety Risk Management for Airports (Neubauer et al. 2015) and consolidates information gathered from Part 139 airports to identify exist- ing challenges and lessons learned from past experience conducting SRAs. This information could be shared to help Part 139 airports that are performing SRAs. Processes used by other industries and foreign airports were reviewed to provide additional insight on the SRA process, which may become routine if the FAA finalizes an SMS rule for Part 139 airports. SRAs can demand a significant workload and involve some challenges; however, it is important to note that the majority of safety issues faced on a routine basis by Part 139 airports do not require con- vening panels and can be performed by the SMS coordinator and other personnel. It is important that SRAs use SRM panels only for the most complex issues, possibly those that affect multiple airport stakeholders. Routine airport safety issues can be handled on a regular basis by airport staff familiar with SMS procedures. Although SRAs at Part 139 airports are associated predominantly with safety issues on the airside, the methodology represents a powerful tool for risk management that can be applied to any safety issue on the airside or landside. Also, because SRM practices at Part 139 are still in their early stages, the majority of the information available and collected for this report was originated at the larger air- ports and represents experience involving safety issues of higher complexity. Nevertheless, smaller airports can benefit from the information presented in this report, which can be adapted to fit less complex safety issues or those that involve fewer stakeholders. This report presents the tasks employed and the material obtained on SRAs, the results of a survey with Part 139 airports, foreign and military airports, and consultants, a summary of current chapter one INTRODUCTION

4 practices worldwide, tools used, and a final chapter with conclusions and suggestions for future research. SAFETY ASSESSMENTS VERSUS SAFETY RISK ASSESSMENTS It is important to distinguish between the terms Safety Assessment (SA) and Safety Risk Assessment (SRA). Because there is no current regulatory requirement for SRM or standardized industry terminol- ogy, both terms have been used in the United States to describe the safety risk management processes using SRM panels. Both have the same objective: to identify hazards, assess risks, and define actions to mitigate risks associated with a proposed change. However, there are some important differences that need to be clarified and understood, particularly because the focus of this report is the current experience with SRAs led by the airports. The term Safety Assessment was defined by the FAA ARP in Order 5200.11, which introduced ARP’s Safety Management System and refers to its SRM internal review and related documentation. If an action or proposed change (e.g., airfield construction, new advisory circular) requires ARP approval, the assessment of the change is led by the FAA and may require convening an SRM panel. If a panel is required by the FAA, an FAA employee or third party consultant will serve as facilitator and the process is managed by the FAA. Therefore, for the purpose of this report, the term Safety Assessment and the acronym SA is used only for risk assessments led by FAA. During the FAA pilot studies, particularly during the SMS Implementation Study, airport staff and consultants used the term Safety Risk Assessment because this was the term used by the FAA when announcing the request for airports to participate in the study. The SRA acronym has been widely used by Part 139 airports to refer to airport’s internal risk assessments and those assessments that are not led by the FAA. For the purposes of this report, the term SRA is used for safety assessments that are not led by the FAA. This report is intended to address SRAs only; that is, those assessments initiated internally by an airport and led by airport staff or a consultant hired by the airport. Therefore, the term SRA instead of SA is used throughout this document. Table 1 shows the key differences between SRAs and SAs. The FAA’s Notice of Proposed Rulemaking (NPRM) proposed a broad five-step SRM process but did not prescribe requirements detailing how to conduct or document those steps. However, some airports have opted to define the process in their SMS manual. It is in an airport’s best interest to have the capability to conduct internal SRAs, particularly those airports that will routinely use SRAs if FAA finalizes an SMS rule. Item Airport-Led FAA-Led SRA SA Airport role Plan, facilitate, report, SME SME, obtain facilitator (when requested by the FAA) FAA role SME (when necessary) Plan, facilitate, report, SME Requirement • Airport SMS manual • FAA Order 8000.369A • FAA Order 8040.4A • FAA Order 5200.11 SRM triggers • Airport SMS manual • Airport management • FAA ARP SMS Desk Reference • Guidance • FAA AC 150/5200-37A (draft) • ACRP Report 131 • ICAO SMM • FAA ARP SMS Desk Reference • FAA ARP SOP 4.00 • FAA ATO SMS Manual Source: ASM Consultants. TABLE 1 SAFETY RISK ASSESSMENT (SRA) AND SAFETY ASSESSMENT (SA)

5 BACKGROUND The 2011 FAA SMS Implementation Study evaluated how airports could implement the SRM and safety assurance elements of SMS. Fourteen airports participated in the program, and many of those airports conducted at least three SRAs to meet the goals set by the FAA. Some of the airports that participated in the SMS Implementation Study continue to convene SRM panels voluntarily and conduct airport-led SRAs for internal purposes. Separately, airports participated in FAA-led SRM panels convened to meet the requirements of FAA Orders 5200.11, JO 1000.37A, Air Traffic Organization Safety Management System, and the ATO Safety Management System Manual. Order 5200.11 describes conditions in which FAA ARP conducts SAs and convenes an SRM panel. Order JO 1000.37A instructs ATO on when to conduct SRM processes, and the ATO SMS Manual provides detailed guidance on how to conduct an SRM and when to convene an SRM panel. Although ARP-led panels have been primarily limited to project approvals at large and medium hub airports, ATO-led panels have been used for many other safety issues at a variety of airports. Foreign airports and military organizations have routinely used SRAs in the past and have clearly defined their specific process, the players and their roles, as well as myriad specific tools and tem- plates for use when conducting risk assessments. This report summarizes some of the guidance avail- able to support future efforts by Part 139 airports when conducting SRAs. In the past few years, several airports have convened panels to conduct risk assessments, in most cases related to capital improvement plans (CIP) on the airside. With the new requirements for SAs, airports have hired consultants and sometimes counted on FAA support to facilitate those risk assess- ments. A few airports have trained staff to facilitate and report SRAs using SRM panels; however, there are many questions about the process used by Part 139 airports to conduct SRAs. Some of those questions relate to the accuracy and effectiveness of the methodology, and there are concerns about how SRA results and actions will be implemented and questions on how airport staff can gain the necessary expertise to make the process sustainable for use by Part 139 airports. OBJECTIVE The objective of this Synthesis study is to present existing practices and tools used by airports and other industries to conduct risk assessments. The intent of this report is to complement existing guidance references, particularly ACRP Report 131: A Guidebook for Safety Risk Management for Airports, published in 2015. The intended audience is Part 139 airport operators who plan to convene SRM panels to conduct SRAs. Although the primary data source of this Synthesis is a small number of airports that have con- ducted risk assessments with airport-led and FAA-led SRM panels, airport operators will benefit from better understanding the airport’s roles and responsibilities in non-airport-led SRM panels. In addition, airport operators can benefit from understanding the experience of other industries that conduct SRM processes. Through the literature review, survey, and interviews with airport-led SRM panel coordinators, existing guidance, most effective practices, and tools are consolidated to support airport operators administering airport-led SRAs supported by SRM panels. This Synthesis study also combines the overall experience and lessons gained during recent years to answer questions about SRAs that have been raised during and after the FAA SMS Implementation Study. This report can be used by airport operators who plan to undertake SRAs in the future. DEFINITIONS Accident—An unplanned event or series of events that results in death, injury, damage to, or loss of equipment or property. Assessment—Process of measuring or judging the value or level of something.

6 Consequence—See Outcome. Control—See Risk control. Credible—Refers to a specific system state and sequence of events supported by data and expert opinion that clearly describes the outcome. It implies that it is reasonable to expect the assumed combination of extreme conditions will occur within the operational lifetime of the system. Hazard—A condition that could foreseeably cause or contribute to an accident. Incident—An occurrence, other than an accident, associated with the operation of an aircraft, that affects or could affect the safety of operations. Likelihood—The estimated probability or frequency, in quantitative or qualitative terms, of a hazard’s effect or outcome. Operational Risk Management (ORM)—A decision-making tool used by personnel at all lev- els to increase effectiveness by identifying, assessing, and managing risks. By reducing the potential for loss, the probability of a successful mission increases. Recently, the term has been changed to Risk Management (RM). Outcome—A specific system state and sequence of events supported by data and expert opinion that clearly describes the outcome. The term implies that it is reasonable to expect that the assumed combination of conditions may occur within the operational lifetime of the system. Preliminary Hazard Analysis (PHA)—An overview of the hazards associated with an opera- tion or project proposal consisting of an initial risk assessment and development of safety-related requirements. Preliminary Hazard List (PHL)—A list of anything that can go wrong, based on the concept, its operation, and implementation. Risk—See Safety risk. Risk analysis—Process during which a hazard is characterized for its likelihood and the severity of its effect or harm. Risk analysis can be either quantitative or qualitative; however, the inability to quantify or the lack of historical data on a particular hazard does not preclude the need for analysis. Risk assessment—Process by which the results of risk analysis are used to make decisions. The process combines the effects of risk elements discovered in risk analysis and compares them against acceptability criteria. A risk assessment can include consolidating risks into risk sets that can be jointly mitigated, combined, and then used in making decisions. Risk control—Reduction of risk severity and/or likelihood, through the application of engineer- ing and/or administrative hazard controls. Risk control can also be anything that mitigates or ame- liorates the risk. Risk matrix—Table depicting the various levels of severity and likelihood as they relate to the levels of risk (e.g., low, medium, or high). Risk mitigation—Any action taken to reduce the risk of a hazard’s effect. Risk register—A basic, ongoing working document that captures and describes risks and oppor- tunities as they are identified together with risk accountabilities, actions where required, and review and completion dates. Root cause analysis—Analysis of deficiencies to determine their underlying root cause.

7 Safety—The state in which the risk of harm to persons or property damage is acceptable. Safety Assessment (SA)—Completion of the applicable SAS, the SRM five-step process of identifying and analyzing hazards, and documentation of the SRM panel’s findings, as applicable. Safety Assessment Screening (SAS)—An FAA form (5200-8, 5200-9, or 5200-10) used to docu- ment the ARP Safety Assessment process. Specifically, the SAS form is used to document the appro- priate level of assessment, the five steps of SRM, and the final signatures and approvals. Safety assurance—The process and procedures of management functions that evaluate the con- tinued effectiveness of implemented risk mitigation strategies, support the identification of new hazards, and function to systematically provide confidence that an organization meets or exceeds its safety objectives through continuous improvement. Safety Management System (SMS)—Formal, top-down, organization-wide approach to manag- ing safety risk and ensuring the effectiveness of safety risk controls. An SMS includes systematic procedures, practices, and policies for managing safety risk. Safety risk—Composite of predicted severity and likelihood of the potential effect of a hazard. – Initial—Predicted severity and likelihood of a hazard’s effects or outcomes when it is first iden- tified and assessed; includes the effects of preexisting risk controls in the current environment. – Residual—Remaining predicted severity and likelihood that exists after all selected risk control techniques have been implemented. Safety Risk Assessment (SRA)—Assessment of a system or component, often by a panel of system subject matter experts (SMEs) and stakeholders, to compare an achieved risk level with the tolerable risk level. The term SRA is used for safety assessments with non FAA-led panels. Safety Risk Management (SRM)—A formal process within an SMS composed of describing the system, identifying the hazards, and analyzing, assessing, and mitigating the risk. Safety Risk Management (SRM) Panel—A group created to formalize a proactive approach to system safety and a methodology that ensures hazards are identified and unacceptable risk is miti- gated before the change is made. An SRM panel provides a framework to ensure that, once a change is made, the change will be tracked throughout its lifecycle. Severity—The measure of how severe the results of a hazardous condition’s outcome are pre- dicted to be. System—An integrated set of constituent pieces combined in an operational or support environ- ment to meet a defined objective. Elements include people, hardware, software, firmware, informa- tion, procedures, facilities, services, and other support facets. Triggers—The requirements, precursors, or organizational plans that lead to initiation of the SRA or SA process. ACRONYMS 5M Mission, Man, Machine, Management, and Media AOA Aircraft operations area ARFF Airport rescue and firefighting ARP FAA Office of Airports ATO FAA Air Traffic Organization AVS FAA Aviation Safety Organization CSA Comparative Safety Assessment CSPP Construction Safety and Phasing Plan FHA Functional Hazard Analysis

8 ICAO International Civil Aviation Organization LOB Line of business OPS SWR operations ORA Operational Risk Assessment ORM Operational Risk Management OSA Operational Safety Assessment PHA Preliminary Hazard Analysis RCA Root Cause Analysis SA Safety Assessment (FAA-led) SAS Safety Assessment Screening SME Subject Matter Expert SMS Safety Management System SOP Standard operating procedure SPI Safety Performance Indicator SRA Safety Risk Assessment (airport-led) SRM Safety Risk Management SWR Somewhere Airport TWR Air traffic tower REPORT CONTENT This report contains seven additional chapters. • Chapter two summarizes the results of the literature review; the survey with Part 139, foreign, and military airports and consultants; and interviews with airport staff involved with SRAs. • Chapter three presents the most common hazard identification and risk assessment techniques that are commonly used by the airport industry in the U.S. and elsewhere. This chapter also provides answers to some common questions raised during the survey and interviews. • Chapter four summarizes the SRA planning process in four phases and identifies the key deliver- ables for each phase. • Chapter five presents practical guidance for hazard identification and risk assessment, and emphasizes the improvements that can be achieved when structuring the hazard identification process. • Chapter six complements the guidance available in ACRP Report 131 on facilitation of SRM panels. It highlights key roles, principles, and techniques that can be used by airport staff to achieve successful and effective SRAs. • Chapter seven describes tools and templates to support SRAs. Some of these materials are avail- able in the appendices. • Chapter eight presents key conclusions and suggestions for future research.