Lessons Learned from Airport Safety Management Systems Pilot Studies (2012)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

13 BACKGROUND Once fully implemented, SMS is an operational system. How- ever, there is a significant amount of planning and activity required to bring SMS from inception to normal operations. Program management is the set of structures, tools, activities, and personnel that is employed by airports to realize a fully functional and operational SMS. The program plan documents all aspects required to imple- ment and operate a SMS. The plan incorporates overall scope, tasks, and dependencies and creates a basis for schedule, staff- ing, and budget. The program plan may be effectively used to set and manage stakeholder expectations. ICAO refers to the program plan as an implementation plan. The program schedule uses the information defined in the program plan to set specific and traceable deadlines, mile- stones, dependencies, and completion dates for SMS tasks. The budget defines cost estimates for SMS implementation and operations and the mechanisms by which those costs will be supported. Organizational structures, staffing, and program manage- ment are developed in conjunction with the program plan and can define team members and the roles and responsibili- ties, including the use of consultants, that will be necessary to implement and operate a SMS, including that of the SMS program manager, who is responsible for the coordination and completion of all SMS implementation activities and who may or may not be the airport’s assigned safety man- ager. In this context, the safety manager was regarded as specific to SMS. There may be multiple airport safety man- agers at airports who are not involved in SMS but, rather, supervise and manage other, non-airside safety activities such as Occupational Safety and Health Administration compliance. FAA SMS Pilot Studies 1 and 2 required participating air- ports to develop a program plan, Study 3 required airports to revise program plans accordingly, and Study 4 required airports to create a study plan. All such deliverables will be collectively referred to as program plans for the remain- der of the report. Other program management elements such as status reporting were required from participating SMS pilot study airports through a series of FAA-defined deliverables. CHAPTER CONTENTS This chapter presents interview respondent findings, analy- sis, and lessons learned relating to the SMS program man- agement, analysis, design, and development aspects of the airport’s efforts including: • Gap analysis process and report, • SMS manual development, • Program plan and schedule development, • Consultant services, • Reference documents, • Organizational structures and program management, and • Budget. FINDINGS AND ANALYSIS Gap Analysis Process and Report As described in chapter one, Study 1 was conducted between 2007 and 2008 (Study 1 included all class types of airports) and Study 2 spanned from 2008 to early 2010 (Study 2 was limited to Class II, III, and IV airports). For many airports that participated in Study 4, the initial documentation devel- oped during Study 1 or 2 had been shelved; however, the documentation was revisited, as required, during the imple- mentation study. To determine whether the initial studies were useful in implementing the SMS, a number of ques- tions were posed to airports regarding the gap analysis, SMS manual, and program plan deliverables developed in the first studies. A gap analysis, as stated in the ACRP Report 1: Safety Management Systems for Airports, Volume 2: Guidebook (hereafter referred to as the ACRP Guidebook), “is intended to identify the processes already existing in your airport, compare what you have against the requirements established by the pillars [components] and elements of the SMS model that you have chosen, and identify what should be done to make these two pictures match” (Ayers et al. 2009). Many of the airports employed the ICAO checklist located in Appendix 2 to Chapter 7 of the Safety Management Manual (SMM) to compare the four SMS components (safety policy, safety risk management, safety assurance, and safety promo- tion) to the current Part 139 airport operation. ICAO further explains: chapter two SAFETY MANAGEMENT SYSTEM PILOT STUDY PROGRAM MANAGEMENT

14 A gap analysis is basically an analysis of the safety arrange- ments already existing within the organization as compared to those necessary for the SMS to function. The gap analysis is important because the basic organizational structures nec- essary to start developing an SMS may already exist in the organization: it will seldom be necessary to build an SMS from scratch because most organizations will have vari- ous activities related to an SMS in place and functioning. A gap analysis thus reveals the resources, structures and safety arrangements existing in the system to address safety vulnerabilities, specified in terms of hazards that arise as a consequence of the interaction of people and other compo- nents of the operational context. It also reveals additional resources, structures and safety arrangements that would be necessary to mitigate safety vulnerabilities and increase operational resilience to the hazards (ICAO 2009). Survey responses for the survey question, “Was the gap analysis useful in developing your SMS program?” resulted in nearly all airports responding “yes” as demonstrated in Figure 2. When asked what elements of the gap analysis were the most useful, a variety of responses ranging from all elements to policy and safety awareness were listed. The most frequent response, reflected in Figure 3, was that the gap analysis allowed the airport to better identify specific needs necessary to implement SMS. Because more than one analysis tool or method was used by various consultants in the pilot studies, the function of the analysis, as explained by ICAO, is to assist airports in identifying and formally docu- menting findings to ultimately develop the SMS. Safety Management System Manual Development The SMS manual is considered the SMS program’s formal documentation and operational guide. The ACRP Guide- FIGURE 2 Usefulness of gap analysis. FIGURE 3 Usefulness of gap analysis elements.

15 book states, “SMS documentation is made up of the airport SMS manual and the specific procedures for the SMS pro- cesses. Overall, the SMS documentation should provide a description on how the SMS will be or has been set up, who is responsible for what, which processes and procedures are going to be used and when” (Ayers et al. 2009). Airports were queried to determine whether the SMS manual developed during the initial studies was used for implementation. The SMS manual was a required deliver- able for both Studies 1 and 2. As presented in Figure 4, when asked whether the gap analysis was useful in developing the SMS manual, a total of 19 airports indicated “yes” and seven airports responded “no.”As discussed earlier, seven of the airports participating in the Synthesis study survey are not implementing SMS, thus the “no” would appear to reflect airports within the nonimplementation group. However, one airport that is implementing reported “no,” the original SMS manual was not used as part of the current implemen- tation program because the initial manual did not reflect the airport’s current strategy to include additional safety topics such as Occupational Safety and Health Administration into the program. One airport that is not implementing reported “yes,” it is using the SRM portion of the SMS manual to support its construction management program. While this airport’s SMS program is not rolled out yet, the SRM risk matrix and processes are being used to formally document the risk assessment process for construction projects throughout the airport. As mentioned earlier in this chapter, the majority of air- ports (22 of the 31) participated in Pilot Study 1 in 2007; because the FAA’s SMS implementation study (Study 4) began midyear in 2010, for most airports more than 2 years had elapsed between the initial and subsequent studies. Some airports continued with their SMS implementation outside of the FAA implementation study, but for many airports the SMS implementation study required a renewed effort with differ- ent personnel, stakeholders, managers (airport staff), and, in some cases, new consultant teams. As part of the Synthesis survey, there was interest in determining whether the initial SMS manual and associated SMS implementation plans from Pilot Study 1 or Pilot Study 2 were used for the SMS imple- mentation study. Therefore, the surveyed airports were asked, “Did you revise any of the content and specifically what sec- tions were changed?” Additionally, if implementations had indicated that airports had not used the SMS manual, an addi- tional question asked for clarification or “why not?” Table 5 provides survey responses. Of the 26 airports surveyed, 19 indicated they had or are currently making significant changes to the SMS manual; all 19 were either participating in the FAA SMS implementation study or were implementing SMS outside of the FAA study. Comments in Table 5 further clarify why airports modified their SMS manuals, including timeline constraints in the initial stud- ies, use of the manual in day-to-day operations, aligning the manual to specific airport operations by either increasing or reducing the level of detail, and leveraging the availability of additional SMS resources and practices to refine content and processes, such as SRM revisions as a result of conducting safety risk assessments (SRAs). All seven airports that have not revised the SMS manual responded that they were not implementing SMS or that the document is still in draft stages. Specific comments are pre- sented in Table 6. Program Plan and Schedule Development In addition to a SMS manual, airports were required to develop a program plan including a program schedule and budget. For similar reasons as the airports’ use of the SMS manual, airports were asked whether or not they developed a program plan as part of Study 1 or 2 and if the developed program plan was used for SMS implementation. Program plans were developed using various mechanisms including Microsoft Project schedules and Gantt charts, Excel spread- sheets, and narrative instructions to budget and timelines. 0 2 4 6 8 10 12 14 Class I Class II Class IV Have you used the program manual [SMS Manual] developed during initial studies? Yes No FIGURE 4 Use of original SMS manual.

16 13. We have not changed anything yet, mostly because we have not started the formal SMS program and we are only using some of the most basic SMS elements for the current operations, such as SRM. 14. All aspects have been revised; we are almost constantly adding details and more specific language to the sections such as the policy, risk analysis, and hazard reporting, as we integrate them into the airport’s operations. 15. An internal manager is currently revising and going through the manual to make the SMS processes more useful, including laminated checklists for the airport staff to use. The manual is being consolidated into useful and specific information for management and the staff. 16. We are completely revamping the document; we are starting from scratch. No. Responses /Comments to: “If yes, did you revise any of the content and specifically what sections were changed?” 1. We have revised the hazard reporting and SRM process, safety policy, and safety assurance components of the manual. There was very little guidance or resources in the initial phase but we did some things that were spot on and others needed reevaluation. The revisions reflect increased knowledge on the part of the airport team and the consultants. 2. At this time we are awaiting final revisions from our consultants. We have made a number of updates but we are not sure specifically what sections will be changed with this round of revisions—likely all of them. 3. As part of the third pilot, the SRM hierarchy of controls and 5-step risk processes were completely revised. In addition the safety assurance scoring methodology was revamped. Also using the Air Traffic Organization (ATO) SMS Guide information we focused on the risk and assurance sections of the manual and revised them accordingly. 4. The risk matrix and severity/likelihood definitions were adjusted, the safety policy was changed; currently the entire manual is under review and will be revised by the end of the FAA Implementation Study. 5. The entire manual was revised except the introduction to the appendix that included the Part 139 gap analysis information. 6. The entire manual is being modified; each section is being changed as we work through the SMS program implementation. New sections and standard operating procedures are being added. The initial manual had the basic structure but it needed to be expanded and have details added for operations. 7. All sections changed. Originally the manual written for the airport was fairly basic. We used everything that was in the manual but went into more detail on programs and expanded it to include the landside. 8. Mainly Section 1 Policy and Organizational changes were made; we are trying to find a different way to manage document control; for this particular section, the initial manual description was too complex. Version control needs to be simplified for the program to be feasible and realistic. 9. Manual revisions come about as part of the second SMS pilot study review and editing process, more so than the SMS Implementation Study. The SRM section was changed as a result of conducting SRAs. 10. When the initial consultant drafted the SMS manual we did not have many resources and the project timeline was short so we used some of the FAA ATO information as a baseline. Recently we have been revising the manual and rewriting sections. Overall we have reduced the contents of the manual to make it more manageable and useful. 11. Yes, we have made changes and amendments; sections that came up that did not work the best for the organization were revised. Multiple collateral duties were assigned, such as quarterly reporting. The effort was too burdensome for the safety manager, so instead we compile reports every 6 months, which are distributed to the airport director, as a more realistic effort. 12. Many of revisions have been made to the policy, forms, processes, etc. The SRM section was updated including forms but the overall process did not change, just the forms to make the process more fluid. We are still changing the audit and evaluation section but we cannot conduct an audit until the program is fully implemented. We have practiced using the forms and made them more congruent. TABLE 5 COMMENTS FOR SMS MANUAL CHANGES

17 ICAO states, Once the gap analysis is complete and fully documented, the resources, structures and arrangements that have been iden- tified as missing or deficient will form, together with those already existing, the basis of the SMS Implementation Plan. Organizations may format their SMS Implementation Plan to suit their individual needs; however, a spreadsheet format, Gantt chart or MS Project type layout is recommended for ease of viewing and tracking . . . (ICAO 2009). Figure 5 illustrates that a total of 20 respondents answered “yes” to the question of whether they had developed an implementation plan (program plan) and six replied “no.” As represented in Figure 5, 20 airports answered “yes” to having developed an implementation plan (program plan) during Study 1 or 2; however, only 10 (Figure 6) airports confirmed the plan was being used as part of the implementa- tion program. When asked if a new program plan was devel- oped as part of Study 4 or implementation outside of the FAA studies, three airports replied “yes” (Figure 7). The remain- ing six airports participating in SMS implementation activi- ties reported that no program plan was used. To assess individual SMS program deployment progress, airport representatives were asked to report on SMS pro- gram components being implemented. As displayed in Fig- ure 8, 11 of the airports surveyed responded that they are implementing all SMS components and eight others within the implementation group indicated that the respondent air- port is implementing one or more of the components, all of which include SRM. Study 4 was limited to implementation of safety risk management and safety assurance components; therefore, half of the implementations are rolling out policy and promotion components to some degree without federal financial assistance. Table 7 provides a summary of SMS components by air- port class; of note is the higher proportion of Class I air- ports implementing “all” components and Class II and IV airports deploying limited portions of the SMS program with a higher occurrence of SRM than other components such as safety policy, safety assurance, and safety promotion. Airport respondents reported that they were actively developing a SRM program and conducting SRAs as part of SMS Study 4 required deliverables and construction projects, as well as participating in Air Traffic Organization-facilitated SRAs. TABLE 6 COMMENTS FROM AIRPORTS NOT REVISING SMS MANUAL Responses/Comments to: If no, why have you not used it (Manual)? 1. The document is in draft format and is under discussion. 2. We have not started the SMS program at this time. 3. Funding/regulatory concerns have prohibited starting the program. 4. Staffing issues are limiting the ability to start the program. 5. No further analysis has been conducted; we have no plans to implement at this time. FIGURE 5 Program plan development. 0 2 4 6 8 10 12 14 16 Class I Class II Class IV Did you develop an implementation plan [program plan] as part of your gap analysis? Yes No FIGURE 6 Program plan used. 0 1 2 3 4 5 6 7 8 Yes Not Used Pending Approval If so, have you used it (Plan) for implementation? Class I Class II Class IV

18 FIGURE 8 SMS components being implemented. 0 1 2 3 4 5 6 7 8 Used Initial Plan Used New Plan Used No Plan If so, have you used the new plan for implementation? Class I Class II Class IV FIGURE 7 New program plan used. TABLE 7 COMPONENTS IMPLEMENTED BY AIRPORT CLASS Components Implemented Class I Class II Class IV Total by Type All 9 2 11 Policy/Risk/Promotion 1 1 Policy/Risk/Assurance 1 1 Risk/Assurance 1 1 2 Risk/Promotion 1 1 2 Policy/Risk 1 1 Risk 1 1 Count by Class 13 3 3 With regard to timelines and the response to the question, “What is your planned schedule to complete implementa- tion?” 16 airports provided various implementation schedules covering the next 5 years. As presented in Table 8, seven air- ports are within a year of completing the implementation of their SMS components (based on the interview date), six air- ports plan to complete their program in 1 to 2 years, three air- ports have 3- to 5-year schedules, and seven stated that they are waiting for the FAA Rule or funding to become available before the SMS program will be rolled out. Of the total airports interviewed, only three did not know the planned implemen- tation schedule or timeline. When responses were reviewed from nonimplementation airports, it was observed that the majority (five of the seven) responded “awaiting regulatory decision.” Also, the three airports replying “don’t know” are currently implementation airports but are unable to precisely determine the program schedule. Many airports commented that based on the possible SMS Final Rule, airport manage- ment is considering adding timeline contingencies to address possible program modifications or adjustments. Consultant Services Airport representatives were queried whether consultants were hired for any of the SMS pilot studies. All, with the exception of one airport surveyed, employed consultants to assist with one or more of the pilot studies. The airport repre- sentative who replied “no” developed the program internally. When asked for reasons why airports engaged consultants, a variety of responses were recorded and are presented in Figure 9. More than half the responses (14) indicated subject matter expertise as the primary reason. The second and third most frequent responses were “due to staff constraints” and “funding requirement of FAA grant.” It was noted that Pilot Studies 1, 2, and 3 did not allow for sponsor “force account” projects, but the Study 4 Participants Guide, Part 139 imple- mentation study, stated “FAA recognizes that some airports may wish to complete all or portions of the [SMS pilot] study using existing staff or infrastructure. In those cases, normal force account approval will apply” (FAA 2010).

19 Multiple airports (15) participated in more than one SMS pilot study (see Table 2). There was interest in whether air- ports sought different skill sets from consultants based on the type of study. Airports that had participated in more than one SMS pilot study were asked if they hired the same consultant for both studies. The local airport knowledge and exposure to the first SMS pilot study analysis offered reduced ramp- up time for consultants participating in the second round of SMS pilot study efforts. Results are presented in Table 9. The majority of airports (eight) hired the same consultant, yet four hired a different consultant. Reasons for hiring new consultants ranged from use of the airport’s competitive bid process, existing master contracts with consultant teams, and the knowledge and abilities of consultants who conducted the first studies to compete on competitive bids. Regarding specific skills expected from consultants, air- ports responded to the question, “Please describe the types of experience you believe a consultant should have to effec- tively help an airport develop an SMS” with the top five skills identified as follows: 1. Part 139 airport experience, 2. SMS program development, 3. SMS program implementation, 4. SMS expertise, and 5. Specific airport knowledge. A list of consultant skills and counts of times mentioned (by airport class) is presented in Table 10. Reference Documents As airports and consultants worked together to develop SMS programs, a variety of industry resources, documents, and references was compiled. A list of resources reported by airports was compiled and is presented in the literature and data search section of chapter one. Additionally, airports TABLE 8 PLANNED SCHEDULE TO COMPLETE IMPLEMENTATION Anticipated Timeline Class I Class II Class IV Total by Type Underway/Implementation Nearly Complete 1 1 2 <1 year 3 1 1 5 1 year 1 1 2 1.5 years 1 1 1 to 2 years 2 1 3 3 years 2 2 4 to 5 years 1 1 Awaiting Regulatory Decision 4 2 6 Pending Funding Decision 1 1 Do Not Know 3 3 Count by Class 18 3 5 FIGURE 9 Why consultants were hired.

20 were questioned as to what document they believed to be the most effective in developing their SMS. The results do not clearly point to a single most useful document, but as presented in Table 11, the FAA AC 150/5200-37, Introduc- tion to Safety Management Systems for Airport Operators, and the ICAO Document 9859, Safety Management Manual, were most frequently mentioned. Many airports commented that because they believed the future SMS rule would likely be based on the AC, they reviewed the document thoroughly in addition to other resources to ensure the program would ultimately comply with the FAA’s existing guidance. Organizational Structures and Program Management A number of questions regarding organizational and project management structures, including staff positions and titles and levels of participation in the SMS studies, were posed to airports as part of the interview process. Most airports participating in SMS Implementation Studies have desig- nated both an accountable executive and a safety manager. In some instances, the airport’s SMS pilot study program manager was a different staff member than the safety man- ager; therefore, a question, “What position (title) within the organization is assigned as the program or project manager for the SMS implementation project?” was asked of the air- ports. Airports not participating in the SMS implementation study were asked to provide the position that was assigned to program management for Pilot Studies 1, 2, and 3. Results by airport size are presented in Table 12. A broad range of positions and departments were represented including opera- tions, maintenance, environmental, safety, risk, and security. As part of any of the four SMS pilot studies, airports were asked what departments had participated or been involved in the SMS activities. Table 13 presents all responses from the 26 airports surveyed. The top seven departments involved in the SMS, in order of number of responses, are (1) opera- tions, (2) maintenance, (3) fire/aircraft rescue and fire fight- ing, (4) engineering/construction/facilities, (5) police/sheriff, (6) risk—note that an airport’s risk function is not always a sep- arate department or duty, and 7) tenants/fixed-base operators. Consultant Hiring Practices Class I Class II Class IV Total by Type Yes, Hired Same Consultant 5 1 2 8 No, Hired Different Consultant 4 4 Only Used Consultant on First 8 2 2 12 Only Used Consultant on Second 1 1 Did Not Use Consultant 1 1 Count by Class 18 3 5 TABLE 9 CONSULTANT HIRING Pa rt 1 39 A ir po rt O pe ra tio ns SM S Pr og ra m D ev el op m en t SM S Pr og ra m I m pl em en ta tio n S M S E xp er ti se Sp ec if ic A ir po rt K no w le dg e A vi at io n E xp er tis e in G en er al P ro gr am M an ag em en t Sa fe ty E xp er tis e R is k M an ag em en t F A A R el at io ns hi ps E du ca tio na l/A na ly tic al B ac kg ro un d P ra ct ic al H an ds -o n E xp er ie nc e C om m . a nd I nf o. D ev el op m en t Sk ill s T ec hn ic al W ri tin g C ri ti ca l T hi nk in g In du st ry A w ar en es s R E : R ul em ak in g T ec hn ol og y E xp er ti se Fa ci lit at io n Sk ill s (f or S R A s) Class I 18 Airports Surveyed 10 10 8 2 2 4 5 2 4 2 2 2 2 1 1 1 1 1 Class II 3 Airports Surveyed 3 2 2 3 3 1 Class IV 5 Airports Surveyed 4 2 4 4 1 2 Total Count 17 12 12 9 9 6 5 4 4 2 2 2 2 1 1 1 1 1 TABLE 10 CONSULTANT SKILL SETS

21 TABLE 11 MOST USEFUL REFERENCES FA A A C 1 50 /5 20 0- 37 IC A O S M M D oc um en t 9 85 9 V ar io us I nd us tr y A C R P G ui de B oo ks V ol um e 1 an d 2 A ir po rt ’s P ar t 1 39 A C M T ra ns po rt C an ad a D on ’t K no w , o r N on e S pe ci fie d IA T A S af et y P ro gr am A ir li ne S M S A dv is or y C ir cu la rs C on su lt an t’ s D oc um en ts FA A A ir po rt S M S N PR M F A A A T O S M S M an ua l O SH A O ve ra ll an d V PP P ro gr am B an ga lo re S M S E xa m pl e FA A O rd er 5 20 0. 11 A rm y/ N av y Sa fe ty P ar t 1 39 D ec on st ru ct ed f or S M S G ap C iv il A vi at io n Sa fe ty A ut ho ri ty (A us tr al ia ) IS O 1 40 01 Class I 18 Airports Surveyed 11 11 9 7 4 4 1 2 3 1 2 2 2 2 2 2 1 1 1 Class II 3 Airports Surveyed 2 1 1 1 1 1 Class IV 5 Airports Surveyed 3 2 2 2 1 1 1 1 Total Count 16 14 12 9 6 5 3 3 3 2 2 2 2 2 2 2 1 1 1 VPP = Voluntary Protection Program. TABLE 12 SMS PROJECT PROGRAM MANAGER TITLES No. Program Manager Title Class 1. Airport Duty Manager Class II 2. Airport Manager Class I 3. Airport Manager (2) Class II 4. Airport Manager Class IV 5. Airport Operations Supervisor Class I 6. Assistant Director of Airfield Operations Class I 7. Aviation Director Class IV 8. Outside Consultant Class IV 9. Director of Operations and Security and Environmental Compliance Class I 10. Director of Safety, Training, and Security Class I 11. Environmental Manager Class I 12. Fixed-Base Operator (FBO) and Airport Operations Manager Class IV 13. Fire Chief Class I 14. Manager Airport Operations Class I 15. Manager of Operations and Maintenance Class I 16. Operations Manager (2) Class I 17. Operations Safety Administrator Class I 18. Operations/Maintenance Supervisor and Safety Coordinator Class I 19. Principal Investigator Class IV 20. Risk Management Administrator Class I 21. Risk Manager Class I 22. Senior Airport Operations Manager Class I 23. Superintendent of Airport Operations, Maintenance, and Security Class I

22 Within a SMS, the role of the accountable executive is to be accountable for all safety policies, practices, and processes. Today, some airports have functional roles that reflect top management accountabilities but may not use the term “accountable executive.” Because the concept of an accountable executive is new, the FAA clarifies in the NPRM that [t]his proposal [NPRM] would require an airport to identify an Accountable Executive. The FAA understands that air- port operations and organizational structures vary widely. Accordingly, the FAA would not prescribe a particular job title. Nevertheless, the Accountable Executive must be a high-level manager who can influence safety-related deci- sions and has authority to approve operational decisions and changes because an effective SMS requires high level man- agement involvement in safety decision making. Accord- ingly, the FAA proposes the international standard definition for an Accountable Executive (i.e., requiring the Account- able Executive to be an individual with ultimate responsibil- ity and accountability, full control of the human and financial resources required to maintain the SMS, and final authority over operations and safety issues). The FAA . . . believes an acceptable accountable executive would be the highest approving authority at the airport for operational decisions and changes (FAA 2010). The ACRP Guidebook defines the accountable executive as follows, “An Accountable Executive should be identified as ultimately responsible for the safety of personnel, busi- ness processes, and activities of the airport organization. Therefore, this should be the person at the top of the orga- nization. This person should demonstrate a commitment to safety by allocating the resources necessary to achieve organizational safety objectives” (Ayers et al. 2009). As an important reflection on management authority and oversight of the SMS, airport representatives were asked to identify the position and title of the designated or proposed accountable executive as presented in Table 14. Although numerous titles were provided, the general finding is that all levels reported represent upper management positions. The top three titles reported and counted include airport manager, chief executive officer, and president. O pe ra tio ns M ai nt en an ce F ir e/ A R F F E ng ./C on st ru ct io n/ Fa ci lit ie s Po lic e, S he ri ff R is k M an ag em en t ( ri sk f un ct io n is no t a lw ay s a se pa ra te de pa rt m en t o r du ty ) T en an ts /F B O P la nn in g T ra in in g A dm in is tr at iv e/ M an ag em en t/ B oa rd s/ C ity /C ou nt y Sa fe ty ( O SH A )/ Se cu ri ty Pr op er tie s/ Fi na nc e C us to m er S er vi ce /P R /H R L eg al E nv ir on m en ta l/W ild lif e In fo rm at io n T ec hn ol og y FA A E m er ge nc y M an ag em en t Class I 18 14 14 13 11 12 8 10 10 6 6 6 3 3 2 4 1 1 Class II 3 3 1 1 2 2 1 2 Class IV 5 5 5 2 2 2 3 1 1 3 3 Count 26 22 16 15 14 13 12 11 11 9 6 3 3 2 1 1 1 ARFF = Aircraft Rescue and Fire Fighting; FBO = fixed-base operator; OSHA = Occupational Safety and Health Administration; PR = Public Relations; HR = Human Relations. 20 TABLE 13 DEPARTMENTS INVOLVED IN SMS IMPLEMENTATION STUDIES No. Accountable Executive Position Title Class 1. Airport Director/Safety Manager Class II 2. Airport Director Class IV 3. Airport Executive Director Class I 4. Airport Manager (3) Class I 5. Airport Manager Class II 6. Airport Manager Class IV 7. Airport Manager/County Commissioner Class II 8. Assistant Director of Aviation Class I 9. Aviation Director Class IV 10. Aviation Director Class IV 11. Aviation General Manager Class I 12. Chief Executive Officer (3) Class I 13. Chief Financial Officer Class I 14. Chief Operating Officer Class I 15. Deputy Director of Operations Class I 16. Director of Airport Operations Class I 17. Director of Aviation Class I 18. Executive Director of the Airport Authority Class I 19. Executive Vice President of Operations Division Class I 20. Fire Chief Class I 21. President Class IV TABLE 14 ACCOUNTABLE EXECUTIVE POSITION WITHIN ORGANIZATION

23 While not a proposed required position by the FAA (in the NPRM), airports were asked whether a safety manager had been designated in addition to the accountable executive. As shown in Table 15, of the 26 airports surveyed, 22 responded “yes” and four replied “no.” Because not all airports are implementing SMS, the survey results were analyzed and nonimplementation airports that replied “yes” indicated that a safety manager had been proactively identified within the organization to establish the position if and when the pro- gram becomes formal. All airports were asked to further define the safety man- ager role as either a full-time or collateral duty position. As shown in Figure 10, of the 26 airports surveyed, four indi- cated full-time and 20 reported collateral duties. The nonim- plementation airports that reported on safety manager duties indicated that collateral duties are anticipated in the future. As a follow-up to the question on the safety manager posi- tion, airports were asked whether they plan on adding staff to support their SMS, specifically, “If you plan on adding staff, how many total positions do you anticipate?” For most airports the staffing plan is proposed for the future. Figure 11 provides a summary of responses by airport class, including the most frequent response of “none” (11) and “one addi- tional staff” as the second most provided answer with six total responses. A review of the collateral duty findings in Figure 10 and the plans to augment staff in Figure 11 indi- cates that airports are not planning to hire additional staff for the SMS and are intending to expand the duties of their existing staff to support the program efforts. Airports were not asked to explain the basis of their replies; therefore, the data results are unable to document the reasons behind the staffing decisions. Although airports were discussing full-time and collat- eral duty staffing, the question was asked, “Please describe any duties that will be performed by existing or new staff.” Table 16 provides a summary of additional duties planned for staff, including program-level duties and specific efforts associated with the four SMS components (safety policy, safety risk management, safety assurance, and safety pro- motion). Additional tasks or duties tended to include hazard identification and reporting, SRA coordination and facilita- tion, training, and overall program management. Budget For each of the SMS pilot studies, airports were eligible for grant funds to support the project efforts. Each SMS pilot study scope of work limited some expenditure types. Allowable funding was typically based on service level and hub type, TABLE 15 DESIGNATED A SAFETY (SMS) MANAGER Safety (SMS) Manager Designated Yes No Total Class I 14 4 18 Class II 3 0 3 Class IV 5 0 5 Count 22 4 26 FIGURE 10 Safety manager position full-time or collateral. FIGURE 11 Additional staff planned.

24 TABLE 16 SMS STAFF DUTIES Class Program Safety Policy Safety Risk Management Safety Assurance Safety Promotion Class I Program development and management, including coordination with information technology SRA logistics including facilitation Class I Participate in safety meetings and manage program Data quality reviews Training Class I Safety manager for all aspects SRM, SRA Forensics, data tracking, reporting, and expanded inspection program Training development and rollout Class I Project manager for SMS, coordinate other department efforts such as maintenance and technology Data management Class I Accountable for program Quality assurance audits and coordination Class I Reporting processes Awareness training for staff and tenants Class I Program manager for all SMS elements including OSHA, supervise staff in department and develop program elements, report to management Assist with policy deployment to staff and tenants Conduct and coordinate SRAs Manage audit program Manage and deliver training program Class I Participate in safety review board and provide oversight Execute and facilitate SRAs including panel logistics Update hazard and audit lists Class I Audits, inspections SMS trainer Class I Coordinate SRAs Collect and analyze data Class I Hazard reporting Class I Overall management of program Risk analysis and reporting Investigations Class I Reporting hazards Promoting safety culture Class I Report to management Maintain hazard log, track mitigations Communicate issues, provide training Class I Coordinate program Provide training Class I Conduct inspections Manage training requirements, oversee training dept. efforts Class I Manage program as needed and expanded, Work with construction to Communicate with airport develop SMS Plan implement SMS community Class II Attend safety meetings Record keeping and reporting, provide statistical reports Conduct annual safety training Class II Hazard reporting and management Class IV Program management Class IV Develop safety plan, manage program Class IV Senior supervisor with safety and training responsibilities Data analysis and reports Provide training Class IV Oversee staff and program from a high-level perspective Class IV Communicate and report concerns from staff and tenants

25 where applicable. The cost amounts in Table 17 were provided by the FAA and are presented by each SMS pilot study as rounded actuals. Note that not all airports pursued reimburse- ment for the total allowable amount of grant funding. Airports surveyed were also asked whether the funded amounts were sufficient to support the SMS pilot study costs. Responses are reflected in Figure 12. Eighteen of the airports reported that “yes” the funding was sufficient to complete the studies, four replied “no,” and the remaining two did not apply for funding and, therefore, the question was not relevant. A set of three questions focused on whether airports believed there were hidden costs within the SMS implemen- tation study project or in the future development or opera- tion of the SMS. Results from the questions are presented in Tables 18 and 19. With regard to the project or program challenges, the majority (13) responded that there were no challenges. The remaining responses included program management, training, SRA mitigations, software, and reporting. Airports were asked to consider foreseeable hidden costs and possible future challenges with funding. Similar results were reported from each question and have been consolidated in Table 18. The top three responses included (1) program man- agement, (2) software procurement, and (3) SRA mitigations. According to the SMS NPRM, “One of the most impor- tant aspects of hazard identification is systematically docu- menting and tracking potential hazards. This documented data allows meaningful analysis of operational safety-related trends on the airfield and of overall airport system safety” (FAA 2010). The NPRM further states, “Few certificated air- ports [within the SMS pilot studies] indicated formal proce- dures to systematically review safety-related data. All pilot study airports have record-keeping and retrieval systems in place, but each indicated room for improvement. Improved systems would allow for trend and other data analysis to pro- actively identify operational hazards and potentially prevent future incidents or accidents” (FAA 2010). In order to collect information from SMS pilot study airports, each of the four studies included assessments or deliverables associated with data collection and reporting, software analysis, and/or software procurement. Studies 1 and 2 asked airport representatives to investigate nonpuni- tive reporting systems and SRM trend analysis and SRM documentation management (electronic or paper). Study 3 included several activities related to data collection, data tracking and trending activities, and reporting systems, including the following: 1. Reviewing third-party data collection, collaboration, and reporting systems to collect, store, and report on SMS events, trends, and activities; SMS Pilot Study Year Amount Range Study 1—Gap Analysis, SMS Manual, Implementation Plan Open to Class I, II, III, and IV Airports 2007 $67,000–$200,000 Study 2—Gap Analysis, SMS Manual, Implementation Plan Limited to Class II, III, and IV Airports 2008 to 2010 $36,000–$100,000 Study 3—Proof of Concept 2008 to 2009 $180,000–$375,000 Study 4—Implementation 2010 to 2012 $78,500–$500,000 TABLE 17 GRANT FUNDING FIGURE 12 Sufficiency of grant amount.

26 2. Developing a tracking and trending program for foreign object debris/damage (FOD) identification and control; 3. Developing a personnel training and record-keeping program with the ability to alert or query data and inte- grate into any of the airport’s related tracking or train- ing programs currently in use; 4. Developing a ramp and baggage makeup area (if appli- cable) self-inspection program and processes for report- ing, tracking, and trending safety issues; and 5. Developing a minimum standard for safe operations on the ramp and in the baggage makeup area (if applicable) to list baseline incidents and accidents for trending. Under the FAA’s list of eligible expenses, Study 4 allowed for the procurement of hazard and mitigation tracking software systems and safety reporting software systems. Additionally, Study 4 included a number of software and data reporting study tasks and deliverables, including the following: 1. Implement a safety reporting and/or data collection sys- tem or applicable processes in conformance with the airport’s SMS manual or other documentation devel- oped for the airport under the first pilot studies; 2. Collect hazard reports, incident and accident reports, and other safety-related data and information under the air- port’s SMS manual or other applicable documentation; 3. Analyze the information collected through the report- ing and/or data collection system or applicable pro- cesses; and 4. Report on (if software was purchased or developed) the usefulness of the software, costs associated with devel- opment, procurement, and maintenance, and challenges or lessons learned using the software. Additional information regarding data collection and trending is presented in chapter five. LESSONS LEARNED This chapter presents survey response information relating to the overall program management, staffing, and budgeting for a SMS. The majority of airports were able to reply to all ques- tions, regardless of their implementation or nonimplementa- tion status. Trends from the analysis by section are presented here, including lessons learned gleaned from the surveyed airports and subsequent analysis of the compiled information. SMS Program Management Lesson Learned: A clear definition of the SMS project develop- ment documents, such as a program plan and schedule, assists with the airport’s ability to design, plan, and deploy the SMS. Gap Analysis Process and Report The overall consensus from the surveyed airports is that con- ducting a gap analysis is useful as a means to assess the cur- rent operational activities and to serve as a baseline needs assessment for SMS development. Many consultants used the ICAO gap analysis checklist as a tool to assess SMS and Part 139 operations because there was not one available from the FAA for their use. ICAO strongly encourages the use of N o C ha lle ng es R ep or te d T ra in in g T ec hn ol og y/ S of tw ar e P ro cu re m en t SR A M iti ga tio ns C ol la te ra l D ut ie s by A ir po rt S ta ff S M S Pi lo tS tu dy D oc um en ts D ef ic ie nt ,N ot U se fu l N ot A de qu at e F un di ng f or I m pl em en ta ti on L ac k of G ui da nc e Is C os tl y Class I 10 3 3 1 2 1 1 Class II 1 1 Class IV 2 1 Count 13 3 3 2 2 1 1 1 TABLE 18 FUNDING-RELATED CHALLENGES WITHIN THE PROJECT P ro gr am M an ag em en t S of tw ar e P ro cu re m en t/ T ec hn ol og y SR A M iti ga tio ns T ra in in g Pr og ra m D at a R ep or tin g Pr om ot io na l C ul tu re P ro gr am A va ila bi lit y of F ut ur e Fu nd in g Class I 7 7 5 2 2 2 2 Class II 3 1 1 Class IV 3 1 Count 13 9 6 2 2 2 2 TABLE 19 POSSIBLE HIDDEN COSTS AND FUNDING- RELATED CHALLENGES IN THE FUTURE Many consultants used the ICAO gap analysis checklist as a tool to assess SMS and Part 139 operations because there was not one available from the FAA for their use.

27 the gap analysis in the SMS manual and indicates that the gap analysis tool assists in developing the SMS Implementation [Program] Plan. At the time, and even currently, this is the best known tool for performing the gap analysis. Lesson Learned: Ensuring adequate or flexible SMS pro- gram timelines to conduct a gap analysis (or other business analysis efforts) assists airports in developing program plans. SMS Manual Development Study 1 and 2 project durations totaled 6 months. Survey respondents reported that the timeline restricted, to some degree, the ability for staff to develop a complete and thor- ough SMS manual. However; all airports and consultants developed and delivered a SMS manual as part of the required SMS pilot study program deliverables. Most implementations indicated that airports used the SMS manuals developed in the initial SMS pilot study as a baseline and have either com- pletely revised or revamped the SMS manual as part of the implementation process. Two of the three airports participat- ing in Study 3, Proof of Concept, indicated that the SMS man- ual was revised as part of the SMS pilot study and was further updated during the SMS implementation study as program components were rolled out and, specifically, as SRM tools (such as the risk matrix) were tested while conducting SRAs. Some airports reported they have increased the contents; others have reduced the amount of information to align the SMS manual with current airport operations. Additionally, some airports stated that the initial SMS manual was devel- oped as an academic exercise and, as the program is being deployed, staff is testing processes and procedures to align with standard operating procedures and, in some cases, with new SMS software programs. Lesson Learned: Allowing adequate time to design, develop, test, and deploy the SMS manual through the life of the program implementation improves the quality of the manual and associated processes. Program Plan Development Most airports developed a program plan during Pilot Stud- ies 1 and 2; however, only approximately half have used the program plan for SMS deployment. Airports commented that the SMS components are being rolled out as needed or encountered (safety risk management, for example, as a result of conducting safety risk assessments) and because no formal milestones are set, the program timeline is flexible. As reflected in the number of airports reporting that they are implementing all SMS components (11), the majority (15) of the group are either not implementing at all (7) or are only implementing some of the components (6). Some airports commented that the program schedule was fluid to ensure that any changes to the SMS Final Rule could be addressed. Lesson Learned: Development of flexible program plans supports the airports’ SMS implementation timelines and staffing constraints. Consultant Services All airports, except one, employed consultants to assist with the various SMS pilot study deliverables and activities. Con- sultants were selected through a variety of means such as requests for proposals, master contract agreements, and other formal processes and procedures. Owing to limited SMS experience at U.S. airports, many of the consultants partici- pating in the first studies brought expertise from non-U.S. efforts (such as Canada and Europe) or from other safety- related professions; however, during the past 4 years, con- sultants have gained U.S. experience. As part of the survey, desired consultant skills with air- ports were discussed. The top three skills reported from the survey that airports believe their consultants need are Part 139 operations expertise, SMS development skills, and SMS implementation experience. Additionally, four of the five Class IV airports indicated airport familiarity and all three of the Class II airports also confirmed that specific airport knowledge was a key skill necessary for SMS consultants. Lesson Learned: Local airport and U.S. Part 139 experience are valuable SMS consultant skills in consultant selection in addition to SMS development and deployment expertise. Reference Documents During the early pilot studies (2007 and 2009), few U.S.- centric guides or documents existed to assist airports and consultants in designing and developing SMS programs. The most frequent reference reported by airports surveyed was the FAA AC 150/5200-37 (2007). Airports commented that as a potential Part 139 addition, the AC served as the single most important available document to assist airports in developing a program that they believed could comply with future FAA requirements. Many airports commented that the lack of guidance documentation from the FAA was a concern. FAA has since committed to updating the AC using lessons learned from the pilot studies. The second most frequently used reference was reported as the ICAO Safety Management Manual. The manual provided a solid SMS outline including sample checklists and forms for SMS program development; however, some of the ICAO- focused aspects of the SMM were incompatible with U.S. The most frequent reference reported by airports surveyed was the FAA AC 150/5200-37.

28 airport operations and were discarded. Airports also reported that industry conferences, presentations, and workshops were of great value (especially in the early stages of the pilot stud- ies) to allow for information sharing and document exchange. Lesson Learned: Formal SMS guidance, documentation, and information-sharing opportunities support the SMS design and development. Staffing Most airports, even those within the nonimplementation group, have identified safety manager and accountable executive positions or titles within the organization. The majority of SMS programs are currently (or will be) man- aged from the operations division or department. Few air- ports have hired full-time staff to support the SMS program; the majority indicated collateral duties will be assigned to support the SMS. Most airports do not plan on hiring staff and only a few airports indicated an additional staff member or two may be employed. Budget constraints were reported as the primary reason for leveraging existing staff instead of employing new hires. Many airports (including implemen- tation airports) are delaying staffing decisions until the rule is finalized. Nearly all airports and consultants involved other depart- ments in each of the pilot studies; however, additional departments, such as legal and properties, were engaged as part of the SMS implementation study to address safety pol- icy development and revisions to leases, licenses, and rules and regulations. The departments most often reported to be involved in the program (other than operations) are mainte- nance and fire/aircraft rescue and fire fighting. Lesson Learned: Collateral duties are the reported norm for current and future SMS staff responsibilities; budget con- straints and delayed hiring decisions based on forthcoming FAA rulemaking are cited as the primary reasons new hires are not planned. Budget Federal grant funding for SMS studies, according to surveyed airports, provided adequate funds to develop the required deliverables. Future budget concerns, including possible hid- den costs, include software and technology, training, and pro- gram management. Airports stated that the need to track and trend data in a software system was either more costly than anticipated (including additional staff time) or was expected to be a significant program cost in the future. Airports also stated that procurement and implementation of a software program were important to the SMS program’s success. Tech- nological solutions to compile, assess, and trend data were cited as critical to safety assurance and audit programs. ICAO states that predictive safety data collection systems are essentially statis- tical systems, whereby a considerable volume of operational data, which alone are largely meaningless, are collected and analyzed, and combined with data from reactive and proac- tive safety data collection systems. The aggregation of data thus leads to the development of a most complete intelli- gence that allows organizations to navigate around obstacles and currents and position themselves optimally within the drift (ICAO 2009). Lesson Learned: Data collection, analysis, and trending are core functions of SMS and require early planning for ade- quate budgets to procure or build technological solutions. Technological solutions to compile, assess, and trend data were cited as critical to Safety Assur- ance and audit programs.