Admissibility and Public Availability of Transit Safety Planning Records (2018)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

20 3 Safety Management Systems and Confidentiality Concerns The Moving Ahead for Progress in the 21st Century Act of 2012 requires transit agencies to develop and implement public transportation agency safety plans. The Federal Transit Administration (FTA) has interpreted this directive in its rulemaking proposals to require safety management systems (SMSs).1 This chapter discusses the origins and basic elements of SMSs, their use in other transportation modes and high-hazard industries, and the types of information generated by these programs that are candidates for confidentiality protections. SMS ORIGINS AND ELEMENTS SMSs originated in the 1970s as an offshoot of the quality management programs promoted by W. Edwards Deming and Joseph Juran, two influ- ential management consultants, after World War II (Swuste et al. 2016). The programs sought continuous improvement in the quality of products and services through means such as worker involvement, statistical process control, and feedback from customers. A core principle of these programs is the “plan, do, check, and act cycle,” which is intended to foster critical thinking and continual improvement. SMSs apply similar organizational, 1 See the 2013 Advance Notice of Proposed Rulemaking for the National Public Transpor- tation Safety Plan, the Safety Certification Training Program, and the Public Transportation Agency Safety Plans (78 FR 61251) and the 2016 Notice of Proposed Rulemaking for the National Public Transportation Safety Plan, the Safety Certification Training Program, and the Public Transportation Agency Safety Plans (81 FR 6343).

SAFETY MANAGEMENT SYSTEMS AND CONFIDENTIALITY CONCERNS 21 managerial, and behavioral strategies. They were introduced to augment the safety benefits conferred by improved engineering and compliance with equipment and operating standards (Hudson 2007). The SMS approach is intended to prompt an organization’s executives, managers, and frontline workers to become vigilant, self-evaluative, and deliberate in identifying and controlling all of their risks, including system- level risks arising from the diverse, complex, and changing interactions among human operators, technologies, and environmental and operating conditions. The expectation is that the firms’ leaders and managers will study their operations comprehensively and develop strategies suited to mitigating the risks they identify (Coglianese 2010). In its rulemaking pro- posal on public transportation agency safety plans, FTA has stated that the SMS approach is intended to identify and address latent conditions by making executive leadership accountable for them; by requiring deference to technical expertise in evaluating and mitigating them; and by fostering a culture of information sharing in the performance of work and the implementation of identified controls and risk management strategies. (FTA 2018) SMSs required by government safety regulators usually define the ba- sic elements of a compliant system. For example, organizations may be required to conduct an internal risk analysis; identify and evaluate risk control options; prepare a written plan for communicating safety-related work rules and ensuring that they are understood and observed; and estab- lish procedures supporting manager and worker training, documentation, and compliance monitoring (Silbey and Agrawal 2011). The SMS plan might include specific steps, such as installation of safety equipment or routine monitoring of operations, as well as practices, such as documenta- tion and record keeping, to ensure that all the planned steps are taken. A particular process and time interval for reassessment of risks and updating of the operations plan would likely be part of the required SMS plan. In addition, the regulations may require periodic program audits and other feedback mechanisms to support efforts to improve the firm’s safety man- agement (Chinander et al. 1998; Coglianese and Lazer 2003; Kunreuther et al. 2002). Citing weaknesses in the “safety culture” of public transit agencies, FTA has maintained that the introduction of SMSs can support the devel- opment of stronger safety cultures (FTA 2018). The claim is that transit service providers, like most public and private transportation organizations, tend to function with a rigid structure of rules to accommodate a dispersed workforce operating on fixed schedules. In many such organizations, work- ers who ignore or break a rule are subject to discipline, which creates a “blame” culture impeding two-way communication between manage-

22 ADMISSIBILITY AND AVAILABILITY OF TRANSIT SAFETY PLANNING RECORDS ment and labor, which is needed to identify and manage risks effectively (Zuschlag et al. 2016). FTA believes that SMSs will lead to and support management’s communication of organizational safety values. The com- munication of these values, in turn, is expected to create a workforce of employees who do not fear reporting incidents (even if the incident results from an employee’s mistake) and who accept personal accountability for safety. SMSs IN TRANSPORTATION AND OTHER HIGH-HAZARD INDUSTRIES As discussed in Chapter 2, the National Transportation Safety Board (NTSB) has been a leading proponent of the adoption of SMSs by trans- portation organizations. NTSB has recommended the development of both government regulations requiring SMSs and voluntary SMS standards in the pipeline, rail transit, and aviation modes.2 In 2011, NTSB included SMSs on its “most wanted list” of actions to improve transportation safety. It stated that accident investigations have revealed that, in numerous cases, safety man- agement system (SMS) or system safety programs could have prevented loss of life and injuries. . . . These programs continually monitor opera- tions and collect appropriate data to identify emerging and developing safety problems before they result in death, injury, or significant property damage. Aviation, railroad, highway and marine organizations should establish SMS or system safety programs. These programs establish pro- cesses to collect and analyze data on potential safety problems and then evaluate mitigations to resolve the safety risk before an accident happens. . . . A fundamental principle of such systems is non-punitive reporting that applies to both operating companies and the personnel involved in transportation organizations. (NTSB 2018) FTA’s proposal for the use of SMSs by transit agencies is consistent with NTSB’s recommendations and with the adoption of SMS principles in other transportation modes. One of the earliest calls for SMSs in the trans- portation domain was in the maritime sector. The International Maritime Organization issued requirements for tankers, bulk carriers, and passenger vessels in international voyages to adopt SMSs by 1996, with all other vessel types to do so by 2002. The U.S. Coast Guard responded in 1998 with applicable federal regulations (Federal Register 1997). In 2006, the International Civil Aviation Organization established an SMS standard for 2 See NTSB recommendations P-12-017 (pipelines), R-14-067 (rail transit), and A-06-36 (aviation).

SAFETY MANAGEMENT SYSTEMS AND CONFIDENTIALITY CONCERNS 23 air carriers operating in international commerce,3 and in 2010 the Federal Aviation Administration (FAA) directed major airlines to establish SMSs. FAA likewise required its air traffic control operations to adopt SMSs.4 In response to a recommendation by NTSB, the American Petroleum Institute (API) has developed a voluntary SMS standard for the pipeline industry (API Recommended Practice 1173). The Pipeline and Hazardous Materials Safety Administration had earlier issued integrity management regulations that had elements of these management systems through re- quirements that pipeline companies engage in formal hazard identification, risk assessment, and risk mitigation planning with respect to their physical assets. API Recommended Practice 1173 goes further by advising pipeline operators on the steps to be taken for managing safety in the broader context of a pipeline operator’s full range of activities in addition to its physical assets. As noted in Chapter 1, the Federal Railroad Administration (FRA) is- sued a final rule requiring intercity passenger and commuter railroads to develop and implement system safety plans that are similar in design to SMSs (Federal Register 2016). FRA has also issued a proposed rule that would require Class I freight railroads and other smaller railroads with inadequate safety records to institute similar system safety plans (Federal Register 2015). FRA’s interest in SMSs was influenced in part by experi- ence in Canada, which has had SMS requirements that apply to its freight railroads since 2007 (Transport Canada 2007). Although SMSs are relatively new to the U.S. transportation sector, they have been in use in other high-hazard industries for decades, often in response to large-scale industrial incidents. SMSs are widely used in the nu- clear power industry and have been advocated by the International Atomic Energy Agency since the 1990s (Durbin 2006; IAEA 2015). The European Union (EU) has made SMSs a cornerstone of its chemical safety initiatives for more than 30 years. In response to a 1976 incident at a chemical plant near Seveso, Italy, which caused the deaths of thousands of animals and destroyed local crops, the EU required chemical plants containing certain levels of hazardous materials to improve and report on their processes for ensuring safety. A second directive issued in 1996 required plants to in- stitute SMSs (Swuste and Reniers 2016). Similarly, the U.S. Occupational Safety and Health Administration established an SMS-like process safety management standard for the chemical industry as part of its regulations implementing the Clean Air Act Amendments of 1990 (OSHA 2000). 3 This requirement is described in FAA’s 2010 proposed rule mandating SMSs for Part 121 carriers (FAA 2010). 4 See FAA Air Traffic Order JO 1000.37, Air Traffic Organization Safety Management Sys- tem (March 19, 2007), and Air Traffic Organization SMS Manual, Version 2.1 (May 2008).

24 ADMISSIBILITY AND AVAILABILITY OF TRANSIT SAFETY PLANNING RECORDS Another early adopter of SMSs was Europe’s offshore oil and gas indus- try. The United Kingdom’s requirements for SMSs had their origins in the regulatory response to the Piper Alpha offshore disaster of 1987 that killed 167 oil rig workers in the North Sea. Norway, which had experienced its own offshore disasters, responded in a similar manner by requiring SMSs for offshore oil and gas operators. Both countries now require offshore operators to prepare and implement a safety and environmental manage- ment system (SEMS) (TRB 2018). In the United States, SEMS had been a voluntary API standard (API RP 75) until the aftermath of the April 2010 Deepwater Horizon explosion and oil spill in the Gulf of Mexico. Since September 2010, the U.S. Department of the Interior has required offshore operators to establish SEMS programs. EMPIRICAL EVIDENCE OF SMS EFFECTIVENESS SMSs have substantial logical and intuitive appeal. They are applied across many industries and companies around the world and are considered to be a “best practice” in the management of safety. However, the effectiveness of SMSs in improving workplace (occupational) safety or avoiding cata- strophic events has not been well established empirically. Evaluation studies of this type are difficult to carry out. SMSs have typically been introduced at the company or organizational level, and few organizations have been willing to allow independent evaluation or to serve as controls (Robson et al. 2007). Efforts to study experience by groups of firms have suffered from high refusal rates for inclusion in the studies (Robson et al. 2007), and they have raised issues about the consistency of outcome measures from one firm to the next (Pedersen et al. 2012). Studies of SMS effectiveness need to continue for years to obtain strong evidence of changes in outcome measures such as reduced workplace injuries and fatalities. Studies of the impact of SMSs on catastrophic incidents are confounded further by the rarity of such events. Despite these challenges, thousands of articles and reports have been prepared on the general topic of SMS effectiveness. Two meta-analyses have been conducted to assess the results in the literature. Robson et al. (2007) provide a comprehensive assessment of the studies in the literature examining the effectiveness of occupational health and safety management systems (OHSMSs). Thomas (2012) follows similar procedures to assess the effectiveness of SMSs in high-risk industries. After conducting a bibliographic search that yielded nearly 5,000 ar- ticles on OHSMSs in the peer-reviewed literature, Robson et al. (2007) determined that only 23 were based on studies whose methodological rigor was sufficient to warrant a deeper evaluation of results. The deeper methodological evaluations caused the researchers to eliminate another 10 of the studies. Robson et al. (2007) found that 7 of the remaining 13

SAFETY MANAGEMENT SYSTEMS AND CONFIDENTIALITY CONCERNS 25 studies examined voluntary OHSMS programs. These studies reported generally positive results, such as improved safety climate and employee reporting of hazards and reduced workplace injuries and disability-related costs. The other six studies examined mandatory OHSMSs. Robson et al. (2007) found that these studies too reported generally positive outcomes, such as increased employee awareness of and participation in health and safety activities, decreased lost-time injury rates, and increased worker productivity. Although the results of the 13 studies were generally positive and none were negative, Robson et al. (2007) viewed the overall body of evidence on OHSMSs as inconclusive because of the inability to draw valid inferences from the methodologies used. The researchers noted that only 3 of the 13 studies had a comparison group of any kind, while the others used a before and after approach or quasi-experimental statistical designs. The single study rated as having a high-quality methodological design was limited to an assessment of the effectiveness of introducing an OHSMS. The broader analysis by Thomas (2012) followed a methodology simi- lar to that of Robson et al. (2007) but restricted the analysis to studies of SMS effectiveness in high-hazard industries. Among more than 2,000 can- didate reports and articles, 14 studies examined the effectiveness of these systems in reducing the risk of low-probability, high-consequence events. However, most of these studies relied on subjective assessments of safety outcomes as opposed to quantitative metrics such as reduced precursor or catastrophic events. While the analyzed studies suggested that SMSs im- prove safety, stronger conclusions were not possible because of the scarce quantitative evidence of effectiveness. SMS DATA AND CONFIDENTIALITY REQUIREMENTS SMSs were first developed and applied by private industry. The nature and types of data and reports generated by private companies to support these systems are not unlike those required by public agencies implementing similar systems. The systems, for example, generate • Information required for hazard identification and risk assessment; • Employee reports of safety concerns or incidents that might have turned into crashes (close calls); • Safety goals and progress reports on achieving those goals; • Inspection and audit reports; and • Crash investigations, including determination of causal factors. Of particular concern to users of SMSs is the potential for disclosure of voluntary reports from employees about hazardous conditions or incidents observed in the field. Incident, or close call, reporting systems are consid- ered integral to the success of SMSs but appear to be particularly important

26 ADMISSIBILITY AND AVAILABILITY OF TRANSIT SAFETY PLANNING RECORDS in transportation industries and for a transportation organization’s learning about and management of safety. Transportation workers encountering risks or making errors may be far removed from operations centers when the incident occurs. Gaining access to such information is complicated when the reporting employee is anxious about legal jeopardy arising from disciplinary procedures that may result in lost wages or reputational harm. Some agencies have volunteered to participate in confidential trans- portation safety reporting, which is also known as “confidential close calls reporting.” These systems facilitate the gathering of reports from employees about close calls related to safety exposures experienced or observed in the field, facilitate collaborative analysis of those data, and share information about operations safety. SMSs welcome but do not require this approach. Preinjury confidential reporting systems capture widely varied safety expo- sures, such as unsafe acts, hazardous conditions, and equipment malfunc- tions, preventively. However, such systems are still relatively rare, in part because their efficacy relies heavily on the trust and open dialogue afforded by their evidentiary protections. FAA, which has had a national-level confidential close call reporting system (C3RS) in place for many years (even before the agency required SMSs in the aviation industry), keeps these reports confidential through the designated role of the National Aeronautics and Space Administration (NASA) in collecting and analyzing reports (NASA 2018). NASA’s statutes and internal policies provide confidentiality protections for persons report- ing close calls. FRA also maintains a C3RS in partnership with the U.S. Department of Transportation’s Bureau of Transportation Statistics, which has been given statutory authority to maintain the confidentiality of all re- ports (FRA 2018). FRA’s C3RS process, which began with freight rail and was later extended to commuter and intercity passenger railroads, consists of three main steps: (1) railroad employees report close calls to a neutral third party (the Bureau of Transportation Statistics and NASA), (2) the third party removes identifying information from the reports and passes the anonymized information to a labor–management peer-review team trained in incident analysis, and (3) the analytic team recommends corrective action to management. On the basis of its own studies of the C3RS program, FRA has con- cluded that the confidential reporting system has been successful in identify- ing safety risks that would not otherwise have been identified (FRA 2013, 2015a,b). The agency’s analyses show a significant reduction in rules viola- tions and human factors–caused derailments after C3RS was introduced. Qualitative measures have led the agency to conclude that the program has strengthened relationships between labor and management and led to a stronger organizational safety culture. FRA has emphasized that such posi- tive outcomes could not be achieved without the confidentiality provisions

SAFETY MANAGEMENT SYSTEMS AND CONFIDENTIALITY CONCERNS 27 that allow for close cooperation and negotiation among federal agencies, worker representatives, and railroad management. Although they are not full SMSs, the safety plans of state highway agencies are illustrative of the kinds of information compiled as part of the safety management programs of public and private organizations. State highway agencies may identify hazardous locations and prioritize improve- ments by combining a wide variety of data and reports: police incident reports; traffic counts; vehicle speed data; and the features of highway loca- tions such as curves, hills, and clear zones. The records may include crash analysis diagrams, videotapes, photographs, and speed studies (TRB 2016). These data are analyzed by engineering staff and used in hazard ranking formulas for prioritizing locations that merit safety upgrades. REFERENCES Abbreviations FAA Federal Aviation Administration FRA Federal Railroad Administration FTA Federal Transit Administration IAEA International Atomic Energy Agency NASA National Aeronautics and Space Administration NTSB National Transportation Safety Board OSHA Occupational Safety and Health Administration TRB Transportation Research Board Chinander, K. R., P. R. Kleindorfer, and H. C. Kunreuther. 1998. Compliance Strategies and Regulatory Effectiveness of Performance-Based Regulation of Chemical Accident Risks. Risk Analysis, Vol. 18, No. 2, pp. 135–143. Coglianese, C. 2010. Performance-Based Regulation: Concepts and Challenges. In Compara- tive Law and Regulation: Understanding the Global Regulatory Process (F. Bignami and D. Zaring, eds.), Edward Elgar, Northampton, Mass. Coglianese, C., and D. Lazer. 2003. Management-Based Regulation: Prescribing Private Man- agement to Achieve Public Goals. Law and Society Review, Vol. 37, No. 4, pp. 691–730. Durbin, N. E. 2006. Review of the International Oversight of Safety Culture in Nuclear Facili- ties. Letter Report Prepared for the U.S. Nuclear Regulatory Commission. Idaho National Laboratory, Battelle Energy Alliance. FAA. 2010. Safety Management Systems for Part 121 Certificate Holders. 75 FR 68224, pp. 68224–68245. https://www.federalregister.gov/documents/2010/11/05/2010-28050/ safety-management-systems-for-part-121-certificate-holders. Federal Register. 1997. International Management Code for the Safe Operation of Ships and for Pollution Prevention. Vol. 62, p. 67492, Dec. 24. To be codified at parts 46 CFR 2, 31, 71, 91, 107, 115, 126, 175, 176, and 189. https://www.gpo.gov/fdsys/pkg/FR-1997- 12-24/pdf/97-33528.pdf. Federal Register. 2015. Risk Reduction Program. Federal Railroad Administration Proposed Rule. 80 FR 10949, pp. 10949–10999. Docket No. FRA-2009-038, Notice No. 1. https:// www.federalregister.gov/documents/2015/02/27/2015-03268/risk-reduction-program.

28 ADMISSIBILITY AND AVAILABILITY OF TRANSIT SAFETY PLANNING RECORDS Federal Register. 2016. System Safety Program. Federal Railroad Administration 49 CFR Part 270, Docket No. FRA-2011-0060, Notice No. 3. https://www.gpo.gov/fdsys/pkg/ FR-2016-08-12/pdf/2016-18301.pdf. FRA. 2013. Another C3RS Site Improves Safety at Midterm. Research Results Report 13-49. FRA. 2015a. Confidential Close Call Reporting System (C3RS) Lessons Learned Team Base- line Phase Report. FRA. 2015b. Continued Improvements at One C3RS Site. Research Results Report 15-17. FRA. 2018. Confidential Close Call Reporting System. https://www.fra.dot.gov/c3rs. FTA. 2018. A Safety Management Systems (SMS) Approach to Strengthening Transit Safety in the United States. https://www.transit.dot.gov/regulations-and-guidance/safety/ safety-management-systems-sms-approach-strengthening-transit-safety. Hudson, P. 2007. Implementing a Safety Culture in a Major Multi-National. Safety Science, Vol. 45, No. 6, pp. 697–722. IAEA. 2015. Management Systems for Nuclear Facilities and Activities. https://www.iaea.org/ NuclearPower/ManagementSystems/index.html. Kunreuther, H. C., P. J. McNulty, and Y. Kang. 2002. Third-Party Inspection as an Alterna- tive to Command and Control Regulation. Risk Analysis, Vol. 22, No. 2, pp. 309–318. NASA. 2018. Aviation Safety Reporting System. https://asrs.arc.nasa.gov. NTSB. 2018. Safety Management Systems. https://www.ntsb.gov/safety/mwl/Pages/mwl-3. aspx. OSHA. 2000. Process Safety Management. OSHA 3132. https://www.osha.gov/Publications/ osha3132.html#problem. Pedersen, L. M., K. J. Nielsen, and P. Kines. 2012. Realistic Evaluation as a New Way to Design and Evaluate Occupational Safety Interventions. Safety Science, Vol. 50, No. 1, pp. 48–54. Robson, L. S., J. A. Clarke, K. Cullen, A. Bielecky, C. Severin, P. L. Bigelow, E. Irvin, A. Culyer, and Q. Mahood. 2007. The Effectiveness of Occupational Health and Safety Management Systems Interventions: A Systematic Review. Safety Science, Vol. 45, No. 3, pp. 329–353. Silbey, S. S., and T. Agrawal. 2011. The Illusion of Accountability: Information Management and Organizational Culture. Droit et Société, No. 77. Swuste, P., and G. Reniers. 2016. Seveso Inspections in the European Low Countries History, Implementation, and Effectiveness of the European Seveso Directives in Belgium and the Netherlands. Journal of Loss Prevention in the Process Industries, Vol. 49, pp. 68–77. Swuste, P., C. van Gulijk, W. Zwaard, S. Lemkowitz, Y. Oosetendorp, and J. Groeneweg. 2016. Developments in the Safety Science Domain, in the Fields of General and Safety Management Between 1970 and 1979, the Year of the Near Disaster on Three Mile Island, a Literature Review. Safety Science, Vol. 86, pp. 10–26. Thomas, M. J. W. 2012. A Systematic Review of the Effectiveness of Safety Management Sys- tems. Final report. ATSB Transport Safety Report XR-2011-002. Australian Transport Safety Bureau. Transport Canada. 2007. Moving Forward—Changing the Safety and Security Culture: A Strategic Direction for Safety and Security Management. TP 14678. TRB. 2016. NCHRP Legal Research Digest 72: Summary of Federal Law Restricting Use of Highway Safety Data in Tort Litigation. National Academies of Sciences, Engineering, and Medicine, Washington, D.C. TRB. 2018. Special Report 324: Designing Safety Regulations for High-Hazard Industries. National Academies of Sciences, Engineering, and Medicine, Washington, D.C. Zuschlag, M., J. M. Ranney, and M. Coplen. 2016. Evaluation of a Safety Culture Interven- tion for Union Pacific Shows Improved Safety and Safety Culture. Safety Science, Vol. 83, pp. 59–73.