Transit Safety Risk Assessment Methodologies (2021)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

7   Literature Review Introduction to Safety Risk Assessment Methodologies SRAs are one segment of the larger risk management process that an agency or organiza- tion engages in to ensure the safety of staff, the public, and the environment. Risk management encompasses the processes involved in assessing, prioritizing, and strategizing to manage and minimize risk. Risk assessments are processes that evaluate the risk after hazards are identified and before mitigation efforts are developed. A hazard is a potential source of harm or an event that could lead to damage or loss of property or environmental damage (MIL-STD-882). Risk is the likelihood of the hazard occurring and the severity of the consequences. A matrix is a common way to combine probability and severity for risk evaluation. There are many varia- tions and iterations of risk matrices, as well as variations on “what if ” event scenarios (Lyon and Popov, 2016). The majority of widely used risk assessment methods are qualitative or semi-qualitative. The Hazard Analysis Guidelines for Transit Projects (Adduci et al., 2000) is published by FTA and has been widely used by the transit industry. It introduces the Prelimi- nary Hazard Analysis (PHA), Failure Modes and Effects Analysis (FMEA), and Operating Hazard Analysis (OHA) as hazard analysis techniques, while suggesting that these analyses should be conducted in accordance with the latest version of MIL-STD-882. Besides what has been commonly adopted in transit agencies, this review will examine and describe some of the most widely used methods in a range of industries including rail, aircraft, and air traffic management, as well as hazardous material transport (Table 1). A more complete summary table of the SRA methodologies presented in this section is in Appendix C: Summary Table of SRA Methodologies. Fact-Finding Interviews During the literature review, the authors had discussions with people in different industries to better understand SRA methodologies being used outside of transit. The team visited with people from freight services, construction, trucking and warehousing, waterways, hazardous materials transport, and oil and gas. The following bullet points summarize the discussion highlights: • Although industries may be focused on safety, a formal SRA may not be part of the practice. • Larger companies have the resources to hire staff to specialize in SRA. In smaller companies, resources are often not available to have specialists conducting SRAs; rather, a person may have SRA as a part of their responsibilities. • Other methodologies mentioned include – Root Cause Analysis—Looks at the relationship between probability and severity, which is used in a more qualitative way. C H A P T E R 2

8 Transit Safety Risk Assessment Methodologies – Five-Step Risk Assessment—Similar to Simplified International Association of Marine Aids to Navigation and Lighthouse Authorities (IALA) Risk Assessment (U.S. Department of Homeland Security, 2019). – Port Access Route Study—Risk assessment to devise traffic separation schemes. – The Ports and Waterways Safety Assessment (PAWSA) identifies major waterway safety hazards, estimates subsequent risk levels, evaluates potential mitigation measures, and sets the stage for implementation of selected measures to reduce risk. The PAWSA process involves convening a select group of waterway users/stakeholders and conducting a 2-day structured workshop to identify and discuss these aforementioned elements of risk mitigation. Includes 24 risk factors divided into six categories (U.S. Department of Homeland Security, 2019). – Rail Corridor Risk Management System—Adopts 27 factors including hazmat volume, trip length, track quality, human factors, population density along track, and so forth. It is a quantitative scoring method, which can be used to compare routes and identify improve- ments (Association of American Railroads, 2020). – Leading and lagging indicators. – Consequence and probability matrix. – Bowtie—Barriers between hazards and threats. – Task Hazard Analysis—A daily form used in construction to identify and eliminate/ reduce risk for specific tasks. • Insurance companies meet with the companies they insure each year to review a loss run report. Top causes or areas causing loss are discussed. The goal is to address these problems with mitigation efforts. Study of Safety Risk Assessment Methodologies Qualitative and Semi-Qualitative Methods The majority of risk assessment methods in use are qualitative or semi-qualitative. Risk levels are often set by subject matter experts (SMEs), and historical information is used to estimate probabilities. Method Industry Qualitative or Quantitative Summary MIL-STD-882 Defense, rail, transit Qualitative The MIL-STD-882 is a risk assessment method that uses a matrix of probability and severity. European Common Safety Method (CSM) Rail Qualitative The CSM is a policy method used in the European Union (EU). Risk is evaluated using historical data and comparisons with similar systems. Event Tree Analysis (ETA) Aerospace, rail, chemical, energy Qualitative The ETA models the sequence of events that results from a hazard that describes how serious consequences can occur. The method is graphical. Fault Tree Analysis Aircraft, rail, aerospace, energy Qualitative The fault tree analysis begins with a “top event” hazard, and an analysis is carried out along a tree path from the top event. The method is graphical. Table 1. Most commonly used safety risk assessments.

Literature Review 9   Military Standard 882 The MIL-STD-882 is the most frequently used risk assessment framework for public transit agencies in the United States. The method involves several key steps. First, the user identifies and documents potential hazards within the whole system as well as the potential impacts to people, property, and the environment. The next key step is the assessment and documentation of risk in terms of severity (how serious is the potential hazard) and probability (how likely is it for the hazard to occur). The severity categories range from 1 to 4 and correspond to descriptive terms. The most severe is 1, and the increasing numbers correspond to lesser levels of severity— 4 being the least severe. The scale of probability uses a lettered scale from A, frequent, to F, eliminated. Some agencies use quantitative data to assess the probability when available. When quantitative data are not available for a given hazard or event, probability is assessed on judgment calls from SMEs or other professionals. The combination of probability and severity creates a matrix (Figure 2) that is widely applied in the United States. MIL-STD-882 is recommended by FTA as a risk assessment tool (U.S. Department of Transportation, 2019). Most transit agencies choose to use the MIL-STD-882 risk assessment method or modify the matrix to fit their specific needs. For example, the Massachusetts Bay Transportation Authority or MBTA (Prominiski and Chronley, 2020) in Boston, Massachusetts, uses a modified risk assessment matrix (Figure 3). In their modification, MBTA created an additional severity category to expand from four to five possible categories and has removed the “eliminated” probability. The modified matrix also features a Step 2b Proactive Assessment column. Rather than including the probability information on the left column, where it is in the MIL-STD-882, probability is contained within the Proactive Assessment column on the far right of the MBTA matrix. Canadian Method The Canadian risk assessment method is very similar to the MIL-STD-882 method and is adapted from the European CSM. The process consists of two main steps: hazard identification and risk evaluation. In the Canadian method, risk acceptability can be evaluated using codes of practice, a comparison to similar systems, and/or an explicit risk estimation. The assessment process is invoked by a proposed change to the system and is an iterative process that concludes Source: Acquisition Management Systems Control (2012). Figure 2. Standard MIL-STD-882 matrix.

Source: Prominiski and Chronley (2020). Figure 3. Modified MIL-STD-882 matrix.

Literature Review 11   when all safety requirements are satisfied (CSA Working Group, 2020). The Canadian method also requires that any codes of practice or risk evaluation methods be widely recognized within the rail industry. If the methods are not used within the rail industry, the methods or codes of practice must be accompanied by a justification for their use to ensure applicability for railway risk evaluation (CSA Working Group, 2020). European Common Safety Method The European CSM for risk evaluation and assessment (CSM RA) is a procedure that ensures a baseline conformity of European countries’ safety regulations. The procedure applies to all countries and rail agencies and operators within the EU. Soon after the publication of the Commission Implementing Regulation (EU) No 352/2009 (European Commission, 2009), the European Railway Agency produced guidance on the application of the CSM RA, which provides further explanation of the CSM (European Railway Agency, 2009), as well as processes and applications that were used in some EU member states prior to the introduction of the CSM RA (Jovicic, 2009). After the Commission Implementing Regulation (EU) 402/2013 was adopted in 2013 (European Commission, 2013), the Rail Safety and Standards Board (RSSB) published six guidance documents that address the different elements of the risk management process: 1. Guidance on Planning an Application of the Common Safety Method on Risk Evaluation and Assessment (GE/GN8640). 2. Guidance on System Definition (GE/GN8641). 3. Guidance on Hazard Identification and Classification (GE/GN8642). 4. Guidance on Risk Evaluation and Risk Acceptance (GE/GN8643). 5. Guidance on Safety Requirements and Hazard Management (GE/GN8644). 6. Guidance on Independent Assessment (GE/GN8645). More recently, former member country the United Kingdom came up with guidance that summarizes and explains the main requirements of the CSM RA and specific points on com- pliance in the United Kingdom (Office of Rail and Road, 2018). The European CSM is designed to layer on top of individual countries’ more specific regula- tions rather than to provide detailed regulation on rail system operations. The method is triggered by proposed changes in the system. The proposer assembles a definition of the change and the system that will be affected by the change. Then, the proposer examines the change against significance criteria in the regulation. If the change is significant according to the criteria, the regulation requires the application of a risk management process. Any and all reasonable hazards are to be identified and their risks classified (RSSB, 2014). Safety requirements are identified through one or more of three risk acceptance principles (RSSB, 2014). These three risk accep- tance principles are identical to those required by the Canadians: applications of codes of practice, comparison with reference systems, and explicit risk estimation. A hazard record is also required (RSSB, 2014). An example from a member country, Sweden, illustrates the relationship between the European CSM and an individual country’s requirements. The Swedish rail administration is discussed in documentation from the European Railway Agency guidance mentioned earlier (Jovicic, 2009). The Swedish rail infrastructure manual requires risk identification rather than hazard identification and an “accident catalogue,” which lists various potential accidents. Swedish guidelines also do not explicitly recommend comparison to similar systems, unlike the CSM. The Swedish guidelines recommend conducting an event tree analysis (ETA) and using historic data (European Railway Agency, 2009). In the case of a proposed change to the Swedish rail system, the Swedish rail administration would need to conform to Swedish guidelines and to those required by the EU in the CSM.

12 Transit Safety Risk Assessment Methodologies Bowtie Method The bowtie method is a widely used qualitative assessment technique. This technique is used in the energy, chemical, and hazardous material transportation sectors. The method consists of plausible risk scenarios around a given hazard and ways to prevent the hazard. The shape of a bowtie is how it got its name, with threats on one side, consequences on the other, and barriers between them and the center circle, which represents the hazardous event (see Figure 4). The barriers represent mitigating measures put in place by the agency or organization. This system provides structure to analyze a hazard, communicate risks, identify areas with the greatest impact, and determine when risks are as low as reasonably practicable (CGE Risk Management Solutions, 2015). The scale of assessment is an important factor in the development of a bowtie. If too much detail is included, the system will be too cumbersome to use; if the information is too general, the system will not be helpful (CGE Risk Management Solutions, 2015). The cost of bowtie software is relatively low. Event Tree Analysis The many industries that use event trees to assess risk include the aerospace, chemical, energy, and rail industries. Agencies and institutions such as the International Electrotechnical Commission (IEC) (2010) and the Bureau of Reclamation (2019) have published guidance on conducting the ETA. Event trees model the sequence of events arising from a single hazard, exploring the severity of the resulting consequences. Figure 5 shows an example of the ETA for a fire alarm and control. The ETA starts with an initiating event (explosion), continues with the possible occurrence of each sequential event, and ends with the frequency of potential outcomes. This method is useful for developing mitigation and countermeasures to reduce severity. The ETA is widely applicable; however, the process of setting up the analysis can be incredibly time-consuming, especially when applied to more complex systems (GAIN Working Group B, 2003). Fault Tree Analysis A fault tree analysis is a risk assessment method commonly used in the aircraft, energy, rail, and space industries. Many agencies and institutions have published reference documents for applying the fault tree analysis in general or in their corresponding industries (Vesely et al., 1981; Federal Aviation Administration, 2000; British Standards Institute, 2007; Pandey, n.d.; Texas Department of Insurance, n.d.). The fault tree begins with an event that would cause a hazard (the top event) and an analysis carried out in a tree path. The fault tree analysis also accounts for a combination of causes and intermediate causes. For the hazard analysis of transit projects, besides the PHA, FMEA, and OHA, it may be necessary to also perform a fault tree analysis for certain safety-critical subsystems such as train control (Adduci et al., 2000). Figure 6 shows a fault tree example of a hazard scenario for tunnel evacuation failure. The fault tree analysis method allows for the integration of probability information to transform the method from an entirely deductive qualitative tool into a semi-qualitative tool. The costs of a fault tree analysis software package are comparable to those of event tree software. This cost is estimated to be relatively nominal. The fault tree analysis can be widely applied to various systems; however, like the ETA, the process can be very time intensive (GAIN Working Group B, 2003). Common Cause Analysis The common cause analysis (CCA) is often used within the defense and aircraft sectors. The CCA identifies accident sequences where two or more events could result from one common event. This analysis is useful for processes with a common element such as a common process, manufacturing, or human error. The CCA is typically used to analyze hardware but can be applied to software and human error analysis. While the CCA is typically a qualitative assessment method, it can be used quantitatively. One tool to analyze common cause failure quantitatively

Source: CGE Risk Management Solutions (2015). Figure 4. Sample bowtie analysis.

14 Transit Safety Risk Assessment Methodologies Source: Kishore (2013). Figure 5. Event tree analysis example of a fire alarm and control. Source: Leitner (2017). Figure 6. Fault tree example of hazard scenario for tunnel evacuation failure.

Literature Review 15   is the quantitative risk assessment system software developed by the University of Maryland for the U.S. government. The estimated cost would be “nominal” if usage were granted by the government (GAIN Working Group B, 2003). Failure Mode and Effect Analysis The FMEA methodology is often used in the aircraft, defense, and rail industries to evaluate potential failures within systems. The purpose of FMEA is to identify and evaluate failure modes and probability, as well as to demonstrate compliance with safety requirements. The Hazard Analysis Guidelines for Transit Projects states that FMEA can be “used to identify and analyze possible failures early in the design phase so that appropriate actions are taken to eliminate, minimize, or control safety” (Adduci et al., 2000). FMEA is effective in evaluating both new and existing processes and systems (Centers for Medicare & Medicaid Services, n.d.). There are open-source FMEA worksheets or templates online that can help with performing the analysis (American Society for Quality, n.d.; Dembski, 1998). Figure 7 shows an example of an FMEA. The failure mode and effect criticality analysis (FMECA) methodology is more commonly used to analyze and control hazards. The FMECA examines only hardware and software failures. Another analysis method within the FMEA is the fault hazard analysis (FHA), a method commonly used in the chemical sector. The FHA is a deductive method commonly used quali- tatively, but the process can be expanded to include quantitative analysis. The FHA is one type of analysis within FMEA that examines only effects related to safety. The FHA is used to answer engineering questions about failure like what, how, how frequently, and how severe (GAIN Working Group, 2003). The IEC 60812:2006(E) standard explains the FMEA and FMECA and gives guidance as to how the methods may be applied to achieve various objectives (International Electrotechnical Commission, 2006). Layer of Protection Analysis The layer of protection analysis (LOPA) is a risk assessment tool that uses categories based on orders of magnitude for frequency, severity, and likelihood of failure. The LOPA is often based on information from a qualitative hazard analysis and implemented using a set of rules. This analysis is designed to establish if there are adequate protective layers in the case of an accident scenario, as shown in the example in Figure 8 (Center for Chemical Process Safety, 2001). This Source: Varghese (2016). Figure 7. Sample FMEA analysis.

16 Transit Safety Risk Assessment Methodologies tool offers an analyst the ability to consistently evaluate the risk of a potential event and can be applied when a scenario is too complex to make reasonable risk judgments based purely on qualitative judgment. The LOPA can also be used as a “screening tool prior to a more rigorous quantitative risk assessment” (Center for Chemical Process Safety, 2001). Quantitative Methods Quantitative methods and probabilistic models are understood to be more robust and reliable in most risk evaluation cases. A number of such methods to assess risk are currently in use in industries such as energy, hazardous material transportation, airline and air traffic manage- ment, and rail. Rail Corridor Risk Management System The Rail Corridor Risk Management System (RCRMS) is a quantitative rail route risk assess- ment method, a joint initiative between railroads and government (USDOT), to analyze and identify the safest and most secure routes for transporting hazardous materials (Association of American Railroads, 2020). A software tool enabled by a geographic information system (GIS), RCRMS employs a scoring method, which considers 27 risk factors including network infra- structure, railroad operating characteristics, at-risk population, and environmental and security- related parameters (Vantuono, 2014; FACTOR, 2021). Additional factors beyond the core 27 can inform the decisions but do not directly affect the final scores. Within the management system, the scoring factors fall into two main categories: security and safety-related factors. Safety metrics are additive in nature and security metrics are based on peak values. The RCRMS tool weighs both categories equally. The final score is calculated by multiplying probability by Source: Center for Chemical Process Safety (2001). Figure 8. Layer of protection analysis example by Center for Chemical Process Safety.

Literature Review 17   consequence severity and then calculating a weighted average. The RCRMS method is similar to many other commonly used risk assessment methods in its components of probability, frequency, and consequence severity. In addition to rail traffic routing, index methods like the RCRMS have been used in assessing pipelines as part of an integrity assessment (Muhlbauer, 2004). Slovakian Railroad Researchers at the University of Žilina in Slovakia developed a risk assessment method for Slovakian railroad operators’ risk management systems. The method is based on accident progress scenarios, which are created using fault tree analysis and ETA methods. Risks to personnel, passengers, and the public (expressed as fatalities and weighted injuries [FWI]) are calculated using average frequency and consequence. The frequency of risk is based on historical data. There are instances where quantified historical data are not available; in these cases, human error probability assessments, safety expert judgment, and statistical modeling such as a Monte Carlo simulation are used. The method is similar to the CSM and examines the full life of the system, including the design and construction. As of the publication date in 2017, a railway risk assessment software package employing these methods was still in development (Figure 9). Source: Leitner (2017). Figure 9. Example of Slovakian Railroad risk assessment software.

18 Transit Safety Risk Assessment Methodologies Source: Tsegaye (2019). Figure 10. Example of Monte Carlo simulation results. e purpose of the soware package is to enable the application of these methods and to apply risk management and information management to a rail system. e completed model will also allow for sensitivity testing. Quantitative Risk Assessment and Probabilistic Risk Assessment Quantitative risk assessments (QRAs) are built on actuarial events with logic models to predict the frequencies and causes of events. Like QRAs, probabilistic risk assessments (PRAs) are built on logic models and actuarial data to predict both the frequencies and consequences of possible events. e PRAs are oen accompanied by ETAs, fault tree analyses, and statistical and other soware analyses (GAIN Working Group B, 2003). One such method is the Monte Carlo simulation (see Figure 10). A popular soware tool, @RISK, is used to conduct the Monte Carlo simulation. is tool is used by aircra and airline companies such as the U.S. Air Force, Boeing, Lockheed Martin, and Air New Zealand (GAIN Working Group B, 2003).

Literature Review 19   Fuzzy Reasoning Fuzzy reasoning is a method used most often within the engineering, artificial intelligence, and financial sectors. The application of fuzzy reasoning to rail safety was investigated at the University of Birmingham in England by Yao Chen in his thesis, Improving Railway Safety: Risk Assessment Study, in 2012. Chen (2012) acknowledges the importance and widespread use of the fault tree analysis and ETA but identifies the shortcomings of these methods when data are incomplete or situations are very uncertain. Chen proposes the incorporation of fuzzy reasoning into the railway safety assessment process to better include approximate or difficult-to-quantify information. Fuzzy reasoning can be used to estimate risk levels by apply- ing failure frequency, consequence severity, and consequence probability more accurately. To incorporate fuzzy reasoning into the assessment process, a fuzzy rule base is created using qualitative descriptors such as “remote” or “reasonably likely” (Chen, 2012). This rule base is used to translate qualitative descriptors from words into numbers that can be analyzed quanti- tatively. Chen also proposes the creation of an “expert index” to establish levels of importance for various factors included within the risk assessment.