National Academies Press: OpenBook
« Previous: 1 Introduction
Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×

2

Overview of TARAM

TARAM IN A NUTSHELL

While the elimination of aircraft accidents and serious incidents remains the goal of the Federal Aviation Administration (FAA), it is recognized that the aviation system cannot be completely free of hazards and associated risks. The only absolutely safe aircraft is one that is out of service. Aviation cannot be guaranteed to be free of errors and their consequences; hence risk management is required for transport airplanes to operate successfully within the bounds of the industry and the public’s tolerance for unsafe conditions.

The Transport Airplane Risk Assessment Methodology (TARAM) is a process used by the FAA for calculating a numerical value for risk associated with transport airplanes whenever continued operational safety (COS) issues occur in the fleet.1 The TARAM process can be triggered by a variety of safety issues, such as an accident or incident, a quality escape of a manufactured component, or an anomaly discovered during maintenance. TARAM is important because its risk-analysis calculations are used to make determinations of unsafe conditions in transport airplanes so that corrective actions can be identified and implemented for lowering the risk.

The TARAM process is a subset of a much broader FAA process used for all types of aircraft2 and is known as the Monitor Safety/Analyze Data (MSAD) process. As defined in FAA Order 8110.107A,3 the MSAD process (see Figure 2.1) is designed to promote an improved COS methodology by incorporating a data-driven, risk-based approach for safety assurance and safety risk management.

The MSAD process requires FAA aviation safety engineers (ASEs) to filter, review, analyze, and trend aviation safety data in order to identify safety issues that occur in the in-service aircraft fleets, and, more importantly, identify corrective actions to mitigate safety risks across the fleet. MSAD uses a standard taxonomy for organizing COS data and promotes quick identification of emerging safety trends. In addition, MSAD establishes a causal analysis approach.

After event data are acquired, the events are filtered using hazard criteria. For the reported events that are identified as potential safety issues based on the hazard criteria, the FAA performs a “preliminary” risk assessment that is qualitative in nature to determine (1) if the safety issue is urgent, and, thus, an emergency airworthiness

___________________

1 Transport airplanes are those used in the commercial airline industry and also by business jet operators.

2 The MSAD process is used not only for transport airplanes but for any type of aircraft certified by the FAA, including helicopters, small recreational general aviation airplanes, gliders, etc.

3 Federal Aviation Administration, 2012, “Monitor Safety/Analyze Data (MSAD),” Order 8110.107A, Washington, DC: Aircraft Certification Service (AIR).

Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×
Image
FIGURE 2.1 The Monitor Safety/Analyze Data (MSAD) process. NOTE: AD: airworthiness directive, AIR: Aircraft Certification Service, ASE: aviation safety engineer, AVS: aviation safety, COS: continued operational safety, MSAD: Monitor Safety/Analyze Data. SOURCE: FAA Order 8110.107A.

directive (AD) or immediately adopted rule is required; and (2) if the reported event indicates that the potential safety issue requires further investigation through the MSAD process. If an unsafe condition is discovered from this rapid qualitative review, and if it is determined to be urgent, the agency will take initial action and follow up later with a full analysis. If urgent action is not required but the necessity for further investigation is indicated, the FAA will pursue a formal, more quantitative risk assessment via the TARAM, based on potential hazards, fleet age, fleet utilization, statistical distribution of failures, and historic outcomes. The TARAM results are used to support two layers of decision-making in the MSAD process: (1) to determine whether the condition under study requires ADs or other mandatory corrective actions; and (2) to select and prioritize corrective actions.

The current version of the TARAM has been used by the FAA for over a decade. The TARAM methodology is also used by most transport airplane manufacturers, in whole or in part, by agreement with the applicable FAA aircraft certification office.

TARAM has evolved significantly from lessons learned over the past decade, and it has also been reduced in scope to be compatible with, and support, the MSAD process. TARAM is only one tool in the FAA’s toolbox for determining whether a condition found in the transport airplane fleet is safe or unsafe, and for selecting the appropriate corrective action for an unsafe condition. However, while the results of TARAM are used to guide the FAA for corrective action, a decision to ground an airplane can only be made by the Administrator, who has a great deal of flexibility and latitude when making safety determinations. No specific thresholds are provided to

Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×

ASEs for determining whether an airplane should be grounded, nor should they be provided. The result from a TARAM analysis is not intended to be the sole basis for determining unsafe conditions, nor does it limit, in any way, the Administrator’s prerogative to make such determinations.

While TARAM is only one facet of the safety decision-making process, it is an important one. Because of this, it is critical that the Administrator and senior staff are made aware of, and understand, the TARAM results from the analysis of a potential unsafe condition that has significant consequences for transport category airplanes. By maximizing the confidence and relevance of TARAM results, the Administrator will be in a better position to make well-informed determinations to improve commercial airplane safety. The FAA is aware of this challenge, and pointed out at meetings with the committee that they would welcome suggestions by the committee and/or any other entity that could provide additional insights to improve TARAM and the use of its results.

Including a systematic risk assessment methodology in the continued operational safety analysis is important for a comprehensive approach to the overall safety of the transport airplane fleet. TARAM is an initial attempt by the FAA to fill such a role. A healthy safety culture requires commitment to continuous improvement. As discussed in detail in Chapters 3 to 6 of this report, recommendations are provided to the FAA to address the gaps and strengthen the TARAM.

Guidance for TARAM is contained in two formal FAA documents: the TARAM Handbook issued in 2011, and the Seattle ACO Transport Airplane Safety Manual issued in 2021. These documents, and their associated set of training slides, are intended to guide FAA ASEs who may perform or oversee risk analysis for transport airplanes as part of the MSAD process cited in FAA Order 8110.107. The TARAM Handbook was issued on November 4, 2011, by the FAA Aircraft Certification Service. The body of the handbook is 38 pages in length with an additional 13 pages of definitions and examples contained in three appendixes. The Seattle ACO Transport Airplane Safety Manual was issued September 1, 2021, by the Seattle Aircraft Certification Office. The main body of this manual is 59 pages in length with an additional 45 pages of definitions, guidance, and examples contained in six appendixes.

The remainder of this chapter provides an overview of TARAM in support of COS decision-makings based on the TARAM Handbook and the Seattle ACO Transport Airplane Safety Manual. In the following subsections, three key aspects are summarized including (i) current TARAM analysis process (with additional details in Appendix A), (ii) input data for TARAM, and (iii) the use of TARAM results in the COS decision-making.

CURRENT TARAM PROCESS

A universal definition for risk does not exist in the federal government. Federal agencies that are responsible for public safety, such as the FAA, the Environmental Protection Agency, the U.S. Nuclear Regulatory Commission, and the Department of Homeland Security (DHS), each have their own definitions. For example, DHS defines risk as the expected loss characterized as the product of threat, vulnerability, and consequences.

As described in the TARAM Handbook, TARAM uses two types of risks: fleet risk and individual risk. Fleet risk is defined as either the expected number of fatal events (accidents involving passengers or ground fatalities) or the expected number of fatalities in a given time period. Individual risk in TARAM is typically measured as the rate of fatal injuries per flight-hour and is used in cases where the fleet risk is calculated to be acceptable due to low fleet exposure or severity, but the risks to individuals flying in high-risk airplanes is not acceptable (Table 1 of the TARAM Handbook).

To assess risk, TARAM calculates both the total uncorrected fleet risk (R(UFleet)) and the individual uncorrected risk (RI(U)) prior to any corrective action to evaluate whether a proposed corrective action is warranted. These risks are evaluated throughout the remaining lifetime of the fleet (i.e., the period over which the total of existing and future airplanes in the fleet will operate) and provide insights about the risk from both the operator (fleet risk) and the end user (individual risk).

The total uncorrected fleet risk represents the expected number of adverse events during the remaining lifetime of the fleet if no corrective action is taken. An adverse event represents an airplane accident (because of the condition under consideration) causing at least one fatality. The uncorrected individual risk is defined as the expected fatal injuries per flight hour during future flights if no corrective action is taken.

Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×

If, based on these risks and possibly other considerations, it is established that a corrective action is needed, then TARAM prescribes to calculate three additional risk metrics: (i) 90-day fleet risk that provides a short-term forecast and helps determine the urgency of the corrective action; (ii) Total fleet risk during the control program; and (iii) Individual risks during the control program. The control program is the period when the corrective action is being accomplished.

These risks measure the acceptability of the corrective action and its duration. Unlike the initially calculated fleet risk that represents the expected number of events that will result in fatalities during the remaining lifetime of the fleet, the 90-day and the control program fleet risks now quantify the expected number of fatalities resulting from such events during the 90-day or the prescribed control program periods of time, respectively.

Figure 2.2 provides a simplified flowchart of the current TARAM process to calculate these risk measures.

The TARAM analysis process begins with understanding and developing the causal chain that could lead from the condition under study to unsafe outcomes. The causal chain establishes a basis for formulating the risk measures in terms of multiple events in the progression of the condition under study up to the unsafe outcomes. The general formula to calculate fleet risk is provided as:

RFleet = E(# of occurrences) * P(unsafe outcomes | occurrence) * Severity [2.1]

The first term represents the expected number of occurrences related to the initial event or condition under consideration during the total remaining lifetime of the affected aircrafts and is calculated based on (i) the rate of occurrence of the condition under study computed by either the constant failure rate or an increasing failure rate denoted as wear-out failure4 in Figure 2.2 and (ii) fleet utilization, remaining fleet life, and the number of airplanes in the affected fleet during the remaining fleet life determined in the exposure factors analysis in Figure 2.2.

The second term of Equation 2.1 represents the conditional probability (CP) of unsafe outcomes given the occurrence of the initial event. This term is referred to as CP throughout the handbook and is one of the elements of the “Determine Outcome Factors” in Figure 2.2. This accounts for cascades of smaller failures leading to a full-blown disaster. The calculation of the CP requires enumeration of all possible unsafe conditions and their causes and is treated in TARAM as a causal chain analysis.

The third term in Equation 2.1, the severity of the potential unsafe outcomes, is determined as part of the outcome factors analysis in Figure 2.2. This term is measured differently depending on the risk being calculated. For the total uncorrected fleet risk, severity is measured as the probability of a fatality given exposure to the unsafe condition(s) and is calculated by computing the fraction of the number of fatalities over the number of exposed occupants (EOs), which is called the injury ratio (IR).5 In contrast, for the 90-day or control program risk assessment, the fleet risks measure the severity as the expected number of fatalities and are calculated as the product of the IR and the EO.

A similar formula is used to calculate individual risk:

RI = F * P(Unsafe outcomes | occurrence) * Severity [2.2]

The first term “F” represents rate of the occurrence of the condition under study, obtained by either the constant failure rate or wear-out failure rate analysis in Figure 2.2. The second term is the CP, and the severity is measured as the probability of a fatality given exposure to the unsafe condition(s), which is obtained as part of the outcome factors determination in Figure 2.2. The initial prescription for the individual risk calculation is to consider all expected outcomes, but then the TARAM Handbook vaguely recommends that when there is a significant variation between flights, to only consider the worst reasonable expected outcomes.

Various Aircraft Certification Offices (ACOs) are using TARAM worksheets to conduct the risk analysis, as recommneded by the TARAM Handbook. The worksheets are templates that contain each major step needed to

___________________

4 See Appendix A for more details about “constant failure rate” and “wear-out failure” analyses.

5 Current practice of TARAM on occasions considers IRs larger than one that do not represent a probability by including fatalities from individuals not considered as exposed occupants (e.g., personnel in the landing area).

Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×
Image
FIGURE 2.2 Simplified flow chart representing the TARAM process. NOTE: CARB: Corrective Action Review Board, MSAD: Monitor Safety/Analyze Data. SOURCE: FAA TARAM Handbook.

determine the risk values. They serve as a guideline for the analysis and provide a means to record the analysis and assumptions. This is noted in the TARAM Handbook that the FAA reviewed with the committee in the “FAA TARAM Training” new PowerPoint presentation.6 The worksheets can be found in Appendix B of the TARAM Handbook. Meanwhile, other ACOs use another methodology referred to as TARA (Transport Airplane Risk Analysis). The Seattle ACO Branch has developed three spreadsheets that cover analysis prescribed in TARA, the constant and wear-out failure rate analyses, and analysis to determine the risk related to maintenance and operational personnel. The spreadsheets can be found in Appendix B of the Seattle ACO Branch Transport Airplane Safety Manual. The FAA could codify these methodologies to be consistant across all ACOs.

The committee acknowledges that the following concepts and approaches in the current TARAM process are reasonable:

  • Definitions of the TARAM risk metrics (Table 1 of the TARAM Handbook);
  • The use of the causal chain for airplane-level events, and the parametrization of each term (i.e., expressing the risk as a product of multiple parameters associated with the causal chain events);
  • Considering two types of models, constant failure rate and wear-out processes, for quantifying the frequency of a condition under study.

Chapter 5 of this report provides the findings associated with gaps that have been identified by the committee in the TARAM analysis process and offers recommendations for improvements.

___________________

6 F. Keller, Aerospace Engineer and TARAM Subject-Matter Expert, FAA (retired), 2021, “TARAM Training” (two sets of presentations), Presentation to the Committee on Transport Airplane Risk Assessment Methodology, October 8.

Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×

INPUT DATA TO TARAM

Estimation and determination of the TARAM inputs, namely the input variables of the risk equations shown in Equations 2.1 and 2.2 above, require input data. Input data for TARAM is collected from various sources. Several of these sources are accessed through the Aviation Safety Information Analysis and Sharing (ASIAS) system.7 In 2007, the FAA and industry launched the ASIAS program, a collaborative safety analysis and information sharing initiative that aids in the monitoring and identification of potential safety issues to proactively detect risks and implement mitigation strategies before accidents and incidents occur. The ASIAS program includes 104 data sources from both public (non-confidential) and protected proprietary (confidential) data systems. Non-confidential databases maintained in ASIAS provide fleet size, usage, and fleet life-related variables; occurrence or detection of failures, malfunctions, and defects from the FAA Service Difficulties Reporting System (SDRS); and accident and incident data from the National Transportation Safety Board (NTSB) Aviation Accident Database and the FAA Accident and Incident Database System (AIDS). Confidential sources include data from aircraft operators extracted from aircraft recorders and textual voluntary safety reports submitted by flight crews.

The disadvantages of the ASIAS database are that it does not include worldwide operational data, and, in some cases, it can take months for requested data, submitted to The MITRE Corporation, which currently operates the ASIAS platform for the FAA, to be delivered. The FAA has not yet established a robust process for prioritizing analysis requests. Also, while the agency plans to make incremental enhancements to ASIAS, it does not expect to fully integrate predictive capabilities until 2025. In addition, while the FAA provides some ASIAS information to aviation safety inspectors, the agency does not provide access to national trend information that could improve their safety oversight. Voluntarily provided safety data play a pivotal role in enabling the transition from a forensic approach to managing safety to a more prognostic and predictive approach, but it is critical to establish trust, protections, and protocols on the use of the data, which takes time to develop and foster across the stakeholder communities. However, the FAA recently stated that, by June 30, 2022, it plans to develop and implement models based on criteria to prioritize requests for ASIAS safety information.8

Additional reports of failures, malfunctions, and defects in products, parts, processes, or articles manufactured are accessed through the 14 CFR Part 21.3 Certification Procedures for Products and Parts required reports database.9 Design approval holders may provide additional data when requested. Underlying data may be made available to the FAA depending on approval holder documentation practices, and this will be at the discretion of the design approval holder. The FAA TARAM policy states: “Affected design-approval holders should know, in general, the data and information that could be requested from them when aircraft certification offices are analyzing the risk associated with continued-operational-safety issues.”10

Historical injury ratios11 for a variety of conditions and outcomes used for injury ratio calculations are from a data set developed by the FAA Aircraft Certification Service utilizing the NTSB Aviation Accident and the FAA AIDS.12

Current TARAM does not adequately characterize the uncertainty associated with the TARAM input data. Chapter 4 of this report provides the analysis and findings associated with the gaps that the committee identified regarding the input data to TARAM.

___________________

7 Federal Aviation Administration, 2011, Transport Airplane Risk Assessment Methodology (TARAM) Handbook, PS-ANM-25-05, Washington, DC: Transport Airplane Directorate ANM-100, pp. 16–17, 26.

8 Federal Aviation Administration, 2021, FAA Has Made Progress in Implementing ASIAS, But Work Remains to Better Predict, Prioritize, and Communicate Safety Risks, Report No. AV2021022, Washington, DC: Department of Transportation Office of Inspector General.

9 F. Keller, Aerospace Engineer and TARAM Subject-Matter Expert, FAA (retired), Written Answers to Committee on Transport Airplane Risk Assessment Methodology Questions, November–December 2021 and January–February 2022, Washington, DC: National Academies of Sciences, Engineering, and Medicine.

10 Federal Aviation Administration, 2011, “Policy Statement: Risk Assessment Methodology for Transport Category Airplanes,” PS-ANM-25-05, Washington, DC: Transport Airplane Directorate ANM-117, p. 2.

11 For the total uncorrected fleet risk, severity is measured as the probability of a fatality given exposure to the unsafe condition(s) and is calculated by computing the fraction of the number of fatalities over the number of exposed occupants (EOs), which is called the injury ratio (IR).

12 Federal Aviation Administration, 2011, Transport Airplane Risk Assessment Methodology (TARAM) Handbook, PS-ANM-25-05, Washington, DC: Transport Airplane Directorate ANM-100, pp. 16–17, 26.

Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×

USE OF THE TARAM RESULTS IN COS DECISION-MAKING

Generally, the current FAA process for considering and approving corrective action utilizes subject-matter expert (SME) opinions to characterize the probabilities and consequences of potential risks. FAA personnel are aware of this issue, pointing out to the committee the need for objective data and revised analytical approaches.

Immediately following a reported incident or accident, the FAA performs a “preliminary” risk assessment that is qualitative in nature to determine if a potential safety issue exists. If an unsafe condition is discovered from this rapid qualitative review, and if it is determined to be urgent, the agency will take initial action and follow up later with a full analysis. If urgent action is not required, the FAA will pursue a formal, more quantitative risk assessment via TARAM, based on potential hazards, fleet age, fleet utilization, statistical distribution of failures, and historical outcomes. The TARAM results are then evaluated against the FAA’s predefined risk guidelines. The formal risk assessment forms the basis of the FAA’s unsafe condition determination and appropriate reaction times necessary for corrective action.

Whether the proposed corrective action should be made mandatory with the issuance of an AD is decided by convening a formal Corrective Action Review Board (CARB).13 The following actions will result following a CARB:

  • Emergency AD—Unacceptably high risk requires immediate action to resolve.
  • Immediate Adopted Rule AD—Actions required do not rise to level emergency AD, but do not afford the opportunity to seek public comment before the effective date. Comments are requested when published and addressed after the effective date of the AD.
  • Final Rule following Notice of Proposed Rulemaking (NPRM)—The FAA seeks public comment to potential rule by issuing a notice of proposed rulemaking, or NPRM. It is inherently a slower, more deliberative process for issues where compliance times and risk levels allow.
  • No AD—No unsafe condition. Safety risks do not rise to the level of mandatory actions.

TARAM Handbook provides guidance for determining risk levels, and presents a table (see Table 2.1) with values that represent a range of risk that may require corrective action. The handbook asserts that the values were correlated, in general, with those used during an extended period of COS program testing in certain branches of the FAA. The results of the FAA’s testing reportedly indicate that the risk results align well with safety decisions made by those branches. The handbook states: “This alignment with ongoing, continuing, operational-safety programs show the risk-level guidance presented here to be generally consistent with the historic level of safety maintained by the transport-airplane AD process.”

The FAA confirmed that the air carrier and general accident rates cited in the TARAM risk levels are based on NTSB accident data from 2002 through 2006.

The handbook further indicates that the guidance is based on the average risk of individual fatal injury per flight hour, experienced by passengers on transport airplanes operated within the United States, which is on the order of 10–8/flight hour. It also states: “Current uncorrected fleet risk guideline is a very general estimate of safety during this period.” The TARAM Handbook has not been updated in the past decade to reflect the current state of the commercial airplane design and operations, and engineering justifications for current risk thresholds identified in TARAM risk guidance table. The FAA is aware of this, pointing out to the committee that efforts were begun in 2015 to update the order, but any suggestions for improvement have not yet been implemented. The committee also learned during its interactions with the FAA that the agency now has only one SME for TARAM, following the recent retirement of the ASE who first developed the process. As a result, TARAM is not broadly communicated or frequently taught.

Recommendation 2 in Chapter 3 addresses that the FAA needs to formally designate multiple employees within its organization as experts for the TARAM process and that these experts need to be responsible for the

___________________

13 The FAA’s AD process is governed by the Administrative Procedures Act, which requires the agency to seek public input to rules prior to enacting them; however, the act makes exceptions for urgent safety issues.

Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×

TABLE 2.1 Excerpt from the Risk Guidelines Table in the FAA TARAM Handbook

Safety Decision-Making Priority Risk Control Decision-Making
Total Uncorrected Fleet Risk Uncorrected Individual Risk 90-Day Fleet Risk Control-Program Fleet Risk Control-Program Individual Risk, Urgent Action May Be Necessary Control-Program Individual Risk, Not Airworthy
>.02 or .04 >10−7/flight-hour N/A >3 >10−6/flight-hour >10−5/flight-hour
Guidance: Use .02 guidance for transport-airplane types and fleets primarily used in commercial passenger operations. Use .04 for other types of operations. Guidance: Use the uncorrected individual-risk-level guidance when risk variables, such as fleet size, fleet age, or exposed-occupant count, result in acceptable total uncorrected fleet risk, but the individual per-flight-hour risk is unacceptable. Guidance: Using the 90-day fleet risk factor as a priority measure in comparison to other pending and envisioned corrective actions. Guidance: Corrective action is required as soon as reasonably practical within the time period associated with the control-program fleet risk-level guidance. The risk-level guidance represents the maximum acceptable risk and is not to be used as a target value. Guidance: Minimize, to the extent practicable, commercial-passenger service operations at individual risk levels above this level. Guidance: Transport airplanes should not operate in commercial-passenger service above this level for any period of time.
For “single-failure” issues, see paragraph 6.1 for guidance.

SOURCE: Federal Aviation Administration, 2011, Transport Airplane Risk Assessment Methodology (TARAM) Handbook, Transport Airplane Directorate ANM-100, November 4, Table 3.

maintenance of TARAM processes, including updating the TARAM Handbook to reflect, among other things, current NTSB accident rates.

The TARAM Handbook on page 5 states that decision-making is not just based on risk, and that there are “other criteria” cited in TARAM guidance and FAA Order 8110.107 that should be used in addition to TARAM results. However, these criteria are discussed only in very general terms. The TARAM Handbook does not provide guidance with regard to FAA decision-making for corrective action, except very generally in the preface.

The Seattle ACO Transport Airplane Safety Manual provides additional guidance on the consideration of the “other criteria” in the COS decision-making in addition to the TARAM risk values. The Seattle ACO Transport Airplane Safety Manual contains a section titled “Economic Risk” on page 9 that states:

In general, the FAA does not directly assess the risk or degree of economic loss due to potential safety issues. However, the FAA internally sometimes considers potential consequences from “loss of public confidence” in the industry and/or the FAA. To assist the CARB in making judgments about issues where loss of public confidence or other economic concerns may be a factor, the Seattle ACO Branch staff will calculate and present the number (or fraction) of fatal events anticipated in the remaining life of the affected fleet. For certain high-visibility events, guideline unsafe condition thresholds for the number (or fraction) of anticipated fatal events that are consistent with the intent of the TARAM “weighted events” guidelines are provided in this Manual; see Section 3.2.

The Safety Manual also contains a section titled “Criteria-Specific Guidance” (page 42) that states:

The “other” criterion is included as a wild card provision to allow the presenting staff to propose that, based on engineering judgment, an unsafe condition exists in a case where none of the other criteria are met. It is also intended

Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×

to allow the decision making board to make a similar decision. When this criterion is used, the justification for considering the issue to be an unsafe condition and supporting rationale must be documented. Some examples of issues that might fall in this category include … issues involving excessive crew workload, particularly in phases of flight where a high workload already exists.

Section 3 of the Seattle ACO Transport Airplane Safety Manual describes further details on criteria for determining whether a condition under study is unsafe by accounting for quantitative risk guidelines based on TARAM as well as for qualitative factors and special considerations (i.e., high visibility events, accident lessons learned, fail-safe design, and risk to maintenance or operations crew). This manual, however, does not provide guidance as to when and how the TARAM results are combined with (or substituted for) other safety principles in the FAA’s decision-making for determining the urgency and priority of corrective actions. Although the Seattle ACO Transport Airplane Safety Manual states that “Numerical risk assessment (TARAM) and CARB judgment will determine the urgency and priority for each issue that CARB determines requires corrective action,” there is no further explanation, and in the current COS decision-making practice, the AD prioritization and the determination of control program times are solely based on the 90-day fleet risk and the Outer Marker Times14 that are computed using TARAM.

With regard to the consideration of uncertainty, the TARAM Handbook, Seattle ACO Transport Airplane Safety Manual, and training materials are mostly silent. The guidance considers only the point estimate of risk with no consideration of uncertainty bounds (or confidence intervals) for comparison with risk thresholds. According to the committee’s interactions with the TARAM SME, the CARB, as stipulated in FAA Order 8110.107, would be briefed by the FAA analyst of any uncertainties in theory. The only mention of uncertainty in the Seattle ACO Transport Airplane Safety Manual can be found in Appendix D, which states, “The choice of initial condition is made on a case-by-case basis, considering the information available and which condition has the least amount of uncertainty in the data.” Based on the FAA briefing regarding the Seattle ACO Transport Airplane Safety Manual, in practice, sensitivity analyses are sometimes conducted to study the impact of different assumptions and input values, and the sensitivity results would be presented to the CARB.

It is acknowledged that the following concepts and approaches for the current use of TARAM results in the COS decision-making process are deemed reasonable:

  • The structure of the MSAD process flow (Figure 2 of FAA Order 8110.107A).
  • The overall framework of the decision-making practice of the Seattle ACO (Chapters 3 and 4 in the Seattle ACO Safety Manual).

In Chapter 6, findings and recommendations related to the use of TARAM results in the COS decision-making process for transport airplanes are provided. The findings and recommendations are focused on the selected areas of the COS decision-making process using TARAM results, for which the Committee identified the gaps and needs for improvements. More specifically, Chapter 6 addresses the recommendations for improving some of the individual steps of the current COS decision-making process so that the key elements, such as uncertainty, risk importance ranking and sensitivity analysis, and potential for a single failure, are addressed quantitatively to the extent practicable, rather than solely relying on qualitative expert judgment. Chapter 6 also provides recommendations to enhance the quality of the implementation of the COS decision-making process, such as creating national-level documentation of the decision-making criteria (currently documented in the Seattle ACO manual), a peer-review process, a structured training system for TARAM analysts, and a continuous research and development program for TARAM at the FAA.

___________________

14 Defined as the “time until the control program risk guideline is reached” assuming corrective action is not taken in FAA Order 8110.107A.

Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×
Page 10
Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×
Page 11
Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×
Page 12
Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×
Page 13
Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×
Page 14
Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×
Page 15
Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×
Page 16
Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×
Page 17
Suggested Citation:"2 Overview of TARAM." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Transport Airplane Risk Assessment Methodology. Washington, DC: The National Academies Press. doi: 10.17226/26519.
×
Page 18
Next: 3 Role of TARAM Within the FAA's Overall Safety Oversight System »
Evaluation of the Transport Airplane Risk Assessment Methodology Get This Book
×
Buy Paperback | $23.00 Buy Ebook | $18.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Transport Airplane Risk Assessment Methodology (TARAM) is a process for calculating risk associated with continued operational safety issues in the U.S. transport airplane fleet. TARAM is important because its risk-analysis calculations are used when making determinations of unsafe conditions in transport airplanes and when selecting and implementing corrective actions. This report assesses the TARAM process used by the FAA in its efforts to improve the overall safety of the transport airplane fleet. A healthy safety culture requires commitment to continuous improvement. This report provides recommendations to the FAA to address the gaps and strengthen the TARAM.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!