1

Introduction

Decades of continuous efforts to address known hazards in the national airspace system (NAS)¹ and to respond to issues illuminated by analysis of incidents and accidents have made commercial airlines the safest mode of transportation. The task of maintaining a high level of safety for commercial airlines is complicated by the dynamic nature of the NAS. The number of flights by commercial transports² is increasing; air traffic control systems and procedures are being modernized to increase the capacity and efficiency of the NAS; increasingly autonomous systems³ are being developed for aircraft and ground systems, and small aircraft—most notably unmanned aircraft systems (UAS)⁴—are becoming much more prevalent. As the NAS evolves to accommodate these changes, aviation safety programs will also need to evolve to ensure that changes to the NAS do not inadvertently introduce new risks.

Maintaining the safety of the NAS as it evolves will require a wide range of safety systems and practices, many of which are already in place. This report focuses on an aviation safety system that could detect and mitigate high-priority safety issues as they emerge and before they become hazards. In particular, the report defines the challenges to establishing such a system and the high-priority research projects that should be implemented to expedite its development.

___________________

¹ The NAS is “the common network of U.S. airspace; air navigation facilities, equipment, and services; airports or landing areas; aeronautical charts, information and services; rules, regulations, and procedures; technical information; and manpower and material.” FAA, 2013, Integration of Civil Unmanned Aircraft Systems [UAS] in the National Airspace System [NAS] Roadmap, Washington, D.C., https://www.faa.gov/uas/media/uas_roadmap_2013.pdf. Some NAS facilities are jointly operated by the FAA and the Department of Defense. The NAS includes all aircraft operating in U.S. airspace, both foreign owned and domestic.

² In this report, “commercial transports” refers to aircraft operated by regional and major passenger airlines as well as cargo airlines.

³ Increasingly, autonomous systems lie along the spectrum of system capabilities that begin with the abilities of current automatic systems, such as autopilots and remotely piloted (nonautonomous) unmanned aircraft, and progress toward highly sophisticated systems that would enable, for example, UAS that could operate independently within civil airspace, interacting with air traffic controllers and other pilots just as if a human pilot were on board and in command (National Research Council, 2014, Autonomy Research for Civil Aviation: Toward a New Era of Flight, The National Academies Press, Washington, D.C.).

⁴ An “unmanned aircraft” is, as the name implies, an aircraft that has no onboard pilot. In this report unmanned aircraft are assumed to have no humans on board either as flight crew or as passengers. A UAS is an unmanned aircraft and its associated elements, including ground control and communications equipment.

A REAL-TIME AVIATION SAFETY ASSURANCE SYSTEM

Real-time system-wide safety assurance (RSSA) is one of six focus areas for NASA’s aeronautics program. NASA envisions that an RSSA system (see Figure 1.1) would provide a continuum of information, analysis, and assessment that supports awareness and action to mitigate risks to safety. NASA’s research plans state that development of an RSSA system will necessitate automating safety assurance of air transportation system components, integrating component-level systems, and reducing the safety assurance cycle time until real-time safety assurance is achieved at the system-of-systems level. The safety assurance system envisioned by NASA will combine air traffic and onboard aircraft technologies as well as air traffic system automated data mining capabilities into a system for continuous safety monitoring and threat prediction.⁵ This system is expected to maintain or exceed the current level of air traffic safety⁶ while accommodating global increases in air travel and rapid introduction of new technologies.⁷ The system would not be expected to directly address issues related to design, development, training, or maintenance because the detection of problems in these areas and the process of implementing corrective actions falls outside the short time horizon of an RSSA. Other systems are already in place to address these aspects of aviation safety. The outer feedback loop in Figure 1.1 would allow for integrated development over time as the system guides the development of improvements in the capabilities for monitoring, assessing, and mitigating safety risks.

___________________

⁵ NASA, 2015, NASA Technology Roadmaps TA 15: Aeronautics, NASA, Washington, D.C., https://www.nasa.gov/sites/default/files/atoms/files/2015_nasa_technology_roadmaps_ta_15_aeronautics_final.pdf.

⁶ Throughout this report, “air traffic safety” generally refers to the safety of aircraft both in the air and on the ground.

⁷ NASA, 2015, NASA Technology Roadmaps TA 15: Aeronautics.

NASA envisions that the process of developing a comprehensive and fully functional RSSA system would include three intermediate milestones:⁸

• Domain-Specific⁹ (Real-time) Safety Monitoring and Alerting Tools (2015-2025). Expanded system awareness through increased access to safety-relevant data and initial integration of analysis capabilities; improved safety through initial real-time detection and alerting of hazards at the domain level and decision support for limited, simple operations.¹⁰
• Integrated Predictive Technologies with Domain-Level Application (2025-2035). NAS-wide availability of more fully integrated real-time detection and alerting for enhanced risk assessment and support of initial assured human and machine decision support for mitigation response selection for more complex operations.
• Adaptive Real-time Safety Threat Management (2035-2045). Fully integrated threat detection and assessment that support trusted methods for dynamic, multi-agent planning, evaluation, and execution of real-time risk mitigating response to hazardous events.¹¹

These milestones provide a reasonable estimate of the long time that it would take to develop a fully functional RSSA.

IN-TIME AVIATION SAFETY MANAGEMENT SYSTEM (IASMS)

One potentially confusing facet of NASA’s RSSA research lies in the descriptor “real-time.” A common understanding of real time is that it describes events that occur at the same time or nearly so. For example, the operator of a remotely piloted vehicle is controlling the aircraft in real time, and if the aircraft is equipped with a camera and video link, the operator can monitor the area around the aircraft in real time. Some elements of an RSSA system would occur in real time, just as the Traffic Collision Avoidance System (TCAS) operates in real time to continuously monitor an aircraft’s position and velocity vector relative to the terrain and to other aircraft to immediately alert pilots when the risk of a collision exceeds a programmed threshold. Other elements of an RSSA system, however, could operate over a period of minutes, hours, or even days to look at operational trends over these time scales to identify risks that cannot be identified in real time. This report therefore refers primarily to an IASMS instead of an RSSA to avoid potential misunderstandings regarding the temporal scope of the safety system addressed herein.

The Federal Aviation Administration (FAA) already requires that U.S. commercial airlines develop and implement a safety management system (SMS) (see Box 1.1), which will address operational safety risks associated with air travel. The FAA’s SMS requirements do not specify a time frame over which those risks arise, are identified, and are mitigated. An IASMS, on the other hand, would continuously monitor the NAS to rapidly assess and mitigate safety risks.

The committee’s vision for an IASMS is as follows:

1. An IASMS will continuously monitor the NAS to collect data on the status of aircraft, air traffic management (ATM) systems, airports, weather, and so on, and then assess that data, as follows:

___________________

⁸ Briefing by J. Nowinski at the first meeting of the Aviation Safety Assurance Committee, January 23, 2016, Washington, D.C., p. 6.

⁹ One example of a “domain-specific tool” would be a tool that monitors the NAS and determines the system state as it applies to a specific class of aircraft operating in the airspace near a specific airport. After such a tool is demonstrated and validated at that airport, continued development could expand its applicability to other classes of aircraft and other airports.

¹⁰ In the context of this milestone, “limited, simple operations” are intended by NASA to refer to, for example, tools with limited automated decision-making and mitigation capabilities that could be demonstrated and validated in low-risk operations, such as those involving small UAS operating in unpopulated areas.

¹¹ In the context of this milestone, “hazardous events” also refers to hazardous trends and conditions.

1. Assess data on a second-by-second, minute-by-minute, and hour-by-hour basis to detect or predict elevated risk states based on rapid changes in system status. (Different elements of a safety assurance system will operate on different time scales.) Data of interest include the status and performance of vehicle systems, ground systems, operators, and weather. However, the system would not be designed to predict or respond to emergencies caused by catastrophic equipment failures, such as an uncontained engine failure or a landing gear collapse.
2. Assess data over periods of days to detect risks based on longer-term trends.¹²
3. Detect and predict elevated risk states that arise from a confluence of factors, none of which by itself would be noteworthy.

___________________

¹² The limited time horizon of an IASMS will allow it to complement rather than duplicate the efforts of other aviation safety systems such as those addressed in the Safety Data section that follows.

2. 4. Assess data in the context of a thorough understanding of (1) the nominal performance of systems and operators, (2) historical data regarding both the occurrence and consequences of off-nominal situations, and (3) the fault tolerance of the NAS and its key elements.
   5. Assess system outputs over long periods of time to identify emergent risks that in some cases should be added to the list of risks that the system is designed to check for.
3. An IASMS will be focused on risks that require safety assurance action in-flight or prior to flight. Preflight safety assurance action may include a decision to postpone or cancel a flight until, for example, flight conditions change or equipment is repaired. An IASMS will not be designed to recommend safety assurance actions that would occur over a period of weeks, months, or longer, such as changes to pilot training programs, operational procedures, equipment design, or the content of scheduled maintenance checks.¹³ The output of an IASMS, however, may be useful to those who are responsible for these longer-term areas of interest.
4. Safety assurance actions generated by an IASMS may take the form of recommendations that operators take action. In some cases when urgent action is required, IASMS may be designed to initiate safety assurance actions on their own.

The preceding description of the committee’s vision does not specify the use of any particular programmatic approach for achieving the vision. Chapter 5, however, notes that an approach with interim deliverables would facilitate development of a consensus in the aviation community to support IASMS research. The approach envisioned by NASA for development of an RSSA is structured to provide such deliverables.

The NAS includes many different classes of airspace, each of which has specific requirements for various types of aircraft and operations. Flight operations in most airspace must take place under the direction of air traffic controllers. Accordingly, the FAA’s automation and surveillance capabilities are focused on controlled airspace rather than uncontrolled airspace, which predominantly exists at very low altitudes (less than 400 ft). As noted in Chapter 2, the UAS traffic management (UTM) system is being developed to facilitate operations of UAS. It remains to be seen how a future UTM system will be designed, operated, and integrated into the existing ATM system. It also remains to be seen what aircraft types and what types of operations in different classes of airspace will be encompassed by an IASMS; that will be determined as the concept of operations (CONOPS) for an IASMS is developed (see Chapter 2).

Figure 1.1 is consistent with this report’s vision for an IASMS, although the outer feedback loop would have the additional task of identifying emergent risks. As with a conventional safety SMS, the tasks in the outer feedback loop would operate on long time scales of months or longer. For example, it will take some time to develop, validate, and incorporate improved models into an IASMS.

SAFETY DATA

The NAS includes a wide variety of aircraft, including commercial transports, general aviation aircraft, rotor-craft, military aircraft, and UAS. While the aviation community strives to ensure the safety of all aircraft operations, there is a particular emphasis on commercial airlines given that the number of flights by commercial transports—and the number of passengers and flight crew aboard those flights—far exceeds those for all other types of aircraft.¹⁴ The impressive safety of U.S. commercial airlines is due in part to the fact that the aviation industry and federal government are voluntarily investing in the right safety enhancements to reduce the fatality risk of travel by commercial airlines in the United States. There is a long-standing safety culture in aviation that is currently supplemented

___________________

¹³ Although scheduled maintenance checks would be outside the temporal scope of an IASMS, it could recommend conducting maintenance on a particular aircraft system prior to the aircraft departing on its next flight or during flight (e.g., by recommending a system reset).

¹⁴ Although the number of UAS flight operations will exceed the number of commercial transport operations in the foreseeable future, aviation safety in terms of human safety will continue to be centered on commercial transport operations for the indefinite future. Nevertheless, as discussed in Chapter 2 the scope of an IASMS will include both general aviation aircraft and UAS.

by industry working together in the Commercial Aviation Safety Team (CAST), which describes its history and operations as follows:¹⁵

CAST was established with two goals: to reduce the U.S. commercial airline fatal accident rate by 80 percent over a 10-year period ending in 2007, and to work with airlines and international aviation organizations to reduce the worldwide commercial airline fatal accident rate.

The work of CAST, along with new aircraft, regulations, and other activities, reduced the fatality risk per million departures for commercial airlines in the United States by 83 percent from 1998 to 2008.¹⁷

CAST has evolved, and the group is moving beyond the historic approach of examining past accident data toward a more proactive approach focusing on detecting risk and implementing mitigation strategies before accidents or serious incidents occur. It aims to transition to prognostic safety analysis, and to reduce U.S. commercial fatality risk by a further 50 percent from 2010 to 2025. The increasing number of flights requires greater emphasis on acquiring, sharing, and analyzing aviation safety data. Using incident data, CAST is examining emerging and changing risks to identify prevention strategies.

Given that there are so few commercial airline accidents—and few common causes for those that do occur—more data points are needed. Voluntary reporting programs, such as the Aviation Safety Action Program (ASAP) and the Flight Operations Quality Assurance (FOQA) program, give airlines and government insight into millions of operations so that potential safety issues and trends can be identified. The Aviation Safety Information Analysis and Sharing (ASIAS) program ties together the safety databases across the industry and is integrated into the CAST process. ASAP, FOQA, and ASIAS all feature nonpunitive reporting so that operators (both individuals and organizations) can provide frank and complete data without concern that the FAA will take action against operators based on the data (see Chapter 3). These programs have matured to the point that the FAA can now look at data from air carriers representing over 80 percent of U.S. commercial airline operations to search for emerging risks. The FAA has increased the number of databases ASIAS can access; expanded ASIAS to include maintenance/air traffic information; increased membership by adding regional air carriers; and increased community stakeholders to include operators¹⁸ of general aviation and military aircraft, including helicopters.¹⁹

ASIAS resources include both public and nonpublic aviation data. Public data sources include, but are not limited to, ATM data related to traffic, weather, and procedures. Nonpublic sources include de-identified data from air traffic controllers and aircraft operators, including digital flight data and safety reports submitted by flight crews and maintenance personnel. ASIAS has the ability to query millions of flight data records and de-identified textual reports to facilitate directed studies, assessments of safety enhancements, monitoring of known risks, and

___________________

¹⁵ Commercial Aviation Safety Team (CAST), “Fact Sheet, Commercial Aviation Safety Team,” December 2011, http://www.cast-safety.org/pdf/cast1201.pdf.

¹⁶ CAST, 2011, “Background,” http://www.cast-safety.org/apex/f?p=180:1:27980477329992::NO::P1_X:background.

¹⁷ SKYbrary, 2017, “Commercial Aviation Safety Team (CAST),” last modified February 6, 2017, https://www.skybrary.aero/index.php/Commercial_Aviation_Safety_Team_(CAST).

¹⁸ In the aviation community, the term “operator” is used to refer both to individual human operators (e.g., pilots and air traffic controllers) and to the organizations that operate aircraft (e.g., airlines and government agencies). This report follows the same convention. Each time “operator” appears in the report, the specific meaning should be clear based on the context.

¹⁹ FAA, 2016, “Fact Sheet—Commercial Aviation Safety Team,” April 12, https://www.faa.gov/news/fact_sheets/news_story.cfm?newsid=18178.

discovery of emergent risks. ASIAS has also established key safety benchmarks so that individual operators may compare their own safety performance against the industry as a whole.²⁰

CAST uses ASIAS information by chartering working groups for in-depth analysis of precursors to the top accident categories in commercial transports. Safety enhancements are then identified to reduce such accidents and to prioritize and coordinate plans for implementing and, finally, monitoring actual effectiveness. Although most participants are from the United States, CAST promotes new safety initiatives by government and industry globally. Accident rates and causes vary by region and do not lend themselves to replicated solutions. With that in mind, CAST coordinates with the International Civil Aviation Organization (ICAO), the Flight Safety Foundation, the International Air Transport Association, the European Aviation Safety Authority, Transport Canada Civil Aviation, and other organizations, many of which have adopted CAST safety enhancements that are appropriate for different regions of the world or at a global scale.²¹

The General Aviation Joint Steering Committee is an industry-government organization dedicated to improve safety in general aviation, which lags far behind the safety of commercial transports.²² The steering committee has partnered with CAST and is coordinating the implementation of certain CAST Safety Enhancements in general aviation. In addition to that, several members of the steering committee such as FOQA and ASAP have joined ASIAS and are contributing voluntary safety information. As of July 2017, ASIAS had 56 general aviation operators contributing safety information.

CAST and the aviation safety systems described above operate over relatively long time frames. It typically takes months for data collection systems such as ASIAS and FOQA to accumulate and disseminate data to airlines and regulators, and it takes more months or years for these data to be assessed, for issues to be identified, for solutions to those issues to be developed, and for those solutions to be promulgated as new or modified procedures, practices, and regulations. An aviation safety assurance system, such as an IASMS, that operates over a much shorter time frame would complement these existing systems to provide a more comprehensive approach to maintaining and improving aviation safety.

PRIORITIZATION PROCESS

This report identifies 14 key challenges that will be the most difficult to overcome in developing and demonstrating advanced technologies and capabilities to achieve the committee’s vision for an IASMS. The discussion of each challenge begins with a summary statement that is followed by a summary of the reasoning for identifying that area as a key challenge. All but one of the challenges focus on technology issues; that one addresses economic issues (see Chapter 5).

The report also identifies 10 high-priority technology research projects that should be included in a national effort to support the development of an IASMS.²³ The selection of the projects was based on the committee’s consensus of the difficulty of completing each project and the urgency with which the research project should be

___________________

²⁰ U.S. Government Accountability Office (GAO), 2015, Aviation Safety: Proposals to Enhance Aircraft Tracking and Flight Data Recovery May Aid Accident Investigation, but Challenges Remain, GAO-15-443, Washington, D.C., http://www.gao.gov/assets/670/669754.pdf.

²¹ CAST, 2011, “Fact Sheet, Commercial Aviation Safety Team,” December, http://www.cast-safety.org/pdf/cast1201.pdf.

²² There were 219 fatal general aviation accidents in the United States with 347 fatalities in 2016, resulting in 8.4 fatal accidents per million operating hours. By comparison, from 2010 to 2014, inclusive, there were 11 fatalities caused by accidents by U.S. commercial airlines, resulting in 0.1 fatalities per million operating hours. FAA, 2017, “Fact Sheet—General Aviation Safety,” October 24, https://www.faa.gov/news/fact_sheets/news_story.cfm?newsId=21274; U.S. Department of Transportation, Bureau of Transportation Statistics, 2016, “U.S. Air Carrier Safety Data,” Table 2-9 in National Transportation Statistics 2016, https://www.bts.gov/sites/bts.dot.gov/files/legacy/NTS_Entire_2017Q1.pdf.

²³ All of the high-priority research projects address technology issues. The scope of the study does not include identifying research projects that are focused, for example, on policy or economics. Nonetheless, in some cases the results of the technology research projects will be useful for those who have the responsibility for addressing policy and economics.

initiated so that its results will be available in a timely fashion.²⁴ These two criteria (difficulty and urgency) reflect several associated considerations:

• The extent to which the current state of the art must be advanced;
• The time and resources needed to make those advances; and
• The time-phased application of research project results to the overall scheme of developing and deploying ever-more-capable IASMS.

The committee has grouped the challenges and research projects into one of four areas, each of which is discussed in one of the next four chapters:

• Chapter 2: IASMS Concept of Operations and Risk Prioritization
• Chapter 3: System Monitoring
• Chapter 4: System Analytics
• Chapter 5: Mitigation and Implementation

Chapter 6 completes the report by presenting the committee’s findings and recommendations that summarize the 14 key challenges and 10 high-priority research projects. Chapter 6 also addresses organizational roles and resources.

All of the high-priority research projects are judged to be both difficult and urgent; if they were not, they would not have been designated as a high priority. For most of the research projects, meeting the needs of an IASMS will likely require a mix of new technologies, improvements to existing technologies, and/or the adaptation of existing technologies developed for other applications. Each research project, as applicable, will need to determine the appropriate mix for that project.

The success of the research projects is hindered by the many unknowns regarding the scope of the IASMS. For example, requirements for the nature and quality of the IASMS data are still unknown, the policies and mechanisms for collecting IASMS data have not yet been determined, and the stakes involved in the performance of an IASMS are tremendously higher (because lives are at stake) than with most other applications. The output of the research projects in Chapter 2 will assist greatly in reducing uncertainties faced by the other research projects. Each research project, as applicable, will need to determine the appropriate mix of technologies: new, improved, and adapted from other applications.

The research project, IASMS Concept of Operations and National Airspace System Evolution, is judged to be of the highest priority (see Chapter 6). The report does not otherwise address the relative priority of the high-priority research projects because execution of most of the projects is most likely to be successful if they proceed in an iterative and integrated fashion that accounts for the many interactions among the different projects in Chapters 2 to 5. An iterative, integrated approach would also (1) allow advances in one area to support advances in other areas, (2) enable each research project to benefit as more detailed information becomes available, and (3) improve the quality of the complex trade-offs that will help guide the goals of each research project.

___________________

²⁴ The prioritization process described here is modeled after the prioritization process described in the first study in this series, each of which addresses the subject of one of the six strategic thrusts established by NASA’s Aeronautics Research Mission Directorate. That first report focused on assured autonomy for aviation transformation and is titled Autonomy Research for Civil Aviation: Toward a New Era of Flight (National Research Council, 2014, The National Academies Press, Washington, D.C.).