Transformation in the Air: A Review of the FAA's Certification Research Plan (2015)

A

FAA Research Plan

The Federal Aviation Administration (FAA) report Research Plan: Methods and Procedures to Improve Confidence in and Timeliness of Certification of New Technologies Into the National Airspace System (Final, Washington, D.C., February 2014) is reprinted here without its contents page or list of acronyms.

Research Plan

Methods and Procedures to Improve Confidence in and Timeliness of Certification of New Technologies Into the National Airspace System

FINAL

February 2014

Change History

Prepared by:

Office of NextGen
William J. Hughes Technical Center
Aviation Research Division

EXECUTIVE SUMMARY

On February 14, 2012, the President signed the FAA Modernization and Reform Act of 2012 (the Act), [1] Section 905 of the Act, entitled “Research on Design for Certification,” instructs the Federal Aviation Administration (FAA) to “conduct research on methods and procedures to improve both confidence in and the timeliness of certification of new technologies for their introduction into the national airspace system.” The Act also instructs the FAA to develop a research plan that contains objectives, proposed tasks, milestones, and a 5-year budgetary profile, and to arrange an independent review of this plan by the National Research Council.

The scope of this research plan is defined in part by 49 United States Code, Section 44505, entitled “Systems, procedures, facilities, and devices.” In preparing this report, we have considered certification processes for the two basic categories of technologies that are central to the Next Generation Air Transportation System (NextGen): (1) those associated with aircraft; and (2) those associated with ground-based systems and air traffic control (ATC).

The procedures we follow to certify the manufacture and installation of aircraft equipage (i.e., avionics that enable NextGen capabilities) are critical to the success of NextGen and the aviation community’s confidence in it. The FAA has completed a number of initiatives focused on the manner and procedures for approving new technologies in aircraft. Most recently, the FAA convened industry experts in the Aircraft Certification Process Review and Reform Aviation Rulemaking Committee (ARC), as required by Section 312 of the FAA Modernization and Reform Act of 2012. The FAA is evaluating the recommendations from this committee and determining implementation plans. Therefore, at this time, the FAA has not specified additional research requirements necessary to improve the effectiveness or timeliness of certification procedures for new aircraft technologies. However, we have identified several key technical areas within the ground-based system and ATC design approval processes that would benefit from further research to determine how their effectiveness and accuracy can be improved.

New ground-based systems and ATC technologies are managed throughout their lifecycle through the FAA Acquisition Management System (AMS) [2], a process designed to increase the quality, reduce the time, manage the risk, and minimize the cost of delivering safe and secure services to the aviation community and flying public. The FAA has been using the AMS process for more than 16 years, and after a recent system-wide evaluation, we have supplemented it with Air Traffic Organization process improvement efforts and Safety Management Systems [3]. AMS areas on the critical path for programs that we believe would benefit from additional research assistance are software assurance, software certification, computer human interfaces, technical interfaces, hand-off interfaces, and verification and validation.

The FAA believes that we provide sufficient oversight to ensure that new technologies, from a programmatic perspective, are efficiently incorporated within the National Airspace System (NAS) and that associated benefits are achieved for the aviation industry in a timely and accurate manner. These oversight vehicles include the formation of the Office of NextGen (ANG), which reports directly to the Deputy Administrator of the FAA; the focus on NextGen transformational programs (i.e., Automatic Dependent Surveillance-Broadcast) that provide core capabilities in achieving the NextGen vision; the incorporation of portfolio capture teams that shepherd related

capabilities through their lifecycle; and portfolio leads, who ensure appropriate integration of systems and capabilities.

This plan fully addresses the scope, tasking, and funds required to conduct the identified research. As a result of this planning effort, a 5-year research budget of $4.6M is required to execute this research plan, make appropriate enhancements to the approval process, and develop associated information technology. This research plan is not currently included or funded in the National Aeronautics Research Plan (NARP), partially because it needs to go through an independent review and adjudication process with the National Research Council, as described below.

1. BACKGROUND AND SCOPE

The Next Generation Air Transportation System, or NextGen, is a set of initiatives that will make our national airspace system (NAS) safer and more efficient while curbing aviation’s environmental impact.

NextGen’s suite of tools includes better traffic management and improved procedures that will help us reduce delays and save fuel. Satellite-based surveillance provides more precision in tracking aircraft, and gives pilots the ability to see other aircraft around them just as air traffic controllers do. Advanced digital communications between the ground and the flight deck reduces opportunities for error, and system-wide information management gives the right information to the right people at the right time.

The FAA is challenged with implementing a complex set of NextGen improvements into a NAS that operates 24 hours a day, every day. One of our challenges is earning the confidence required for operators to equip their aircraft to take advantage of NextGen capabilities as they become available.

On February 14, 2012, the President signed the FAA Modernization and Reform Act of 2012. [1] Section 905 of the Act, entitled “Research on Design for Certification” instructs the Federal Aviation Administration (FAA) to “conduct research on methods and procedures to improve both confidence in and the timeliness of certification of new technologies for their introduction into the national airspace system.” The Act also instructs the FAA to develop a research plan that contains objectives, proposed tasks, milestones, and a 5-year budgetary profile, and to arrange an independent review of this plan with the National Research Council.

The scope of the research plan is dictated in part by the language in Section 905 that amends 49 United States Code (USC) Section 44505, “Systems, procedures, facilities, and devices.” The primary focus of Section 44505 is to meet the needs for safe and efficient navigation and traffic control of civil and military aviation. More specifically, this section addresses research on human factors and simulation models, and developing and maintaining a safe and efficient system.

The FAA has convened industry experts in the Aircraft Certification Process Review and Reform Aviation Rulemaking Committee (ARC), as required by Section 312 of the FAA Modernization and Reform Act of 2012, and the Office of Aviation Safety (AVS) has developed comprehensive plans for the implementation of aircraft certification technologies, including an Aviation Safety Work Plan for NextGen 2012 [4] that addresses the aircraft certification activities for new technologies. Therefore, certification activities specifically related to aircraft, aircraft engines, propeller, and appliances (as described under 49 USC, Section 44504) are not included in the scope of this research plan. This plan does address areas that require close coordination (i.e., software assurance, air-ground communications/interfaces, and controller training).

The AMS [2] process governs the acquisition of new technologies for ground-based systems. Continuing improvements to the AMS and Safety Management System processes will ensure that new technologies are incorporated within the NAS in a timely and safe manner. In addition, other process improvement efforts provide coordination of program portfolios that ensure the

benefits of NextGen technologies are delivered in a consistent and timely manner within the NAS.

Incorporation of new technologies is intended to enhance the efficiency of the NAS (i.e., reduced separation, preferred routing, and fuel efficiency) or to increase the productivity and safety of travelers, air traffic controllers, and pilots (i.e., increased throughput, reduced operational errors, reduced pilot deviations, and shared situational awareness). Whenever new technologies are incorporated for efficiency, safety, or enhanced productivity, there is typically an impact on three critical areas:

1. New technology integration almost always includes a complex software development effort along with detailed internal and external interfaces that allow the technology to interoperate with other NAS components and external NAS stakeholders,
2. New technologies intended to benefit the users typically entail enhancements to the human interfaces (e.g., air traffic controllers and facility management personnel) involved in ATC operations, and
3. Integration of new technologies within the NAS typically affects other key areas that include, but are not limited to, key elements of the AMS process including requirements, verification and validation, and training.

An overview of the approval process is depicted in Figure 1 below. The research effort will evaluate the entire approval process with a focus on how the introduction of new technologies is handled during specific steps of the process. The research focus areas are shown capitalized within the System Design Approval, and this effort will not address the other two categories.

Figure 1: – NAS Ground-System Approval Process Overview

Additionally, the research will examine identification, development, and implementation of measures that would provide FAA managers and oversight personnel with objective insight into the status of the ground-based system design approval process.

2. ASSUMPTIONS

It is assumed that this research will only address the approval of new technologies being integrated into the NAS and will not specifically address aircraft or aircraft equipage certification activities. Aircraft equipage is covered under Section 215 of the FAA Modernization and Reform Act of 2012. Likewise, aircraft certification procedures are addressed under Section 312 of the FAA Modernization and Reform Act of 2012.

It is assumed that when the term “approval process” is used throughout this document, it is referring to the focus areas identified in Figure 1 of this report.

For the purposes of this research plan, the term “timeliness” refers to the overall time to define, develop, test, and deploy a design for ground or air-ground systems within the NAS.

For purposes of this research plan, the term “confidence” refers to the expectation that a system will perform within the specified reliability, maintainability, and availability as described in the system design. Likewise, the term confidence also means that a system design is fully and completely verified and validated.

For purposes of this research plan, the term “certification” refers to the overall “approval process” for the design of a ground or air-ground system within the NAS in accordance with the AMS. In addition, the term certification refers to the approved design of critical software components (i.e. software certification levels) and ground based equipment (i.e. ILS certification).

It is assumed that, from a programmatic perspective, the FAA is addressing the delivery of key technologies for the evolution of NextGen. Therefore, it is assumed that this research plan will address the technical aspects of the approval process versus the programmatic concerns.

3. RESEARCH OBJECTIVES, OUTPUTS, AND OUTCOMES

The major objective of this research is to identify methodologies, procedures, and information technologies that can increase the efficiency (i.e., timeliness) and effectiveness (i.e., confidence) for integrating new technologies within NAS ground- or air/ground-based systems (integrated NAS systems). As new methodologies and procedures are explored, there is an opportunity for both the FAA and the aviation community to achieve process improvements and minimize costs.

By researching the current and planned processes of ATC software assurance, software certification, human factors, interfaces, and training, it will be possible to create some objective measures surrounding the current and future processes. Properly analyzing this information could give FAA managers better insight into the efficiency and effectiveness of ongoing procedure approval efforts. A more forward-looking objective would be to continually monitor the process, which will allow the Air Traffic Organization (ATO) to take a proactive approach for overseeing the process and allow for more transparency into the process. Overall, all stakeholders can better understand the impacts and benefits of new technologies before final implementation.

Outcomes (i.e., timeliness and effectiveness) are not specifically measurable. More specific outputs for timeliness and confidence (i.e. measurable outputs) will be developed and delivered for each of the following AMS [2] research focus areas:

• Software assurance,
• Software certification,
• Requirements management and traceability,
• Human technologies and computer human interaction,
• Verification and Validation, Testing, and Operational Evaluation,
• Controller Training Systems,

• Technical Interfaces, and
• Information Security.

For each of the above research focus areas, the following outcomes and associated outputs will be developed:

Outcome—Timeliness

• Output—Measure time and collect data for the current design approval process
• Output—Measure time and collect data for the individual process elements of the design approval process
• Output—Collaboratively identify and recommend modifications to the current design approval process to increase timeliness
• Output—Prototype modifications to current approval process and develop/modify tools (e.g., human interaction, interface simulation, software analyses, modeling, and information security)
• Output—Apply prototype modifications to several ongoing approval process efforts (pilot studies)
• Output—Measure time and collect data on the new approval process
• Output—Produce ongoing measures and trends of the approval process
• Output—Integrate measures and trends into ATO oversight and decision-making process
• Output – Report the research results

Outcome—Confidence

• Output—Develop analysis capabilities that support air traffic personnel in evaluating information related to the design approval process
• Output—Develop information-sharing technologies that support the FAA and industry involvement earlier in the design approval process

4. RESEARCH TASKING

As instructed in the Act, the FAA will ask that the National Research Council perform an independent review of this plan and work cooperatively with the FAA to modify the plan in accordance with the review.

The research will undertake a systematic approach to achieve the required outcomes and objectives.

1. Perform background analyses and data collection of current processes and technologies and plan for addressing new technologies and planned NextGen enhancements.
2. Conduct interviews with current FAA and industry personnel who frequently perform the design approval process (e.g., software assurance, software certification, human factor assessment, training development, and information security).
3. Identify measures and potential enhancements to the processes, procedures, and information technologies surrounding the design approval process.
4. Perform a gap analysis on the long-term needs and currently funded initiatives.
5. Design modifications to current design approval processes to help increase the timeliness and effectiveness of the process.
6. Design and develop information technology and human performance analysis prototypes to support the design changes and identified measures. Develop applicable tools and facilities to support research focus areas (e.g., software assurance, software certification, requirements, verification and validation requirements).
7. Analyze new processes, procedures, and technologies (benefits analysis).
8. Implement new processes, procedures, and technologies (via pilot studies of new and technologies) that will result in enhanced design approval processes.
9. Monitor and collect data on new processes, procedures, and technologies.
10. Validate measures and results of new processes, procedures, and technologies.
11. Produce a final report with results, findings, and suggestions for recommended of processes, procedures, and applicable technologies that can be integrated the ATO workforce.
12. Integrate new modifications within the ATO.

5. CHALLENGES

Several known challenges are affiliated with this type of research effort.

While the agency embraces positive change, it can take large organizations time to assimilate it throughout the operation. Careful steps must be taken to educate the workforce participating in the pilot projects so there is an awareness that the research effort is of benefit to them. Research results will allow for a more efficient and timely process so procedures can be developed faster without degrading the effectiveness of the process.

FAA also needs to invest in efforts with a positive cost-benefit ratio.

6. PROJECT SCHEDULE, MILESTONES, AND BUDGETING

This section describes the key dates anticipated from the research effort. Key dates for the project should be integrated with all stakeholders for the research project.

7. RESOURCE REQUIREMENTS

It is estimated that the 5-year research effort will require two in-house technical full-time equivalents (FTE) and a half programmatic FTE for the duration. It is also estimated that research contractor support will be required in the amount of $4.6M over the 5-year research effort as detailed by the research tasks identified in Section 6.

Below is the 5-year funding profile requested for the entire research effort.

8. REFERENCES

1. FAA Modernization and Reform Act of 2012, Pub. L. No. 112-95, 126 Stat. 11 (February 14, 2012).
2. FAA Acquisition Management System, available at “http://fast.faa.gov”
3. FAA JO 1000.37, “Air Traffic Organization Safety Management System,” March 19, 2007.
4. FAA AVS, “AVS Work Plan for NextGen 2012,” March 2012