TRANSFORMATION IN THE AIR
A REVIEW OF THE FAA’S CERTIFICATION RESEARCH PLAN
Committee to Review the Federal Aviation Administration Research Plan on
Certification of New Technologies into the National Airspace System
Aeronautics and Space Engineering Board
Division on Engineering and Physical Sciences
NATIONAL RESEARCH COUNCIL
OF THE NATIONAL ACADEMIES
THE NATIONAL ACADEMIES PRESS
Washington, D.C.
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
This study is based on work supported by Agreement DTFAWA-14-A-80003 between the National Academy of Sciences and the Department of Transportation/Federal Aviation Administration. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the agency that provided support for the project.
International Standard Book Number-13: 978-0-309-37460-6
International Standard Book Number-10: 0-309-37460-X
Cover: Design by Tim Warchocki.
Copies of this report are available free of charge from
Aeronautics and Space Engineering Board
National Research Council
The Keck Center of the National Academies
500 Fifth Street, NW
Washington, DC 20001
Additional copies of this report are available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
Copyright 2015 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
THE NATIONAL ACADEMIES
Advisers to the Nation on Science, Engineering, and Medicine
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. C. D. Mote, Jr., is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Victor J. Dzau is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. C. D. Mote, Jr., are chair and vice chair, respectively, of the National Research Council.
OTHER RECENT REPORTS OF THE AERONAUTICS AND SPACE ENGINEERING BOARD
3D Printing in Space (Aeronautics and Space Engineering Board [ASEB], 2014)
Autonomy Research for Civil Aviation: Toward a New Era of Flight (ASEB, 2014)
Pathways to Exploration: Rationales and Approaches for a U.S. Program of Human Space Exploration (ASEB with the Space Studies Board [SSB], 2014)
Solar and Space Physics: A Science for a Technological Society (SSB with ASEB, 2013)
Final Report of the Committee for the Review of Proposals to Ohio’s Third Frontier Program, 2012-2013: Innovation Platform Program 2013 (ASEB, 2013)
Continuing Kepler’s Quest: Assessing Air Force Space Command’s Astrodynamics Standards (ASEB, 2012)
NASA Space Technology Roadmaps and Priorities: Restoring NASA’s Technological Edge and Paving the Way for a New Era in Space (ASEB, 2012)
NASA’s Strategic Direction and the Need for a National Consensus (Division on Engineering and Physical Sciences, 2012)
Recapturing NASA’s Aeronautics Flight Research Capabilities (ASEB, 2012)
Reusable Booster System: Review and Assessment (ASEB, 2012)
Final Report of the Committee to Review Proposals to the 2011 Ohio Third Frontier Wright Projects Program (OTF WPP) (ASEB, 2011)
Limiting Future Collision Risk to Spacecraft: An Assessment of NASA’s Meteoroid and Orbital Debris Programs (ASEB, 2011)
Recapturing a Future for Space Exploration: Life and Physical Sciences Research for a New Era (SSB with ASEB, 2011)
Advancing Aeronautical Safety: A Review of NASA’s Aviation Safety-Related Research Programs (ASEB, 2010)
Capabilities for the Future: An Assessment of NASA Laboratories for Basic Research (Laboratory Assessments Board with ASEB, 2010)
Defending Planet Earth: Near-Earth-Object Surveys and Hazard Mitigation Strategies: Final Report (SSB with ASEB, 2010)
Final Report of the Committee to Review Proposals to the 2010 Ohio Third Frontier (OTF) Wright Projects Program (WPP) (ASEB, 2010)
Limited copies of ASEB reports are available free of charge from
Aeronautics and Space Engineering Board
National Research Council
The Keck Center of the National Academies
500 Fifth Street, NW, Washington, DC 20001
(202) 334-2858/aseb@nas.edu
www.nationalacademies.org/aseb.html
COMMITTEE TO REVIEW THE FEDERAL AVIATION ADMINISTRATION RESEARCH PLAN
ON CERTIFICATION OF NEW TECHNOLOGIES INTO THE NATIONAL AIRSPACE SYSTEM
S. MICHAEL HUDSON, I Power Energy Systems, Co-Chair
WILLIAM S. LEBER, JR., PASSUR Aerospace, Co-Chair
JANDRIA S. ALEXANDER, The Aerospace Corporation
STEVEN J. BROWN, National Business Aviation Association
VICTORIA COX, Victoria Cox Solutions, LLC
JOSEPH M. DEL BALZO, JDA Aviation Technology Solutions
R. JOHN HANSMAN, JR., Massachusetts Institute of Technology
AMY R. PRITCHETT, Georgia Institute of Technology
AGAM N. SINHA, ANS Aviation International, LLC
EDMOND L. SOLIDAY, Indiana General Assembly
RAYMOND VALEIKA, Delta Air Lines (retired)
EDWARD L. WRIGHT, University of California, Los Angeles
Staff
DWAYNE DAY, Study Director, Aeronautics and Space Engineering Board
MICHAEL H. MOLONEY, Director, Aeronautics and Space Engineering Board and Space Studies Board
ANDREA REBHOLZ, Program Coordinator, Aeronautics and Space Engineering Board
AERONAUTICS AND SPACE ENGINEERING BOARD
LESTER LYLES, The Lyles Group, Chair
PATRICIA GRACE SMITH, Patti Grace Smith Consulting, LLC, Vice Chair
ARNOLD D. ALDRICH, Aerospace Consultant, Vienna, Virginia
ELLA M. ATKINS, University of Michigan
STEVEN J. BATTEL, Battel Engineering
MEYER J. BENZAKEIN, The Ohio State University
BRIAN J. CANTWELL, Stanford University
ELIZABETH R. CANTWELL, Lawrence Livermore National Laboratory
EILEEN M. COLLINS, Space Presentations, LLC
MICHAEL P. DELANEY, Boeing Commercial Airplane Group
VIJAY K. DHIR, University of California, Los Angeles
EARL H. DOWELL, Duke University
ALAN H. EPSTEIN, Pratt & Whitney
KAREN FEIGH, Georgia Institute of Technology
PERETZ P. FRIEDMANN, University of Michigan
MARK J. LEWIS, IDA Science and Technology Policy Institute
RICHARD McKINNEY, Independent Consultant
JOHN M. OLSON, Sierra Nevada Corporation
HELEN L. REED, Texas A&M University
AGAM N. SINHA, ANS Aviation International, LLC
ALAN M. TITLE, Lockheed Martin Advanced Technology Center
DAVID M. VAN WIE, Johns Hopkins University Applied Physics Laboratory
MICHAEL H. MOLONEY, Director
CARMELA J. CHAMBERLAIN, Administrative Coordinator
TANJA PILZAK, Manager, Program Operations
CELESTE A. NAYLOR, Information Management Associate
MEG A. KNEMEYER, Financial Officer
SANDRA WILSON, Financial Assistant
Preface
The FAA Modernization and Reform Act of 2012 required the Federal Aviation Administration (FAA) to develop a research plan for the certification of new technologies into the National Airspace System and to have the National Research Council (NRC) review that plan. The FAA has produced various internal planning documents both to guide its research and to determine how it will introduce new technologies, but these are generally not publicly distributed. Starting in the latter 1990s, the FAA began a program named NextGen (Next Generation Air Transportation System) to introduce many new technologies both on the ground and in aircraft to improve the tracking of aircraft and operations. NextGen is the primary program for introducing new technologies in the National Airspace System, but Congress did not explicitly call for a NextGen research plan in the 2012 act. In February 2014, the FAA Office of NextGen produced Research Plan: Methods and Procedures to Improve Confidence in and Timeliness of Certification of New Technologies Into the National Airspace System1 in response to the 2012 act.
In response to an FAA request, the NRC established the Committee to Review the Federal Aviation Administration Research Plan on Certification of New Technologies into the National Airspace System in late summer 2014 to review the research plan. The committee was co-chaired by Michael Hudson and William Leber and consisted of 10 additional members. The membership’s expertise included familiarity with FAA management, regulations and operations, commercial, general and business aviation, and relevant areas of research such as software and human factors. The committee met three times: November 19-21, 2014, in Washington, D.C.; January 21-23, 2015, in Irvine, California; and March 3-6, 2015, in Washington, D.C.
The committee’s statement of task was as follows:
An ad hoc committee under the auspices of the National Research Council will conduct an assessment of the Federal Aviation Administration’s plan for research on methods and procedures to improve both confidence in and the timeliness of certification of new technologies for their introduction into the National Airspace System (NAS).
The FAA research plan, which will be publicly available, focuses on air-ground and ground-based system approval. The research focus areas in the plan include:
- Systems engineering
- Requirements development
________________
1 FAA, Research Plan: Methods and Procedures to Improve Confidence in and Timeliness of Certification of New Technologies Into the National Airspace System, Final, Office of NextGen, Washington, D.C., February 2014; reprinted in Appendix A.
- Human factors
- Software development
- Verification, validation, and testing
- Software design assurance
- Training
- Operational evaluation
The committee will:
- Provide an analysis of the FAA’s proposed research, as described in the research plan, focused on how implementing the plan may improve both timeliness and confidence of certifying new technologies into the NAS as requested in the FAA reauthorization section 905;
- Identify for each focus area any specific actions that may have been successful in similar activities within and external to the FAA;
- Provide an assessment of planned research outputs in meeting the wording in the 2012 FAA Modernization and Reform Act “to conduct research on methods and procedures to improve both confidence in and timeliness of certification of new technologies for their introduction into the national airspace system”;
- Comment on general readability and clarity of presentation of the research plan; and
- Provide comments on or recommendations on other issues determined by the committee.
A key aspect of the committee’s efforts was determining how the research plan interacted with other FAA planning documents and how it might guide future research. The plan is a relatively short document (10 pages), and in this report the committee offers a number of findings and recommendations about how the FAA may produce a better research plan in the future that may be more representative of what lawmakers were seeking when they called for it in 2012.
Acknowledgment of Reviewers
This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s (NRC’s) Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report:
Jay Apt, Carnegie Mellon University,
Bruce J. Holmes, NextGen AeroSciences, LLC,
Butler Lampson, Microsoft Corporation,
John K. Lauber, Independent Consultant, Vaughn, Washington,
J. Victor Lebacqz, VICC Associates,
David E. Liddle, U.S. Venture Partners
Dinesh Verma, Stevens Institute of Technology,
David Victor, University of California, San Diego,
Steven Winter, Raytheon Company, and
Andres Zellweger, FAA (ret.)
Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor did they see the final draft of the report before its release. The review of this report was overseen by M. Granger Morgan, Carnegie Mellon University. Appointed by the NRC, he was responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.
This page intentionally left blank.
Contents
1 A REVIEW OF THE FAA RESEARCH PLAN
Components of an Effective FAA Research Plan
2 SPECIFIC SHORTFALLS IN THE FEBRUARY 2014 RESEARCH PLAN
Safety and Performance Assurance—Software Assurance and Software Certification
Safety and Performance Assurance—Verification, Validation, and Testing
Operational Test and Evaluation
3 LEARNING FROM OTHER PROJECTS AND OTHER ORGANIZATIONS
Gulf of Mexico Helicopter Initiative
Characteristics of NextGen Successes
Other Agencies and Organizations