Flight to the Future: Human Factors in Air Traffic Control (1997)

The nation's air traffic control system, which is part of the national airspace system, is responsible for managing a complex mixture of air traffic from commercial, general, corporate, and military aviation. Despite the strong safety record achieved over the last several decades, the system does suffer occasional serious disruption, often the result of outdated and failed equipment. When equipment failures occur, system safety relies on the skills of controllers and pilots. Under these circumstances, safety is maintained by reducing the number of aircraft in the air. Pressures to provide the capacity to handle a greater number of flights in the future and to maintain high levels of safety and efficiency have led to proposals to provide more reliable and powerful equipment and at the same time increase the level of automation in air traffic control facilities—that is, to use advances in technology to take over tasks that are currently performed by humans. Such proposals have raised concern from members of the Subcommittee on Aviation of the Public Works and Transportation Committee of the U.S. House of Representatives that automation not compromise the safety or efficiency of the system by marginalizing the human controller's ability to provide the necessary backup when disruptions occur.

As a result, the Panel on Human Factors in Air Traffic Control Automation was convened at the request of the Federal Aviation Administration (FAA) for the purposes of gaining an understanding of, and providing recommendations on, the human factors characteristics of the current air traffic control system, the national airspace system, and future automation alternatives in terms of the human's role in the system. The panel's charge divides the tasks into two phases. The first focuses on the current system and its development as a means to:

(1) understand the complexities of and problems with the current air traffic control system that automation is intended to address; (2) describe the manner in which some levels of automation have already been implemented; and (3) provide a baseline of human factors knowledge as it relates to the functions of the air traffic controller in the system. The second phase is to assess future automation alternatives and the role of the human operator in ensuring safety and efficiency in the air traffic control system.

This report provides the results of the panel's work during the first phase.

The goal of the air traffic control system is to satisfy and balance the two critical goals of safety and efficiency. Human participants in the system must make continuous adjustments in flight scheduling and flight paths to maximize efficiency without compromising safety. The many redundant components in the system, and the smooth communications between its operators (both on the ground, and between ground and air) have generally allowed it to recover gracefully from failures, without accident. Because perfect system reliability can never be assumed, it is important that planners not change the system in ways that will destroy these critical failure-recovery aspects.

Conclusions Despite the complexity of the national airspace system and of its many semiautonomous air traffic control facilities, the system has operated with a remarkably good safety record. However, the skills of air traffic controllers are being increasingly relied on to compensate for the limited capacity and declining reliability of aging equipment. Although new procedures and technologies that represent increases in automation are being considered as means of meeting projected increases in air traffic, human controllers are expected to maintain responsibility for the safe and efficient flow of air traffic for the foreseeable future. No matter how well the system is engineered and tested, some level of system unreliability (or some degree of system failure) is inevitable. And some level of human error is also inevitable, so long as human operators remain in the system. Such errors are not likely to be damaging to system performance if they can be caught and corrected by error-tolerant systems.

Recommendation The panel recommends that the FAA expand current efforts to reduce errors by employing good human factors in design and by adopting a fault-resistant and fault-tolerant philosophy of system design. Such a philosophy would render the system less susceptible to catastrophic failures in the case of errors by the human operator (controller, pilot, maintenance specialist) or failures of equipment.

The FAA is considering increased application of automation to air traffic

control as a means of achieving minimum accidents and maximum efficiency of air travel. However, these goals can potentially be in conflict; the FAA has expressed its resolve that, under such circumstances, safety will remain the number one priority.

Conclusions Unclear definitions of efficiency factors, efficiency measures, and acceptable levels of efficiency, as well as lack of knowledge about the effects of proposed improvements on the cognitive tasks of controllers, inhibit predictive assessment of whether and to what extent proposed improvements to equipment and to procedures—including automated features—will actually contribute to safety or to efficiency.

Recommendations The panel recommends that the FAA specify acceptability criteria for safety and efficiency and foster the development and application of models of the controller and the national airspace system that: (a) clearly identify indicators and measures and make use of the levels of acceptability for safety and for efficiency set by FAA management; (b) assess the interaction of safety and efficiency factors; (c) take into account the cognitive tasks of controllers in balancing the pressures of both safety and efficiency in tactical and strategic decisions and behavior; and (d) take into account the variables associated with different air traffic control options, regions, and facilities. The developed models should be applied to the evaluation of proposed changes in equipment, software, and procedures.

Controllers are trained for their duties by a combination of formal classroom instruction and on-the-job training. The selection and screening criteria have varied over the years, with current emphasis on a ''train for success" philosophy, designed to reduce training program attrition. As a consequence, the major component of both training and selection takes place within the facilities where developmental controllers (i.e., trainees) receive on-the-job training from full-performance-level controllers, who serve as instructors. Much of this training is received while developmental controllers handle live traffic, closely supervised and evaluated by other controllers.

Conclusions The FAA has recognized that, in order to select new controllers effectively, the agency must validate its selection procedures against on-the-job performance. Detailed job performance criteria are necessary to enable effective selection, training, and performance assessment of controllers. The FAA has commenced an assessment program with the goal of establishing clear definitions of the tasks and criteria that characterize effective performance.

Recommendations The panel recommends that efforts be continued to develop

job-related performance criteria as prerequisites for both personnel selection and evaluation of training procedures. We further encourage the FAA to reexamine controller job tasks and performance criteria when new air traffic control technology, including automation, is introduced; and to coordinate the development of performance criteria with the development of a comprehensive selection battery for controllers, as well as with continued study of the relationship between performance and the personality and demographic characteristics of controllers.

Conclusion Due to reductions in staffing, full-performance-level controllers may not have sufficient time to leave their assigned facilities to receive refresher training or training in use of new or upgraded software or equipment introduced into the current air traffic control system.

Recommendations The panel recommends the accelerated development and the use of computer-based training simulations that incorporate both performance assessment and the particular characteristics of a facility's airspace. These simulations should include a capability to provide augmented feedback for training. SATORI (situation assessment through re-creation of incidents) is a good example of such a capability; it provides a graphic display of data along with synchronized voice that can be used to review performance on a simulation exercise.

Controllers in all types of air traffic control facilities develop strategic plans for traffic flow, monitor these plans with visual inputs to update their "big picture" of the traffic flow, and communicate heavily with pilots and other controllers to ensure continued safety and efficiency. Controllers in the towers depend heavily on direct visual sightings of traffic at the airport, while those in the TRACON (terminal radar approach control around airports) and in the en route environments are supported by computer-based, partially automated radar displays. All controllers must be prepared to deal with unanticipated events—for example, equipment failure, weather emergency, or pilot noncompliance with instructions—in a flexible manner that preserves safety even if it temporarily disrupts efficiency.

Conclusions The technique of cognitive task analysis has revealed several strengths of the skilled air traffic controller, along with a number of vulnerabilities inherent in human information processing. The strengths include the ability to bring experiences stored in long-term memory to bear in solving novel unexpected problems. Weaknesses include vulnerability in detecting subtle and infrequent events, in predicting events occurring in a three-dimensional space, and in

temporarily storing and sometimes communicating information. Considerable human factors knowledge exists as to how these vulnerabilities can be addressed by design and training.

Recommendations The panel recommends that changes to air traffic control systems should include not only efforts to retain and capitalize on the controller's cognitive strengths, but also efforts to compensate for weaknesses. Such compensation includes making subtle and infrequent events more prominent, providing explicit (and reliable) predictive displays whenever possible, providing redundant communications and visual backup for working memory when errors can be critical, providing visible feedback for state changes, and using display techniques to improve individual and shared situation awareness, both among controllers and between controllers and pilots.

Workload is one of the most critical characteristics of the controller's task. It is driven by objective characteristics of the air traffic control system (e.g., number of aircraft, complexity of sector routes, quality of displays) and is experienced by the controller (e.g., measurable by behavioral or physiological indices). Skilled controllers adapt their performance strategies in handling aircraft as workload increases, in order to prevent excessive workload or loss of safety.

Conclusions Increases in air traffic density and complexity have led to substantial demands on the mental workload of controllers. Very high workload can lower performance and set an upper limit on traffic-handling capacity. Very low workload may result in boredom and reduced alertness, with consequent implications for handling emergencies. Factors influencing controller mental workload include airspace variables, display factors, work team dynamics, and controller-pilot communications. Most controllers use various adaptive strategies to manage their performance and subjective perceptions of task involvement. When evidence relating air traffic control operational errors to performance and workload has been found, the errors have been linked to both low and high task load conditions. Such conditions increase demands on controller monitoring and vigilance, and they could increase in the future as the system becomes more automated. Current work-rest schedules have not been documented to have a negative impact on controller performance, although subjective complaints of fatigue occur. However, shift work and the consequent disruption of circadian rhythms and sleep loss continue to be a major source of concern.

Recommendations The panel recommends that workload assessment span the entire range of air traffic control workload, from low to high (underload to overload). Current performance measures are not adequate to provide indices of performance potential—and hence safety. These must be accompanied by measures of controller workload. We therefore recommend developing predictive air traffic control workload models similar to those used for flight control and management

tasks, initiating additional studies to rectify the relative paucity of studies of underload and boredom in air traffic control, and encouraging the scheduling of controller shift and work-rest cycles that will be consistent with the state of the art in research on fatigue, circadian rhythms, and sleep loss. We also recommend that the FAA discourage current shift-work patterns (e.g., phase-advanced shifts and compressed work weeks), which may be associated with degraded performance.

The individual controller is part of an interlocking set of teams. Communication is vital to these team functions. Much communication can be analyzed from an information processing perspective, and much of it depends on both sender and receiver sharing the same mental model or awareness of the situation. But several critical aspects of team communication relate more to the discipline of social and personality psychology. Over the last 20 years, these have been revealed through the study of cockpit resource management on the flight deck. Breakdowns in flight deck crew resource management that resulted in accidents have spawned a series of training programs for pilots that have been successfully implemented in many airlines. These programs have a record of improved safety and improved attitudes toward teamwork. A corresponding program, called air traffic teamwork enhancement (ATTE), has been developed for air traffic controllers.

Conclusions Teamwork, reflected in communication among controllers and their supervisors and between controllers and flight crews, is a critical component of air traffic control. Cockpit resource management (CRM) team training has proved effective in improving team coordination, communications, and task management for aircraft flight crews. Similar training for air traffic controllers, their supervisors, and their trainers has the potential to provide similar enhancement of teamwork. The air traffic teamwork enhancement program, a team training program developed for controllers based on CRM principles, contains positive features, but it does not provide the necessary recurrent training or hands-on practice and reinforcement of team skills. In general, there is a severe lack of research pertaining to teamwork aspects of air traffic control, including teamwork aspects of selection, training, performance appraisal, communication, cognitive behavior, shared situation awareness, workload, and system design.

Recommendations The panel recommends that the FAA initiate a systematic effort to reinforce the value of teamwork within its organizational culture. Ways to do this are to define team coordination as part of task descriptions, to include evaluation of team skills as part of performance assessment, and to consider team factors during investigation of operational errors. We further recommend that an improved program of team training for controllers, their supervisors, and on-the-job training instructors should be a centrally funded program required at all air

traffic facilities. The program should include training in controller-to-supervisor interface issues and team-related automation issues. The training should be refined on the basis of empirical data derived from analysis of team issues in operational errors, surveys of controllers' attitudes regarding team issues, behavioral measures of team performance based on simulation, and evaluations of the program by participants. Team training should provide for recurrent training, hands-on practice, and reinforcement of team skills. We further recommend that the FAA initiate efforts to fill gaps in knowledge of teamwork aspects of air traffic control, including research on teamwork aspects of selection, training, performance appraisal, communication, cognitive behavior and performance, workload, and system design.

Broader than simply teams, the air traffic control system incorporates a much larger organizational context within which the controller works, a context that is characterized by procedures, regulations, a labor-management structure, and performance-based rewards and penalties. All of these factors contribute, in a complex but poorly understood way, to the performance of the controller both as an individual and as a team member. In addition to the organizational structure, which can be documented by formal written procedures, different facilities are also characterized by a less formal, but equally potent organizational culture, defined by attitudes, ways of doing things, and informal delegations of responsibility that may be quite different from formal responsibilities. These differences in culture may have dramatic influences on the ways in which new technologies are received within a facility and the ways in which emergencies are handled.

Conclusions Both formal and informal (cultural) organizational context factors contribute to safety and efficiency and are implicated in the successful introduction of new technologies. From this perspective, key aspects of the FAA's formal organizational context include: policies governing safety and efficiency; allocation of authority and responsibility; procedures for selecting, training, managing, and evaluating the workforce; legal liability; labor-management relations; and processes for procuring and implementing new technologies and for introducing changes. The informal cultural context includes such important safety-and efficiency-related factors as: informal rules and norms, subculture differences, job satisfaction, and attitudes toward change. Organizational culture affects and is affected by organizational structure.

There is a lack of research data that would permit identification of the specific mechanisms by which formal and informal organizational contexts within the FAA interact and how they affect organizational climate and controller performance.

Studies to date have shown, however, that during high-tempo conditions (represented by such events as responding to very high traffic loads, large-scale outages, and weather contingencies) informal teaming arrangements, leadership roles, and procedures are often invoked; these informal responses may help to account for the remarkable safety record demonstrated to date during such conditions.

Recommendations The panel recommends that the FAA initiate a systematic and comprehensive research program to study the effects of introducing new air traffic control technology, including automation, on the accomplishment of safety and efficiency goals. This program should include attention to the ways in which both formal and cultural organizational context factors contribute to or detract from the effective introduction of the new technology and, conversely, the ways in which the FAA's organizational context may be modified as a consequence of the new technology. The FAA should conduct further research to clarify, within the context of air traffic control, the ways in which formal and informal organizational context factors affect one another and the ways in which they both affect the performance of controllers. Specific study should be undertaken of informal teaming and procedures during high-tempo operations with the goal of eliciting recommendations for ways in which formal procedures and organizational structures may be improved.

Conclusions The results of the FAA's biennial employee attitude survey (EAS) provide information about organizational culture and climate, including employees' perception of the extent to which the formal structure and practices meet their needs. The results of the most recent survey (1995) indicate that air traffic employees generally consider their jobs to be satisfying, but they are generally dissatisfied or only moderately satisfied with management practices and with the organizational context within which they perform their jobs. Results also indicate that a significant proportion of FAA employees are reluctant to express dissatisfaction or disagreement to their management; communication difficulties may therefore be influencing other indicators of job satisfaction.

Recommendations The panel recommends that the FAA utilize its employee attitude survey as a useful source of supporting information for studies of how formal and informal FAA organizational contexts affect one another and how they can affect controller performance. The surveys could be used to support detailed study of discrepancies between managers' and controllers' perceptions of organizational context factors; ways in which controller-management communications could be improved; ways in which different air traffic control options and geographic locations may form different subcultures with different perceptions; and ways in which the introduction of new technology, including automation, is perceived by controllers.

Conclusions Human factors activities within the FAA, including both research and practice activities, are extensive but fragmented. Research activities are not adequately coordinated across research centers, research is not always systematically performed to support system design needs, and research findings are not systematically applied. System design activities are sometimes uncoordinated and rely heavily on subcontractor efforts that are not managed by FAA human factors professionals.

Recommendations The panel recommends that the FAA focus the overall management of human factors research and development activities for the agency and provide the necessary authority, responsibility, and resources to ensure that such activities are systematically conducted and applied; that they are adequately coordinated across research centers (including government research organizations such as the National Aeronautics and Space Administration, the Department of Defense, and the MITRE Corporation); and that human factors contractor efforts are effectively overseen by FAA human factors professionals. We understand that the issue of human factors management within the agency is currently being addressed by a subcommittee of the FAA's Research, Engineering, and Development Advisory Council.

Airway Facilities specialists are critical partners of air traffic controllers with respect to supervisory control and restoration of the air traffic control system. They are responsible for such critical tasks as system monitoring, diagnostics, certification, maintenance, and restoration of equipment, systems, and services after outages.

Conclusions To date the introduction of new equipment has involved far more automation of Airway Facilities functions than of air traffic control functions. By comparison with the human factors efforts that have been devoted to air traffic controllers, attention to human factors characteristics of Airway Facilities operations, personnel, software, and equipment has been meager. Airway Facilities personnel are currently faced with a bewildering array of equipment and workstation devices that are provided by different vendors, apply different levels of automation, and present different computer-human interaction procedures and characteristics.

The trend toward centralized maintenance control centers has not alleviated this concatenation of equipment and workstation devices, which does not reflect the effective application of human factors analysis, design support, or evaluation. The decreasing reliability of national airspace system equipment, an associated requirement to develop creative workarounds, the lack of workstations designed using human factors principles, and a progression of large numbers of Airway

Facilities personnel toward near-term eligibility for retirement threaten to overwhelm efforts to continue to maintain the operation of the system. The recent creation of the GS-2101 automation specialist job classification acknowledges that Airway Facilities specialists are increasingly required to serve as systems engineers, but the job classification has not been accompanied by the development of validated selection or assignment procedures, performance assessment procedures tied to clear job criteria, or tailored training programs. Although the FAA has recently produced a human factors design guide applicable to Airway Facilities, the paucity of research and analysis to support continued maintenance and updating of knowledge bases for human factors applications to Airway Facilities is of deep concern.

Recommendations The panel recommends that the FAA significantly expand its application of human factors research, analysis, design, and test and evaluation activities to Airway Facilities. These activities should be directed toward the development of integrated workstations, teamwork and communication strategies, and procedures for selecting, assigning, training, managing, and assessing the performance of Airway Facilities specialists.

Human factors research should support long-range innovation by evaluating developmental concepts and should also serve in the solution of immediate design problems by assessing specific design options. To fulfill these imperatives, human factors researchers need to employ a wide array of information resources and study methods. The invention of methodological refinements and the precise determination of which methods to marshal for each research question as it arises confront the human factors research community as continuing challenges.

Conclusions The inherent complexity of the air traffic control system and the unpredictability of system error mean that all potential sources of human factors engineering data must be used in research and system design activities. In many instances, different methods or approaches to data collection must be combined in an integrated collection process, with one method, set of tools, or data source often complementing another. Most methods have inherent constraints or limitations. For example, databases of design guidelines do not always address future design issues; interpretation of accident analyses may be ambiguous; the databases of reporting systems are sometimes difficult to access and integrate and are not always available in user-friendly form; user opinions, either from surveys or from rapid prototype evaluations, are often biased; many air traffic control models are not validated, and existing models focus more on system performance than on human performance; studies using simulations must trade off precision and complexity with expense; and field studies impose particular constraints on the experimental control of variables. In all behavioral analysis techniques (i.e.,

all of the above except for modeling), there is a need to assess representative samples of users across all levels of expertise.

The representativeness of participating controllers, operational scenarios, and determinant variables ultimately determines the validity of the assessment data that are collected in air traffic control research. Contemporary advances in computer simulation have made it possible to undertake economical rapid prototyping in order to address most design questions, although such prototyping should augment, rather than substitute for, comprehensive real-time simulation with actual performance data.

Recommendations In order to overcome the current constraints on the usability of air traffic control reporting systems, the panel recommends accelerated efforts to provide user-friendly access to the aviation safety reporting system and other reporting systems. Human performance models should be developed and validated to support research and design activities in air traffic control system development. Part of the modeling effort should be to address the articulation of universally recognized quantifiable dimensions of controller performance, including dependent variables that define performance across the range of operational contexts.

The panel recommends systematic work to formalize the role and enhance the contribution of rapid prototyping in determining the characteristics of human-computer interaction. Whenever feasible, a cost-effective simulation capability should be included within design programs that will enable progressive acceptance testing and assessments of the risk of poor design, which may engender operational problems and necessitate costly redesign, to be carried well beyond preliminary rapid prototyping and into design validation activities. Early field testing should be conducted as a means of further mitigation of the risk of poor design. Additional human engineering standards and guidelines should be developed for design validation during system development to ensure that determinants of controller (and system) reliability are adequately addressed prior to system implementation.

Successful human factors programs closely link research and development activities to ensure that research activities are responsive to developmental needs and that research findings are applied in product development, testing and modification. Several factors are critical: extensive user input into the design process at all stages, often employing rapid prototyping; extensive involvement of human factors practitioners, who are knowledgeable about human factors design guidelines and about appropriate assessment techniques that capitalize on users' expertise; frequent opportunities for behavioral testing (not just expert opinion) of interfaces, and for refinement of those interfaces at several points throughout the

design cycle; and sensitivity to the special needs, wishes, and organizational cultures of different facilities at which technology will be ultimately introduced.

Conclusions Although specifications for new systems typically detail the functional and performance criteria for equipment, the human factors design specifications and guidelines, traditionally applied as part of the definition of system requirements, do not detail the functional and performance criteria for the human controllers who also constitute critical elements of the system. The evolution of human-computer interaction characteristics of new systems therefore often relies on user evaluations of these characteristics as the design progresses. User participation, however, is not a substitute for the expert knowledge of the human factors specialist; rather, the two should function as complementary team-mates during the acquisition process. One example of an apparently successful form of this team relationship is the center-TRACON automation system (CTAS), in which a key feature is the continuous involvement of customers and users in both the design and evaluation process. The result has been a useful set of tools for projecting and automatically sequencing aircraft approaches to airports that has been well accepted by controllers. Human factors analysis, test, and evaluation activities are important for both developmental and nondevelopmental items, including commercial-off-the-shelf items; the application of such items may involve both trade-offs of capabilities and transition issues that affect controller tasks.

Recommendations The panel recommends that the FAA include both representative users (e.g., controllers and maintainers) and human factors specialists on product acquisition and development teams. Also, they should systematize user inputs to the design process according to human factors procedures for performing analyses and for conducting prototype and pilot trials to inform user requirements and assess the likely impacts of design features on controller task performance and workload.

Prototyping, simulation, and modeling exercises—carefully designed by human factors specialists and supported by user participation—should be applied to assist in making design trade-offs and to distinguish between user preference and usability. The FAA should use such studies to refine and tailor existing research databases and design standards and to coordinate studies and results with operational data and recommendations derived from the FAA's safety databases and systems effectiveness databases. Human factors analysis, test, and evaluation activities should be applied, for a given application, during the acquisition process for nondevelopmental and commercial-off-the-shelf items to ensure that they are compatible with the capabilities and limitations of users.

There has recently been a great deal of discussion of the concept of human-centered automation, which is viewed by many as a critical issue to the successful

introduction of automation. Unfortunately, however, there are many different attributes that researchers have identified with this concept, and not all of these are consistent with one another. Furthermore, not all of them are necessarily consistent with the goal of attaining the best (i.e., safe and expeditious) system performance. Detailed consideration of the human centered automation concept as it applies to air traffic control will form the core of the second phase of our work.

Conclusions A number of components of automation have been introduced into the air traffic control system over the past decades in the areas of sensing, warning, prediction, and information exchange. These automated systems have provided a number of system benefits, and the attitudes of controllers have generally been positive. There are also a series of lessons that have been learned from other domains about the appropriate and inappropriate implementation of automation as it affects the human user or supervisor of that automation. Of particular concern is the human being's possible loss of alertness and awareness of automated functions and system functioning, which may become critical if sudden manual intervention is necessary. Humans may distrust the automation because they fail to understand its complexities, and it is possible that reliance on automation may lead to a loss of human proficiency in the skills that the automation replaces.

Recommendations The panel recommends that lessons learned from other domains be carefully heeded in the further introduction of air traffic control automation and that research be pursued to establish the generalizability of the research findings to the air traffic control domain.