Air Traffic Control Facilities: Improving Methods to Determine Staffing Requirements: Improving Methods to Determine Staffing Requirements -- Special Report 250 (1997)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

4.vi W4. - SL REPORT 250 AIR TRAMC CONT'ROL FACILITIES 1APROVING AETfloDs To DETIRAINE STAfFING REQuIREAENTS TRANSPORTATION RuARdll BOARD NATIONAL RESEARCH Comm

TRANSPORTATION RESEARCH BOARD 1997 ExEcunvE COMITTEE CHAIRSIAN: DAVID N. WORMLEY, Dean of Engineering, Pennsylvania State University, University Park VICE CHALItsIAN: SHARON D. BANKS, General Manager, AC Transit, Oakland, California EXECUTIVE DIRECTOR: ROBERT E. SKINNER, JR., Transportation Research Board BRIANJ. L. BERRY, Lloyd Viel Berkner Regental Professor, University of Texas at Dallas LILLIAN C. BORRONE, Director, Port Commerce Department, The Port Authority of New York and NewJersey, New York City (Past Chairman, 1995) DAVID G. BURWELL, President, Rails-to-Trails Conservancy, Washington, D.C. E. DEAN CARLSON, Secretary, Kansas Department of Transportation, Topeka JAMES N. DENN, Commissioner, Minnesota Department of Transportation, St. Paul JOtIN W FISHER, Director, ATLSS Engineering Research Center, and Professor of Civil and Environmental Engineering, Lehigh University, Bethlehem, Pennsylvania DENNISJ. FITZGERALD, Executive Director, Capital District Transportation Authority, Albany, New York DAVID R. GOODE, Chairman, President, and CEO, Norfolk Southern Corporation, Norfolk, Virginia DELON HAMPTON, Chairman and CEO, Delon Hampton & Associates, Chartered, Washington, D.C. LESTER A. HOEL, Hamilton Professor, Department of Civil Engineering, University of Virginia, Charlottesville JAMES L LSMMIE, Director, Parsons Brinckerhoff, Inc., New York City BRADLEY L. MALLORY, Secretary of Transportation, Commonwealth of Pennsylvania, Harrisburg ROBERT E. MARTINEZ, Secretary of Transportation, Commonwealth of Virginia, Richmond JEI'EREYJ. MCCAIG, President and CEO, Trimac Corporation, Calgary, Alberta, Canada MARSHALL W MOORE, Director, North Dakota Department of Transportation, Bismarck CRAIG F. PHILIP, President, Ingram Barge Company, Nashville, Tennessee ANDREA RINIKSR, Deputy Executive Director, Port of Seattle, Seattle, Washington JOHN M. 5,tsfljxr5, Vice President—Operating Assets, Consolidated Rail Corporation, Philadelphia, Pennsylvania WAYNE SHACKELFORD, Commissioner, G&rgii Department .of Transportation, Atlanta Ls STERMAN, Executive Director, East-West Gateway Coordinating Council, St. Louis, Missouri JOSEPH M. SUSSMAN, JR East Professor and Professor of Civil and Environmental Engineering, Massachusetts Institute of Technology, Cambridge (Past Chairman, 1994) JAMES W VAN LOBEN SELs, Director, California Department of Transportation, Sacramento (Past Chairman, 1996) MARTIN WACHS, Director, University of California Transportation Center, and Professor of Civil Engineering and City and Regional Planning, University of California, Berkeley DAVID L. WINSTEAD, Secretary, Maryland Department of Transportation, Baltimore-Washington International Airport, Maryland MIKE Acorr, President, National Asphalt Pavement Association, Lanham, Maryland (ex officio) Roy A. ALLEN, Vice President, Research and Test Department, Association of American Railroads, Washington, D.C. (ex officio) JOE N. BALLARD (Lt. Gen., U.S. Army), Chief of Engineers and Commander, U.S. Army Col ps of Engineers, Washington, D.C. (ex officio) ANDREW H. CARD,JR., President and CEO, American Automobile Manufacturers Association, Washington, D.C. (ex officio) THOMASJ. DONOHIJE, President and CEO, American Trucking Associations, Inc., Alexandria, Virginia (ex officio) MORTIMER L. DOwNire, Office of the Secretary, U.S. Department of Transportation (ex officio) FRANCIS B. FRANCOIS, Executive Director, American Association of State Highway and Transportation Officials, Washington, D.C. (ex officio) DAVID GARDINER, Assistant Administrator, Office of Policy, Planning and Evaluation, Environmental Protection Agency, Washington, D.C. (ex officio) JANE F GARVEY, Acting Administrator, Federal Highway Administration, U.S. Department of Transportation (ex officio) - ALBERTJ. .HERBERGER (Vice Adm., U.S. Navy, retired), Administrator, Maritime Administration, U.S. Department of Transportation (ex officio) T. R. LSKSHMANAN, Director, Bureau of Transportation Statistics, U.S. Department of Transportation (ex officio) GORDONJ. LINTON, Administrator, Federal Transit Administration, U.S. Department of Transportation (ex officio) RICARDO MARTINEZ, Administrator, National Highway Traffic Safety Administration, U.S. Department of Transportation (ex officio) WILLIAM W. MILLAR, President, American Public Transit Association, Washington, D.C. (ex officio) JOLENE M. MoLrroRIs, Administrator, Federal Railroad Administration, U.S. Department of Transportation (ex officio) DHARMENDRA K. SIIARMA, Administrator, Research and Special Programs Administration, U.S. Department of Transportation (ex officio) BARRY L VALENTINE, Acting Administrator, Federal Aviation Administration, U.S. Department of Transportation (ex officio)

S PEC it RPOR1 250 AN Ifif f IC CON1901 f ACI 11110 IMPROVING AETHODS TO DITIRAINI STAMING REQuIREAENTS Comm iftee To Study the federal Aviation Administralion's Methodologies for [stimating A i r T r a f f i c Controller Staffing Standards Transporafion Research Board Nafional Research Council Nafional icodemy Press Washington, D. C. 1997

Transportation Research Board Special Report 250 Subscriber Category V aviation Transportation Research Board publications are available by ordering indi- vidual publications directly from the TRB Business Office, through the Internet at http://www.nas.edultrb/index.html, or by annual subscription through or- ganization or individual affiliation with TRB. Affiliates and library subscribers are eligible for substantial discounts. For further information, contact the Transportation Research Board Business Office, National Research Council, 2101 Constitution Avenue, NW., Washington, D.C. 20418 (telephone 202- 334-32 14; fax 202-334-2519; or e-mail kpeterse@nas.edu). Copyright 1997 by the National Academy of Sciences. All rights reserved. Printed in the United States of America. NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special compe- tencies and with regard for appropriate balance. This report has been reviewed by a group other than the authors accord- ing to the procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The study was sponsored by the Federal Aviation Administration of the U.S. Department of Transportation. Library of Congress Cataloging-in-Publication Data Air traffic control facilities: improving methods to determine staffing requirements / Committee to Study the Federal Aviation Administration's Methodologies for Estimating Air Traffic Controller Staffing Standards, Transportation Research Board, National Research Council. P. cm. ISBN 0-309-05966-6 1. United States. Federal Aviation Administration—Officials and employees. 2. Air traffic controllers—United States—Forecasting. 3. Air traffic control—Safety regulations—United States. I. National Research Council. Committee to Study the Federal Aviation Administration's Methodologies for Estimating Air Traffic Controller Staffing Standards. II. Series: Special report (National Research Council (U.S.). Transportation Research Board) ; 250. TL725.3.G6147 1997 387.7'40426'0973—dc2l . 97-1188 CIP

Committee To Study the Federal Aviation Administration's Methodologies for Estimating Air Traffic Controller Staffing Standards AARON COHEN, Chairman, Texas A&M University, College Station CHARLES B. AALFS, Federal Aviation Administration (retired) RUSSELL A. BENEL, The MITRE Corporation, McLean, Virginia GEORGEJ. CouLuRis, Seagull Technology, Inc., Cupertino, California PIusJ. EGBELU, Iowa State University, Ames JoE D. HINSON, Federal Express, Memphis, Tennessee PAUL F. HOGAN, The Lewin Group, Fairfax, Virginia WILLIAM C. HOWELL, American Psychological Association Science Directorate, Washington, D.C. DONALD A. KIMBALL, Federal Aviation Administration (retired) THOMAS M. MCARDLE, SABRE Decision Technologies, Southiake, Texas NORMAN T. O'MEARA, Logistics Management Institute, McLean, Virginia PHILIPJ. SMITH, The Ohio State University, Columbus KAY M. STANNEY, University of Central Florida, Orlando Liaison Representatives CRAIG CAMPBELL, National Association of Air Traffic Specialists, Gainesville, Florida SUSAN GODBY HELZER, Federal Aviation Administration, Washington, D.C. ANNE MAyOR, Commission on Behavioral and Social Sciences and Education, Washington, D.C. ANDREW PITAS, Air Traffic Control Association, Arlington, Virginia RICHARD E. SWAUGER, National Air Traffic Controllers Association, Washington, D.C. Staff NANCY P. HUMPHREY, Study Director HAROLD P. VAN COTT, Van Cott and Associates, Consultant

PREfACE The appropriate level of staffing for air traffic control has long been contro- versial. Following the controller strike in 1981, which resulted in the firing of two-thirds of the controllers, congressional concerns about staffing, were fo- cused primarily on the overall size of the work force. In various congressional hearings, the Federal Aviation Administration (FAA) has been questioned about the rebuilding of the work force and whether the number of controllers is sufficiently large to handle the growth in air traffic since 1981.. Recently, congressional concerns have shifted to questions about whether staffing levels are appropriate at the agency's highest-activity air traffic locations. FAA has long had difficulty staffing its air route traffic con- trol centers (ARTCCs), terminal radar approach control (TRACON) facili- ties, and other terminal facilities in metropolitan areas such as New York, Chicago, and Los Angeles. In addition to being among the most demand- ing locations because of the volume and types of traffic, they are among the locations with the highest cost of living. Concerns about stressful work- ing conditions and the amount of overtime required of workers at these locations have also been raised regularly by the controllers' union and sometimes by members of Congress. FAA has developed staffing standards for estimating its current and future need for controllers. However, in reports to Congress the agency has ac- knowledged the limitations of using these standards to make precise esti- mates of staffing requirements at individual facilities. Hence, Congress re- quested this study of the development of a comprehensive methodology whereby FAA could determine the required number of controllers at each of its facilities. (The congressional request is quoted in full in Appendix A.) At the outset of the study, it was recognized and agreed by the sponsor and congressional staff that the National Research Council, acting on behalf of the National Academy of Sciences, could not actually develop a compre- hensive methodology. Rather, it would examine and provide guidance to FAA and Congress on the potential for developing such a methodology. It

vi PREFACE was also recognized that, although Congress posed a highly technical ques- tion, any study of methods to determine appropriate staffing levels would raise hotly debated questions about work rules, productivity, compensation, management practices, and other issues. Although these issues do affect total staffing levels, they extend well beyond the technical boundaries of this study and are not examined in detail as part of this report. To conduct the study, which was funded by FAA, the Transportation Re- search Board (TRB) formed a panel of 13 members under the leadership of Aaron Cohen, Zachry Professor of Engineering in the College of Engineer- ing at Texas A&M University. The committee includes experts in staffing methods and models, industrial engineering and operations research, econo- metrics, work load measurement and human factors, and air traffic control operations and management. Panel members—drawn largely from universi- ties, consulting firms, and private business—reached consensus on all the findings and recommendations, which are presented at the outset of the re- port. The report is directed toward a nontechnical audience. More technical descriptions of the staffing standards and critiques of them can be found in the appendixes. The committee wishes to acknowledge the assistance of many individuals who contributed to the study. The committee received numerous briefings on the development and use of FAA's staffing standards. Thomas Cullinane of Northeastern University and Gavriel Salvendy of Purdue University sum- marized the results of recent independent reviews of the staffing standards for ARTCCs and TRACONs, respectively. Craig DePauw and Michael Mc- Cormick of FAA explained how staffing requirements for individual facili- ties are determined in the Western-Pacific Region and for the New York TRACON, respectively. Susan Helzer, Peter Kovalick, and Elliott McLaugh- lin of FAA headquarters staff provided an overview of the staffing system and the mechanics of applying FAA headquarters staffing standards to estimate facility-level staffing requirements. Craig Campbell and Richard Swauger— representatives of the major bargaining units, the National Association of Air Traffic Specialists and the National Air Traffic Controllers Association, re- spectively—and Andrew Pitas of the Air Traffic Control Association, the pro- fessional association for air traffic control—provided the committee with their perspectives on the difficulty of estimating staffing requirements for in- dividual facilities. Finally, project staff interviewed several of the contractors who were involved in the development of the most recent staffing standards, including Neal Schmeidler andJohn D'Avanzo of Omni Engineering & Tech- nology, Inc., Michael Watson of KPMG Peat Marwick, Inc., and B. Allen Benn of Benn Associates, Inc. The committee also examined staffing standards in related systems and wishes to thank Roger Yates of the U.S. Army, Major Tim Holst and Sergeant Robert James of the U.S. Air Force, and Commander Frank Olic and Master

PREFACE vii Chief Petty Officer Kathleen Shanahan of the U.S. Navy, who described how staffing requirements are estimated for air traffic control at military facilities; Jack Knight, who described how staffing requirements are determined for flight controllers at the National Aeronautics and Space Administration's Johnson Space Center; and Kathy Fox, who provided information about the determination of staffing and scheduling requirements for air traffic controllers in Canada. The committee would also like to thank the staff of the air traffic control facilities it visited for their presentations on facility staffing and operations and informative tours. Committee members visited at least one of each major type of air traffic control facility—the Southern California TRACON, the air traffic control towers at the San Diego International and Orange County Air- ports, the Washington Center, and the automated flight service Station at the Leesburg Municipal Airport. Nancy P. Humphrey, TRB Senior Staff Officer, managed the study and drafted major portions of the final report with the assistance of Harold P. Van Cott, a human factors expert and Chief Scientist of Van Cott and Associates. Joseph A. Breen, TRB Senior Program Officer for Aviation, provided infor- mation on staffing standards in the U.S. military and assisted with the site visits and committee meetings. The report was prepared under the guidance of the committee and the supervision of Stephen R. Godwin, Director of Studies and Information Services. Suzanne Schneider, Assistant Executive Director of TRB, managed the report review process. The report was re- viewed by an independent group of reviewers in accordance with the National Research Council report review procedures. The final report was edited and prepared for publication under the su- pervision of Nancy A. Ackerman, Director of Reports and Editorial Services, TRB. Special appreciation is expressed to Norman Solomon, who edited the report, and to Marguerite Schneider, who assisted in meetings logistics and communications with the committee and provided word processing support for numerous drafts.

CON If N IS FINDINGS AND RECOMMENDATIONS . 1 INTRODUCTION .................................10 Request for Study and Scope of Work 10 The Airport and En Route Air Traffic Control System 11 Staffing Air Traffic Control Facilities 15 Outline of Report 20 2 CURRENT STAFFING PROCESS: DESCRIPTION AND SHORTCOMINGS ................... 22 Overview of Staffing Process 22 Problems in Application of Staffing Standards to Local Facilities 30 Conclusion 38 3 STRATEGY FOR AN IMPROVED STAFFING PROCESS .........40 Concept of an Improved Staffing Process 40 Improving the Headquarters Staffing Process 42 Improving the Regional Staffing Process 47 Developing a Headquarters-Level Oversight Process 47 Developing Performance Measures for Validating Facility Staffing Estimates 48 Learning from Related Systems 49 Summary 50 APPENDICES A Congressional Request for Staffing Standards Study ............54 B Calculation of Facility Staffing Requirements ..........55 C Review of Shortcomings in Current Staffing Standards for Application to Individual Facilities ........73 STUDY COMMITTEE BIOGRAPHICAL INFORMATION ............89

fINDINs AND R[COMM[NDAIIONS The Federal Aviation Administration (FAA) has developed quantitative models known as staffing standards to estimate the number of air traffic con- trol specialists (ATCSs) necessary to operate the nation's air traffic control system. In recent years, discrepancies between estimates provided by FAA headquarters, perceived local staffing needs, and actual staffing levels at spe- cific facilities have raised concerns about the accuracy of FAA's forecasts for individual facilities. The agency maintains that, whereas its mathematically modeled estimates are defensible in the aggregate and support the overall budgeting process, they are less precise at the individual facility level (FAA 1996, 4). Congress requested this study to determine whether a compre- hensive methodology could be developed whereby FAA could estimate the required number of controllers at each air traffic control facility. The committee formed to conduct the study accepted the technical charge—to examine and provide guidance to FAA and Congress on the po- tential for developing a comprehensive methodology. It recognized that the issue of appropriate staffing levels is. not simply a question of science and models but involves a long and frequently contentious debate over work rules, productivity, compensation, management practices, and other issues. These issues extend beyond the expertise of and charge to this committee. FINDINGS The committee met its charge by addressing three questions, the responses to which are summarized in this section. 1. Do FAA staffing standards provide estimates of facility-level staffing that can be used with a high degree of confidence? If not, why not?

AIR TRAFFIC CONTROL FACILITIES It is not possible to test the validity of the FAA headquarters staffing stan- dards at the facility level in an empirical, scientific way, because FAA does not have formal performance criteria or systematically collected measures of air traffic control system performance related to staffing against which to compare model-predicted estimates. For a model to have validity in the scientific sense, predictions of the model must be tested, directly or indirectly, against some external measure (i.e., criterion) of the "truth" of the predictions. In the case of FAA, a test of the validity of the staffing standards would require a comparison of the staffing predicted by the models with an external criterion related to staffing against which to judge the "correctness" of the predicted staffing. Theoreti- cally, the correct level of staffing is the level at which the increased cost of additional staffing is just equal to the value of the resulting increase in per- formance. At staffing levels that exceed this optimum, the resulting increase in performance is valued less than the increase in cost; at levels below this optimum, the increase in the value of performance from additional staffing exceeds the increase in cost. FAA, however, has no formal performance cri- teria related to staffing or any systematic way to measure air traffic control system performance that would allow determination of this theoretically cor- rect level of staffing. (Developing such criteria would be difficult because they involve measures of both safety and efficiency and value choices about the appropriate level of each.) In the absence of such criteria, staffing re- quirements predicted by the staffing standards are frequently compared with actual staffing levels at individual facilities, which, of course, are observable but uninformative. Actual staffing is not necessarily the correct or optimal level of staffing. In sum, the validity of the staffing standards at the facility level cannot be tested directly because there is no developed or agreed-upon measure of the true staffing requirement. The validity of the staffing standards could be tested indirectly, but again only if performance measures were available. For example, a shortage of staff predicted by the model could be tested against measures of performance at the facility. Such indicators as increased frequency of operational errors and increased delay times for aircraft are worth exploring for their links to staffing shortages. Greater use of overtime, less use of annual leave, and more time on position, among other measures, are worth exploring as possible compensatory adjustments to accommodate staffing shortages. At facilities where staffing exceeds the model predictions, such performance measures and compensatory adjustments should be in the opposite direction (e.g., lower-than-average delay time, reduced use of overtime). Hence the predic- tive capability of the model could be tested by correlating the predicted staffing shortages and surpluses with system performance or other measures, taking into account other factors that are known to affect air traffic control

FINDINGS AND RECOMMENDATIONS system performance. A high, statistically significant correlation would imply that the staffing standards are valid. However, FAA does not systematically collect system performance or other measures related to staffing at each fa- cility that would permit even an indirect, empirical test of the validity of the staffing standards. FAA headquarters staffing standards do not necessarily provide accurate predictions of staffing requirements at individual facilities because of the way the staffing standards were developed and are applied. The staffing standards were developed to provide aggregate estimates of staffing requirements at the national and regional levels for the purposes of budgeting and resource allocation.' Estimates for individual facilities are com- puted under the staffing standards and summed to regional and national to- tals. However, they are not intended to be highly reliable at the facility level. Strategies for sampling, data collection, and model design are geared to the development of national staffing estimates. For example, the con- troller work models that provide the basis for estimating staffing require- ments are built on aggregated data, pooled from sample sites, to develop generic models for each major type of air traffic control facility. Different models were produced for different work load situations (e.g., single- and multiple-controller work-time models for TRACONs) and different con- troller positions (e.g., radar controller and flight data controller), but the data inputs to the models were pooled across all of the sampled sites. Sam- ple sizes are only large enough for national estimates, and the models do not have parameters to account for unique local conditions and operating procedures. Once facility staffing estimates are projected, an allowance is applied to account for leave, off-position activities, and 7-day facility operation. The adjustment factor, which accounts for a large portion of the final staffing number, is based on averages of systemwide data. Furthermore, the process for determining staffing requirements has two steps involving FAA headquarters and the FAA regions, and the staffing stan- dards, as presently used, are only one component of that process. Using the headquarters model-derived, aggregate staffing estimates as a budget target, the regions develop their own estimates of facility-level staffing require- ments—often with facility input—to determine the final allocation of man- power resources to individual facilities within their respective regions. I The staffing standards cover only that part of the FAA air traffic work force that separates and con- trols air traffic and provides flight information services. Separate standards are available for each of the four major types of facilities—air route traffic control centers, terminal radar approach control (TRACON) facilities, airport traffic control towers, and automated flight service stations.

AIR TRAFFIC CONTROL FACILITIES 2. Can the current staffing standards be modified to provide more accurate estimates? It is unlikely that the current staffing standards can be modified to provide stand-alone estimates of individual facility staffing requirements at the level of precision implied in the congressional request. For the most part, the current models are based on simple, empirically ob- served relationships between controller work and the level of air traffic ac- tivity—the primary factor that drives that work.2 They do not attempt to ex- plain the complexity and dynamics of the air traffic control system, and they cannot, in part because they are not based on a solid theoretical understand- ing of the factors that contribute to differences in controller performance, such as operational characteristics (e.g., aircraft mix, number of climbing and descending flights relative to through flights), controller fatigue and stress, and individual differences among controllers. Nor are they based on an ade- quate amount of measurement data. Although more sophisticated models could be developed, model-driven estimates of staffing requirements are likely to deviate from perceived local staffing needs and actual staffing levels at certain facilities because staffing decisions are not driven exclusively by the staffing requirements, however well they are estimated. Other legitimate factors external to the models, such as difficulties in moving controllers to certain locations, are likely to produce continuing discrepancies between modeled estimates and staffing levels at some facilities. Finally, Congress cannot rely on facility-level model estimates as long as there is no way to validate the estimates. It would not be prudent at this time to make more than modest investments in refining current staffing standards in an effort to improve facility-level staffing estimates. in the first place, as previously discussed, there is no valid way to verify such improvements. Second, replacement and upgrading of air traffic con- trol equipment is under way, and in the longer term introduction, of ad- vanced technologies and related changes in operational procedures are likely to alter the way controllers work and may even change the theoretically op- timum staffing levels. It is not possible to forecast precisely when or how these changes will be introduced or what effect they will have on staffing requirements. They will probably require a major recalibration of current models, which are based on existing controller tasks. They could require an altogether different modeling approach. The exception is the simulation model developed to measure controller work time at air traffic control towers. The model does not cover all ATCS work activities, however.

FINDINGS AND RECOMMENDATIONS 3. What alternative approaches could be pursued? More credible and widely accepted estimates of facility-level staffing re- quirements are more likely to be achieved by improving the process for de- tennining staffing requirements than by attempting to focus on the devel- opment of improved model-based staffing standards. The committee prefers an approach that does not rely solely on models. Rather, it advocates a process that combines formal modeled predictions with less formal methods based on expert judgment concerning staffing require- ments at individual facilities. An improved process could be achieved by (a) strengthening the headquarters staffing estimation process to enhance the usefulness of the staffing standards, (b) developing a uniform regional approach for estimating facility staffing requirements that incorporates the "best practices" of individual regions, (c) establishing a headquarters- level oversight process for resolving differences between headquarters and regional estimates of staffing for individual facilities, and (d) developing performance measures related to staffing to provide criteria for testing the validity of facility staffing estimates. Current quantitative staffing standards should be retained and refined as one component of this improved staffing process. Although the current staffing standards cannot provide technically valid estimates of staffing re- quirements for individual facilities for the reasons previously discussed, their usefulness may be increased by making methodological improvements (e.g., larger sample sizes) that are known to enhance measurement precision in other contexts. Such improvements would enhance the usefulness of the staffing standards at headquarters for initial facility staffing estimation and for subsequent oversight purposes. An improved staffing process is not a substitute for an external measure (i.e., a defensible performance criterion) of the correct level of staffing at in- dividual facilities. However, it should help move the parties involved in mak- ing facility-level staffing decisions—FAA headquarters, the regions, and the facilities—toward consensus on appropriate facility staffing levels. A data tracking system consisting of a series of indicators of facility operation and performance could, in time, provide a systematic basis for determining whether the process is producing the correct level of facility staffing. RECOMMENDED STRATEGY The committee recommends the following strategy for improving the cur- rent staffing process. The strategy is based on several refinements and ad- justments to the current process. It seeks to reconcile formal "top-down" es- timates of staffing requirements from FAA headquarters with those generated

6 AIR TRAFFIC CONTROL FACILITIES in the regions by a less formal "bottom-up" approach (see accompanying diagram). Refine FAA'S current staffing standards and consider a new modeling ap- proach in the future, if needed. A model, even if presently incapable of validation at the facility level, is a valuable part of the headquarters component of the staffing process. Its function is to provide an initial point of reference from which to begin the process of reconciling staffing estimates. In the short run, the current FAA headquarters staffing standards, or refinements of them, should prove sufficient for this limited role. They would not, however, be sufficient to provide stand-alone estimates of facility-level staffing requirements. Within the current headquarters budget for staffing standards of approx- imately $500,000, pilot studies should be conducted to examine the poten- tial for refining the current models and - to explore the feasibility and cost of alternative modeling approaches. Modifications to the current staffing standards, such as the following, are worth serious consideration. I FAA i f Headquarters I Model-Driven Staffing Estimates i Performance I Measures to I Oversight Team to I Validate Staffing _ Reconcile Facility Facility Staffing Estimates Staffing Differences i Estimates Ftxpert Judgmentaffing Estimates i FAA Regions I with Facility Input I Elements of an improved staffing process.

FINDINGS AND RECOMMENDA TIONS Collection of more data and trial use of state-of-the-art clustering techniques to group facilities, or sectors3 within or among facilities, with like characteristics to support development of separate models for these groups if statistically significant differences are found. Inclusion of more of the factors that affect controller performance di- rectly in the models. Reexamination of systemwide allowances (for leave, off-position ac- tivities, 7-day facility coverage) for adjusting facility staffing estimates to de- termine whether allowances are used consistently in constructing the staffing standards and whether they adequately reflect operating differences across facilities and regions. Targeting of model improvement efforts on the facilities (a) that ac- count for the largest share of ATCS staffing and (b) where automated data are available to support model building. Consideration should be given to developing simpler, position-based models for smaller facilities, which operate with fewer positions and more fixed staffing patterns. A more sophisticated modeling approach should be considered in the longer run if pilot studies suggest that refinements to the current models can only be accomplished at high cost (millions of dollars) or if greater automation and new operational procedures result in major changes in the nature of the controllers' work. Investment in a more flexible model could be justified if changes in the air traffic control system are likely to render the current models obsolete. The committee is not aware of any available systems that can be adapted from other air traffic control operations or from related operations (e.g., airline operations, package express services) for application to FAA ATCS staffing. However, aspects of these systems, such as the military's position-based staffing approach for air traffic control and scheduling models (described in the text), could be examined for their application to the recommended staffing process. Whatever modeling approach is developed, more attention should be paid to careful documentation of model development and key model assumptions than has been evident in the development of the current staffing standards. In addition, FAA should present staffing estimates with statistical confidence intervals to indicate to the user the degree of precision with which the staffing estimates can be used. Under the leadership of FAA headquarters, in consultation with the re- gions and with input from the facilities, develop a uniform regional ap- proach to determining facility staffing requirements. FAA headquarters, in consultation with the regions and with input from the facilities, should adopt a uniform approach and a standard terminology For air traffic control purposes, the airspace system has been divided into many small, contigu- ous sectors. Each sector (or position) is the responsibility of an ATCS at a particular facility.

8 AIR TRAFFIC CONTROL FACILITIES and reporting format for providing regionally based estimates of facility staffing, which are one element of the overall staffing process. The most de- sirable approach for strengthening the regional component of the staffing process would be for headquarters to gather and share information with the regions on best practices in individual regions and, once consensus on an approach is reached, to establish it uniformly across the regions. FAA headquarters should provide staff support to the regions for this effort. It should assist in the development of common terminology, report- ing format, and information sharing, including examination of relevant approaches outside of FAA. It should also set a timetable for implementation of a uniform regional staffing process and provide training, if needed, on its use. Establish a headquarters-level oversight process to reconcile differences in facility staffing estimates. An oversight process—agreed upon by FAA headquarters, regional, and facility staff—is needed to review and reconcile differences between head- quarters and regional facility-level staffing estimates. The oversight function should reflect the expertise of those involved in making facility staffing decisions, including FAA regional and facility man- agers as well as the technical staff at FAA headquarters. The committee leaves to FAA discretion the determination of how best to organize and implement this function, whether as a permanent oversight team at headquarters or as an oversight team (perhaps drawn from various parts of the agency and sup- ported by headquarters technical staff) convened for a specified period to help reconcile facility staffing differences. The procedures and methods adopted by the oversight team should be well documented and clearly articulated to the regions and the facilities. They should also be uniformly applied to ensure an efficient and consistent distribution of resources to the regions and to facilities within the regions. Develop performance measures for validating facility staffing estimates. FAA should develop an information system to track the operating and per- formance characteristics of air traffic control facilities, building on data al- ready collected by the regions or headquarters. The system would include fa- cility information. Hours of operation, type and volume of traffic, and staffing characteristics are examples of facility information worth exploring for their links with staffing levels. The system would also include performance mea- sures. Sector staffing relative to aircraft activity levels, savings from reduc- tions in air traffic delays attributable to air traffic control service; and use of overtime are examples of performance measures worth pursuing. Information based on these data would be provided to the oversight team in a summary form to assist them in comparing the costs and service impli-

FINDINGS AND RECOMMENDATIONS 9 cations of different levels of facility staffing, which, over time, could provide the basis for validating facility staffing estimates. NEXT STEPS The committee strongly urges that FAA headquarters ensure that action is taken on the elements of the improved staffing process recommended in the preceding section. FAA headquarters should start with its own operations. It should develop a proposal for establishing an oversight function and examine the potential for improving its modeling capacity through pilot studies. FAA should be provided funding to conduct research on the links between air traffic, oper- ational characteristics, and controller performance to provide a more solid theoretical foundation on which to model staffing requirements. FAA headquarters should also provide leadership to the regions, manag- ing the information sharing on regional approaches for determining facility staffing requirements and on development of a facility-level data base and information system. The annual resource conferences attended by head- quarters and regional staff could provide one location for such an informa- tion exchange. The recommended strategy for improving the current staffing process will not result in statistically significant estimates of staffing requirements at each of FAA's facilities. However, it promises to produce staffing estimates that are more credible and more widely accepted than the current staffing approach and, most important, should help to provide a systematic basis for deter- mining whether the process is producing the correct level of staffing at each facility. REFERENCE FAA. .1996. Report to Congress: Air Traffic Controller Staffing Requirements. Report to the House Transportation Infrastructure Committee and the Senate Com- merce, Science, and Transportation Committee pursuant to Section 120 of Public Law 102-581. U.S. Department of Transportation, March.

INTRODUCTION All models are wrong but some are useful. —G.E.P. Box (1979) Air traffic controllers provide a vital service in ensuring the safe and effi- cient flow of air traffic through the nation's airspace. Since 1961, the Federal Aviation Administration (FAA), the agency with responsibility for the Na- tional Airspace System and its facilities, has used formal staffing standards to determine staffing requirements for air traffic control facilities. In recent years, variances between estimates provided by the staffing standards and actual staffing levels at specific facilities have raised ques- tions about the accuracy of FAA's forecasts for individual facilities. In par- ticular, difficulties in staffing some high-activity air traffic locations, such as New York, Chicago, and Los Angeles, have raised congressional con- cerns about FAA estimates of the required number of controllers at each air traffic control facility. In legislatively mandated reports to Congress (FAA 1996a; FAA 1994), FAA maintains that the methods it uses to estimate staffing levels are defensible in the aggregate—for national and regional projections—but are "inherently less reliable at the individual facility level" (FAA 1996a, 4). REQUEST FOR STUDY AND SCOPE OF WORK Congress expressed concern that FAA does not have a good grasp of how many controllers are required at each of its facilities (U.S. Congress 1995), a problem that could become more acute as the agency continues to downsize and experienced controllers retire.' Thus, Congress directed FAA through the National Academy of Sciences to "study the development of a compre- hensive methodology whereby the FAA could determine the required num- I About one-quarter of the most senior air traffic controllers become eligible for retirement in 1999. These controllers either did not participate in the strike in 1981 or were hired after it. Controllers must retire at age 56, although the Secretary of Transportation can grant an exemption until age 61 for a controller with exceptional skills and experience (Public Law 92-297). IN

INTRODUCTION 11 ber of controllers at each of its facilities" (Appendix A). The expert panel convened by the Transportation Research Board to undertake the study Reviewed the methodologies by which FAA estimates and applies its staffing standards, Examined the feasibility and cost of modifying agency staffing stan- dards and developing alternative approaches for application to individual fa- cilities, and Recommended an improvement strategy. in the following sections, a brief overview of the air traffic control system, the major facilities in which controllers work, and the nature of controllers' work is provided. An outline of the remaining chapters of the report is given in the final section. THE AIRPORT AND EN ROUTE AIR TRAFFIC CONTROL SYSTEM The nation's airspace system is a large complex of airports, airways, naviga- tional aids, and air traffic control facilities that supports the commercial, private, and military uses of aircraft. The air traffic control component of the system provides three general forms of service: (a) navigation aids in- cluding landing aids, (b) flight planning and in-flight advisory information, and (c) air traffic control (OTA 1982, 25). Its main function to airspace users is physical separation of aircraft from one another. The need for this service derives from the fact that under Instrument Flight Rules (IFR) and some Visual Flight Rules (VFR), pilots may be unable to see other aircraft in the surrounding airspace and need assistance to maintain safe separation. Air Traffic Control Facilities Two types of facilities provide air traffic control: air route traffic control cen- ters (ARTCCs) and terminal facilities (Figure 1-1). The former handle aircraft along air routes or over oceanic areas. The latter include airport traffic con- trol towers (ATCTs), which serve aircraft at and in the immediate vicinity of an airport, and terminal radar approach control (TRACON) and nonradar ap- proach control facilities, which manage the airspace surrounding airports. Centers Twenty-one centers provide for control and separation of aircraft flying be- tween destinations in the continental United States, and 3 offshore facilities

En Route 7 Tower M Center controllers clear the lane to land — Navigation Transmitter - on !gatIo ' Transmitter SurfaceN9ation En Rou 4 Center Inside the en route - center, flights are handed detection from controller to kirpo r controller, each Te1 overseeing a particular slice of airspace 6 As the plane nears its destination, control _ 3 The fit ht is then transmitters goes to a radar-approach controller _________ Terminal radar- taken ovjby an route send signals approach control centers across the to help pilots the who guides the plane to the runway (TRACON) nation stay on right air route Terminal surveillance radar -_.. Controllers in the airport tower give clearance and the plane taxis to the runway. Some airports have surface radar to help track planes on the ground, but most on-ground controlling is done by looking Out the windows, sometimes with binoculars For about the first 64 km (40 ml) of the flight, or until an altitude of about 3050 m (10,000 ft), the plane is guided by the Radar Approach Control FIGURE 1-1 Air traffic control facilities (FAA Air Traffic Operations Graphics).

INTRODUCTION 13 TABLE 1-1 AIR TRAFFIC CONTROL SYSTEM FACILITIES BY TYPE (P. KOVALICK, AIR TRAFFIC BRIEFING TO COMMITTEE, MAY 30, 1996) FACILITY TYPE NUMBER Centers Continental United States 21 Offshore 3 Total 24 Terminals TRACONs 28 Tower/TRACONs' 149 Limited radar towers 43 Nonradar approach control towers 8 VFR towers 151 Contracted towers5 94 Total 473 Flight Service Stations Automated facilities 60 Nonautomated facilities 31. Total 91 NOTE: TRACON = terminal radar approach control; VFR = Visual Flight Rules. These facilities have two distinct functional units—a TRACON and a tower cab (Schmeidler and D'Avanzo 1994, 12). 51n FY 1994 FAA began a program to contract out approximately 100 low-traffic-density towers (FAA 1996a, 7). handle traffic over certain islands and oceanic routes (Table 1-1). Each ARTCC typically monitors a geographic area of more than 259 000 km2 (100,000 mi2) of airspace (STI 1991a, 1) comprising multiple sectors through which traffic is controlled (OTA 1982, 37). The ARTCC airspace is subdivided into low-altitude sectors, which are used mainly by propeller- driven aircraft, and high-altitude sectors, which are used mainly by jet aircraft flying at 8230 m (27,000 ft) or higher. The primary function of a center air traffic controller is to ensure the sep- aration of flights between airports for those aircraft flying under IFR (STI 1991a, 1).2 When aircraft are in level cruise, traffic management is less de- manding and problems are minimal unless severe weather or adverse winds are encountered or there are many crossing routes within the sector. The sec- tors that are difficult to control are those near major airports, which have nu- merous ascending and descending flights serving major cities (OTA 1982, 37). When a major airport is operating at or near capacity or when severe 2 Other functions of ARTCCs involve provision of traffic advisories, gathering of information, mon- itoring of aircraft flying under VFR if the aircraft are in areas with radar coverage, and assistance to aircraft in distress (STI 1991a, 1).

14 AIR TRAFFIC CONTROL FACILITIES weather is present, the control task is even more demanding. Center con- trollers may be required to delay the passage of aircraft out of their sectors to meter traffic flow into terminal areas (OTA 1982, 37). I erminals More than 400 terminal facilities (Table 1-1) provide for control of both IFR and VFR aircraft traffic in the vicinity of one or more airports. By far the most numerous of these facilities—ATCTs—handle the separation and sequenc- ing of aircraft landing, taking off, and taxiing on the airport surface (STI 1991b, 1). ATCT controllers are the only air traffic control specialists who visually observe and separate aircraft, although at large airports they use radar to assist their visual observation (OTA 1982, 37). At smaller airports, aircraft entering or leaving control of an ARTCC may pass directly to control by an ATCT. However, for airports with moderate- to high-density traffic, TRACONs handle the arriving and departing traffic, sequencing and spacing arrivals for landing on active runways and some- times at more than one airport (STI 1991b, 1). TRACONs also direct de- parting aircraft along departure routes into en route airspace or other adja- cent TRACON airspace (OTA 1982,37). Approach and departure controllers at TRACONs are responsible for the separation of aircraft; they control air- craft progress and coordinate with the ARTCCs and other TRACONs, from which they receive or to which they hand off traffic, and with the ATCTs that handle takeoffs and landings at the airports. Other Facilities Flight service stations, rather than directly controlling air traffic, offer a wide range of other services, primarily to general aviation aircraft. The main ser- vices include performing flight planning activities, providing preflight and in-flight weather briefings, communicating with pilots en route, and assist- ing pilots in distress (STI 1994, 1). These services are provided by 60 auto- mated flight service stations (AFSSs) and 31 nonautomated facilities, which are being phased out (Table 1-1). Finally, the Air Traffic Control System Command Center—a one-of-a- kind facility serving the entire United States—plays a national role in re- solving systemwide problems by controlling the flow of air traffic to centers and ATCTs. The traffic flow is managed by restricting the number of air- craft entering the system, by adjusting spacing, and by holding or rerouting aircraft to resolve congestion, which usually results from bad weather.

INTRODUCTION 15 Sectors For air traffic control purposes, the airspace system has been divided into many small, contiguous sectors. Each sector is defined in terms of its horizontal and vertical extent and is the responsibility of an air traffic controller at a particu- lar facility (OTA 1982,36). A group of sectors makes up an area. ARTCCs have responsibility for several areas, whereas terminals typically are responsible for a single area.3 The exceptions are the large TRACONs—New York, Southern California, and Oakland—which have responsibility for multiple areas. Sectors have one or more assigned radio frequencies used by the aircraft operating in them. The progress of a flight at cruise altitude moving from one sector to another is monitored and potential conflicts with other air traffic are resolved. Upon reaching a sector boundary, the pilot is told to change fre- quencies and establish contact with the next controller. The controller must perform this sector boundary "hand ofr' according to strict procedures. The next controller must indicate his ability to safely accept the incoming aircraft and establish positive control when the pilot makes radio contact before relieving the first controller of responsibility for the flight. Sectors are designed to equalize work load. For example, sectors near busy airports are designed so that the arriving and departing traffic is channeled into corridors within which aircraft are spaced so as to arrive at sector bound- aries at regular intervals (OTA 1982, 36). If conditions warrant, delays can be imposed on the traffic to reduce the work load to manageable levels. From the facility perspective, the physical configuration of sectors is fixed, but how they are used and staffed is not. For example, depending on the den- sity of the air traffic and local facility management practices, sectors can be combined or operated separately with different levels of staff support. Cer- tain sector designations, such as arrival and departure sectors, are not fixed but can change with the flow of the traffic. In addition these designations are not mutually exclusive; arrival sectors may contain some departing traffic and vice versa. These variations in sector staffing, use, and designation over time, and from facility to facility, increase the difficulty of modeling facility- level controller staffing requirements. STAFFING AIR TRAFFIC CONTROL FACILITIES Today there are approximately 17,000 controllers nationwide in the FAA work force (P. Kovalick, air traffic briefing to committee, May 30, 1996). Terminals generally use the term 'position" to describe a "sector." The two terms are inter- changeable. For the purposes of this report, the term 'sector" rather than "position" is used throughout.

16 AIR TRAFFIC CONTROL FACILITIES FAA's staffing standards apply to controllers whose primary responsibility is the separation and control of aircraft or the provision of flight information services—the air traffic control specialists (ATCSs);4 they do not apply to su- pervisory personnel. The specific work requirements of ATCSs and the staffing needed to perform them can vary widely from facility to facility and also within each facility depending on the time of day, the day of the week, and the season of the year. Staffing standards have been developed for each of the major types of air traffic control facilities to account for some of these differences. Facility Operating Requirements ATCS staffing reflects the operating needs of different facility types. At centers, depending on the amount of traffic, some sectors can be staffed with up to four ATCSs—a radar controller, a data or manual controller, a handoff controller, and a coordinator (STI 1991a, 5).5 Two-person sector staffing (i.e., radar and data controller) is the most common arrangement (STI 1991a, 17), and, on the average, each of the centers is staffed with between 200 and 300 ATCSs (FAA 1996b). Terminals at the larger airports generally have two operating environ- ments—the radar room and the ATCT. At the radar room or TRACON, de- pending on the amount of traffic, each sector can be staffed with up to three ATCSs—a radar controller, a handoff controller, and a coordinator.6 In addition, one or two flight data positions are generally staffed within the TRACON to provide support for all the sectors.7 It is usually not practical for more than three ATCSs to work one sector at a TRACON because of the con- figuration of the controller workstation (STI 1991b, 43). The most common arrangement is one-person sector staffing (i.e., radar controller) (STI 1991b, About 14,400 ATCSs who work in centers and terminals and about 2,500 ATCSs who work in AFSSs are included (P. Kovalick, air traffic briefing to committee, May 30, 1996). The radar controller is responsible for all communication with aircraft in the sector. The data or manual controller has responsibility for posting, marking, and sequencing flight progress strips— paper strips with flight plan data that must be moved to appropriate operating positions as an air- craft progresses in its flight—and providing any other necessary assistance to the radar controller. The handoff controller concentrates on monitoring the radarscope, accepting and initiating hand- offs, and performing keyboard and other control functions while the radar controller focuses on communicating with the aircraft. Finally, the coordinator helps monitor aircraft activities, oversee the controller operations, and monitor the radar screen, and provides assistance to the other con- troljers as required (STI 1991a, 5-6). 6 The radar and handoff controllers perform the same functions as their counterparts in the ARTCCs. The coordinator is responsible for coordinating arrival and departure flows among the TRACON's sectors, with adjacent approach controls, and with the ARTCC (STI 1991b, 8-9). The flight data controllers are responsible for posting flight strips at the appropriate controller po- sitions, verifying and monitoring the most recent flight data information, relaying weather reports, coordinating clearances, and maintaining hourly traffic records (STI 1991b, 9).

INTRODUCTION 17 43). Two-person sector staffing is preferred at some of the largest TRACONs.8 The very largest TRACONs are staffed with 200 or more ATCSs; the smaller TRACONs have one-fifth to one-quarter that number (FAA 1996b). The ATCT is staffed somewhat differently. The primary control functions include local control, ground control, flight data, and clearance delivery.9 The staffing and use of these positions is highly dependent on the specific airport configuration, the number of runways, user demand, and local air traffic patterns. The ATCTs can also vary widely in staffing. The large tow- ers have ATCS staffing levels of between 50 and 100, and the small towers have ATCS staffing levels of between 10 and 30 (FAA 1996b). The special role of AFSSs—to provide a wide range of services to general aviation pilots and other facilities within the air traffic control system—calls for different ATCS positions. Two of the positions assist general aviation air- craft while the pilots are flying; the remaining positions assist pilots before or after flights and also handle communications with other air traffic control facilities (STI 1994, 5). Up to eight positions may exist within a large AFSS, with ATCS staffing levels of approximately 75, but the positions typically are combined in smaller facilities and even in the larger AFSSs as the traffic fluctuates (STI 1994, 7). Factors Affecting Controller Work The primary factor affecting controller work is the demand created by air traffic, that is, the volume of air traffic in a sector and the resulting density (i.e., number of aircraft handled at one time) (GAO 1988, 22). From a staffing perspective, the peaks in demand are as important as or more im- portant than total volume. Adequate numbers of controllers must be avail- able to cover the peaks in traffic caused by weather and daily, weekly, or seasonal variations. Peak traffic periods increase the potential for conflicts among aircraft in a sector that must be resolved by the controller. The availability of additional personnel during these periods is important so that additional positions (i.e., handoff and coordinator controllers) can be staffed, allowing the radar con- troller more time to issue and observe control instructions and resolve po- tential conflicts. Additional personnel can also enable the opening of adja- cent sectors to help handle the work load. If the additional personnel are not 8 On the basis of a discussion of sector staffing in a presentation by Michael McCormick, Assistant Air Traffic Manager for the New York TRACON, at the third Committee meeting on September 19, 1996. Local controllers handle an aircraft's takeoff and landing. Ground controllers control the ground movement of aircraft to and from the runways and ground-based vehicles. The flight data and clear- ance delivery controllers are responsible for processing IFR clearances and providing information on air traffic and weather (Schmeidler and D'Avanzo 1994, 15-16).

18 AIR TRAFFIC CONTROL FACILITIES available, traffic flow is restricted or aircraft are held outside of the affected area to ease the work load. Thus system efficiency is reduced to avert any ad- verse effects on safety. in addition to traffic peaking, which can be caused by weather as well as traffic volume, complexity of air traffic operations is often mentioned as an- other factor affecting controller performance and the need for additional staffing (NATCAi'FAA Classification and Compensation Workgroup 1996, 12). Complexity refers to the physical aspects of a sector (e.g., its size, geo- graphic dimensions, airway configuration) and the movement of air traffic through that airspace (e.g., the number of climbing and descending flights, through flights, and aircraft mix) (Mogford et al. 1995, 3).10 Many of these factors are said to interact to increase air traffic control complexity as traffic density increases (NATCA/FAA Classification and Compensation Work- group 1996, 13). For example, an airport may have a complex runway configuration, but at low traffic volumes the ATCS's work remains routine. Only when the airspace becomes congested and the traffic increases does the difficulty of handling a complex runway configuration manifest itself. Over the years considerable research has been conducted to define the key factors that contribute to air traffic control complexity and to measure their effect on controller performance (Mogford et al. 1995, 2). Numerous complexity factors have been defined, but successful modeling and quan- tification of their effects on controller work have proved elusive (Mogford et al. 1995, 19). Moreover, air traffic control complexity is not experienced or handled in the same way in all situations. Controller responses to com- plexity are affected by the cognitive strategies the controller uses to process air traffic information, the quality of the equipment, and individual differ- ences between controllers, including age and experience (Mogford et al. 1995, 4). The difficulty of adequately measuring the effects of complexity on con- troller performance has implications for the adequacy of staffing standards, particularly at the facility level. For example, it would be useful to develop statistical estimates of how long controllers can work rapidly and without error under various traffic conditions in diverse levels of sector complexity. Such information could help explain more of the variability in the time con- trollers take to perform the same tasks at different facilities and at the same facility under different conditions. The information would also be useful in designing methodologies for observing and measuring controller work under 10 Other operational factors can affect complexity (NATCAIFAA Classification and Compensation Workgroup 1996). For centers, these factors include the number of terminals and the density of traffic at terminals in the center's control area and areas adjacent to the center's airspace, terrain fea- tures, and restricted and military operating areas (p. 34). For terminals, they include airport con- figurations (runway and taxiway layout, lengths, and capacities), provision of control services for secondary airports, and proximity of other airports (p. 13).

INTRODUCTION 19 different facility or sector conditions.'1 Without a better understanding of the factors that affect controller performance, the current staffing standards can only provide a gross measure of the differences in staffing requirements within and across facilities. Expected Changes in the Work Environment Expected changes in the air traffic control environment are likely to affect the nature of controllers' tasks and possibly the level of staffing. Certain parts of the long-delayed program to replace and upgrade air traffic control equip- ment are expected to be implemented over the next several years. The first priority is to replace aging equipment with new equipment of greater speed and capacity.'2 The equipment will not bring revolutionary changes in how controllers work, but it should increase their productivity. It should also pro- vide more automated data about controller communications and air traffic activity that would be useful in developing staffing standards. In the longer term, greater automation and related changes in operating procedures could pave the way for changes in the way air traffic is controlled. Advanced communication, navigation, surveillance, and air traffic manage- ment technologies could allow greater flexibility in airspace use and associ- ated alterations in controller and pilot responsibilities and procedures. For example, technological advances could allow pilots to determine their own flight routes, speeds, and altitudes, shifting much of the responsibility for air traffic control from the controller to the pilot (Planzer and Jenny 1995).' Today, aircraft must fly in designated air corridors controlled by ATCSs. II For example, it could help define the appropriate length of time for observing controller work and the number of observations needed. Many of the controller work-time models in the current staffing standards are based on observing controllers for 15-min intervals. This may not be long enough, or it could be too long in some cases, depending on the average time of the tasks being observed. 12 FAA has divided this modernization program into three components. The Display System Re- placement for the ARTCCs will provide high resolution color radar screens that automatically present a graphic picture of the sector map and of aircraft information. The Standard Terminal Automation Replacement System for TRACONs will replace existing TRACON computers and workstations—a prerequisite for further automation programs. The High Availability Basic Tower Control Computer Complex for ACTCS will replace workstations and supporting computers (FAA 1996c). ° Some technological advances under consideration include (a) advanced sensing and communi- cations systems that would provide precise and continuous information both to the cockpit and to the appropriate air traffic control facility (e.g., air-ground data links would transmit controller in- structions, aircraft position and status data, meteorological data, and other information between controllers and pilots); (b) the Global Positioning System, which would support precise navigation by aircraft in oceanic, en route, and terminal airspace; and (c) automated air traffic control systems that would support aircraft conflict prediction and resolution, sequencing and spacing, and other traffic management functions.

20 AIR TRAFFIC CONTROL FACILITIES Automated air traffic control systems may change the role of the controller and therefore the requirements for staffing. However, until these systems are operationally tested and procedures of use are established, changes in staffing requirements would be 'difficult for FAA to forecast. Nevertheless, the need for staffing standards that are flexible and adaptable to changes in the air traffic control system should be kept in mind as alternatives to the current staffing standards are considered. OUTUNE OF REPORT The remainder of this report examines the question posed by 'Congress— whether a comprehensive methodology can be developed to improve the ac- curacy of FAA estimates of the required number of controllers at each of its facilities. In Chapter 2, the current staffing process is described and the rea- sons why the current staffing standards do not provide reliable facility esti- mates are identified. A strategy for an improved staffing process to develop more credible facility-level staffing requirements is elaborated in Chapter 3. REFERENCES ABBREVIATIONS FAA Federal Aviation Administration GAO General Accounting Office NATCA National Air Traffic Controllers Association OTA Office of Technology Assessment STI Standards Technology, Inc. Box, G.E.P. 1979. Robustness in the Strategy of Scientific Model Building. In Robust- ness in Statistics (R.L. Launer and G.N. Wilkinson, eds.), Academic Press, New York. FAA. 1994. Report to Congress: Controller Staffing Requirements. Report to the House and Senate Appropriations Committees pursuant to House Report 102-156, House Report 102-639, and Senate Report 102-35 1. U.S. Department of Trans- portation, April. FAA. 1996a. Report to Congress: Air Traffic Controller Staffing Requirements. Report to the House Transportation Infrastructure Committee and the Senate Com- merce, Science, and Transportation Committee pursuant to Section 120 of Public Law 102-581. U.S. Department of Transportation, March. FAA. 1996b. ATCS Comparisons of Staffing Standards to On-Board. 1995 edition. U.S. Department of Transportation, May 30. FAA. 1996c. Aviation System Capital Investment Plan. U.S. Department of Trans- portation, Jan. GAO. 1988. FAA Staffing: Improvements Needed in Estimating Air Traffic Controller Requirements. GAO-RCED-88-106. June, 66 pp.

INTRODUCTION 21 Mogford, R.H., J.A. Guttman, S.L. Morrow, and P. Kopardekar. 1995. The Complex- ity Construct in Air Traffic Control: A Review and Synthesis of the Literature. DOTIFAAJCT-TN95/22. CTA Inc., Pleasaniville, N.J., July, 24 pp. NATCAIFAA Classification and Compensation Workgroup. 1996. Position Classifi- cation Standard for Air Traffic Control. Series ATC-2153. Aug., 1354 pp. OTA. 1982. Airport and Air Traffic Control System. OTA-STI-175.Jan., 141 pp. Planzer, N., and M.T.Jenny. 1995. Managing the Evolution to Free Flight. Journal of ATC, Jan—March, pp. 18-20. Schmeidler, N.F., andJj. D'Avanzo. 1994. Development of Staffing Standards for Air Traffic Control Functions in Tower Cabs. Technical Report. .DTFAO1-89- Y-01037. Operational Technologies Services, Inc., Vienna, Va., and OMNI En- gineering & Technology, Inc., McLean, Va., Sept. 21. STI. 1991a. FAA Staffing Standardsfor Air Route Traffic Control Centers. Validation and Revision Report. DTFAO1-88-Y-01016, Work Order 5. Rockville, Md., Nov. 15. STI. 1991b. FAA Staffing Standards for Terminal Radar Approach Control Facilities. Technical Report. DTFAO1-88-Y-01016, Work Order 1. Rockville, Md., Nov. 15. STI. 1994. FAA Staffing Standards for Automated Flight Service Stations. Technical Re- port. DTFA01-87-Y-01016. Rockville, Md., Nov. 15. U.S. Congress. House Committee on Appropriations. 1995. H. Rept. 104-177, on H.R. 2002, Department of Transportation and Related Agencies Appropriations Act, July 11.

PA CURRENT STAfflN6 PROCESS: DESCRIPTION AND SHORTC0MIN6S A cen' tral mission of the Federal Aviation Administration (FAA) is to man- age airspace and control air traffic operating in that airspace. The number of air traffic controllers required to perform this mission safely and efficiently is a critical issue for the agency. Since the early 1960s, FAA has used formal staffing standards to forecast staffing requirements for its air traffic control facilities. FAA's staffing standards, including their purpose and current use, are described in this chapter. The standards are discussed as part of a broader system of estimating staffing requirements and making manpower alloca- tions involving FAA headquarters as well as the FAA regions and local facil- ities. In the second part of the chapter, reasons why the staffing standards do not provide more accurate estimates of facility-level staffing requirements are given. OVERVIEW OF STAFFING PROCESS Congressional concern about the adequacy of FAA's estimates of controller requirements for individual facilities has focused on the staffing stand- ards developed by FAA headquarters. In fact, facility-level staffing deci- sions are made in a two-step process (Figure 2-1). In the first step, FAA headquarters applies the staffing standards to estimate regional re- quirements for air traffic control specialists (ATCSs). The estimates pro- vide the basis for allocating nationally budgeted funds for staffing to the FAA regions. In the second step, the regions develop independent estimates of facility-level staffing requirements, typically with input from the facili- ties. The regions determine the final allocation of staffing among the facil- ities, staying within the aggregate budgeted funds initially provided by FAA headquarters. 22

I STEP1 I FAA Headquarters Develops National and Regional Staffing Estimates Based on Staffing Standards Each Region in Proportion to Staffing Standard Estimates, Adjusted for Actual Funding Available + Can Request Changes to FAA Headquarters Funding Allocation + Negotiates Final Funding Allocation with FAA Headquarters Develops Estimates of Facility-Level Staffing Requirements Within FAA Headquarters Funding Allocation + Negotiates Staffing Levels and Allocates Staffing Among Individual Facilities FAA Facility FIGURE 2-1 Overview of FAA staffing process.

24 AIR TRAFFIC CONTROL FACILITIES FAA Headquarters Staffing Process The staffing standards are the centerpiece of the FAA headquarters process of estimating ATCS staffing requirements. What Is a Staffing Standard? According to FAA, a staffing standard is a mathematical model that incorpo- rates one or more equations based on empirically derived measurements of the times required to perform a specified set of behaviorally observable tasks (S. Helzer, air traffic briefing to committee, May 30, 1996). The models de- veloped for FAA relate the tasks required of experienced ATCSs to measures of aircraft activity—the primary factor that generates controllers' work. The models estimate the time needed to accomplish the tasks and the resulting staff required, taking into account work peaks and scheduling constraints (STI 199 la, 5). Allowances for annual and sick leave, holidays, training, off- position activities, and other required noncontrol functions are included (STI 1991a, 5). Use of the term "standard" is a misnomer. It implies an external criterion or benchmark against which the staffing levels predicted by the models can be judged. At present, there is no way to verify that the models predict the "correct" or "optimum" number of ATCSs relative to a standard of perfor- mance (e.g., level of service provided, efficient use of controller personnel). Estimates are often compared against on-board staffing as a measure of their validity. The actual staffing levels, however, merely reflect current staffing, which is not necessarily the correct or optimum level of staffing. On-board staffing can be low or high relative to modeled estimates depending on local management practices, recent retirements, and persistent staffing imbal- ances. The comparison does not offer any insight into how many ATCSs are needed to meet some performance goal. Development of Current Staffing Standards Staffing standards are sometimes used by government agencies to ensure pro- ductive use of agency personnel. In response to executive branch and agency directives, FAA has used formal standards to determine its staffing require- ments for air traffic control facilities for nearly 35 years (GAO 1988, 14,56).' The standards have evolved from rather simple formulas (the original 1961 A brief history of FAA's staffing standards through 1985 is provided in a GAO report (1988, 56-59).

CURRENT STAFFING PROCESS 25 standard), to more scientifically based standards using modified industrial engineering techniques to measure controller work load (first introduced in 1973), to today's more sophisticated computer models (GAO 1988, 14, 56). The current staffing standards were developed in the late 1980s. They cover that portion of the work force specifically engaged in the tasks of sep- arating and controlling air traffic—the ATCSs. Separate staffing standards are available for each of the four major types of facilities—air route traffic con- trol centers (ARTCCs), terminal radar approach control (TRACON) facili- ties, airport traffic control towers, and automated flight service stations (AFSSs). The steps invoN'ed in constructing the staffing standards are similar for each of these facility types, although the specific modeling and statistical ap- proaches differ (see text box and Appendix B for a lengthier discussion of how the staffing standards work). The standards are designed to provide con- servative estimates. For example, controller work is observed during busy times when staffing is higher than during average conditions; the staffing for- mulas provide for controller staff coverage for 90 percent of observed busy work time and for 90 percent of expected daily peak air traffic conditions; and the adjustment factor—for time off position, leave, and training—is based on average rather than peak conditions, yielding a larger factor. The results of applying the standards—ATCS staffing forecasts—are up- dated annually by FAA as new facility work load data and published agency aviation forecasts of air traffic become available.2 Agency evaluations of the representativeness of the work-time models and the staffing adjustment factor are conducted less frequently, typically on a 5- to 6-year cycle. Objectives and Use of Staffing Standards According to FAA (S. Helzer, air traffic briefing to committee, May 30, 1996), staffing standards are used at the national level for planning, budgeting, and resource allocation. More specifically, they provide An objective too13 for determining and forecasting staffing require- ments; A guide for efficient and equitable distribution of staffing resources; 2 Air traffic forecasts are produced annually by FAA's Office of Aviation Policy and Plans and are usually published early in the calendar year using data from the preceding fiscal year. The staffing standard forecasts are published shortly after the official FAA aviation forecasts, usually by April or May (FAA 1996a, 1). The staffing standards are characterized as objective, but the reader should be aware that any model has a certain amount of subjectivity built in through the selection of model form, inclusion or exclusion of specific variables, and choice of data inputs.

FAA HEADQUARTERS STAFFING STANDARDS— How THEY WORK IN BRIEF In general, staffing estimates are built on empirical measurements of controller work as a function of aircraft activity (S. Helzer, air traffic briefing to committee, May 30, 1996). Time study and work sampling are the primary methods used to measure controller work times on specific tasks for short intervals (typically 15 mm), mainly during busy work times. Work-time or simulation models are developed that relate observed work time to work load measures, using data collected from a sample of facilities. Model adjustments are made to ensure that staffing requirements are adequate to cover 90 percent of total ob- served work time. Then the models are applied to each facility, using detailed data on facility traffic, to estimate the amount of time needed to handle the work load for each period of the day and the resulting staff required. Once detailed staffing estimates are available, a scheduling algo- rithm is applied to ensure adequate staffing to cover daily shift sched- ules and allowances for lunch and breaks. The result is a daily facility staffing estimate. Forecasting models are then developed, which relate different lev- els of air traffic to schedule-adjusted daily staffing requirements. Air traffic is forecast for a future day on the basis of anticipated growth in air travel, and the associated daily staffing requirements are estimated from the forecasting models. FAA has chosen the 37th-busiest day as its forecast day to ensure that staffing is adequate to handle a volume of air traffic equal to or greater than the volume of traffic on 90 per- cent, or the remaining 328 days, of the year (GAO 1988, 56). Facilities are expected to use overtime, if necessary, to handle traffic on days that exceed the 90th percentile standard. Finally, adjustments are made to the daily staffing forecasts to take into account leave, off-position time, and 7-day facility operation to produce annual facility staffing requirements (S. Heizer, air traffic brief- ing to committee, May 30, 1996).' At present a 1.76 adjustment factor is used to convert daily into annual facility staffing requirements. A separate pipeline model, based on historical data on controllers entering and leaving the work force, is used to compute the training pipeline needed to meet the staffing standard— estimated requirements.

CURRENT STAFFING PROCESS 27 A basis for informing management of how resources are used; and A baseline for "what if" analyses of future scenarios, including pro- posed program changes, expansion, or reduction; consequences of not staffing at full levels; and impacts of using overtime. In practice, FAA headquarters staffing standards are used primarily as a budgeting and resource allocation tool. The agency uses the staffing stan- dards to generate multiyear forecasts of staffing requirements to support budget requests to Congress. Once funds are made available, the estimates provide the basis for determining how staffing is to be proportionately al- located to each of the nine FAA regions (FAA 1996a, 4) (Figure 2-2). For example, if application of the staffing standards indicates that a certain region should have 15 percent of the national staffing requirements to handle air traffic in that region, then ideally the region should receive 15 percent of the total available resources .4 If the resources made avail- able through the budgeting process are less than the staffing standards indicate are needed, the allocations are factored down to match available resources.5 Although the staffing standards are applied to every facility to arrive at a total staffing requirement by region, they are not intended to provide a highly precise prediction of staffing requirements for individual facilities. As FAA noted in a recent report to Congress: Since the staffing standards process is based on statistical averages, it produces forecasts that are highly accurate at the aggregate (national or FAA regional) levels, but inherently less reliable at the individual facility level. This characteristic limits direct usage of the staffing standard forecasts at the facility level. (FAA 1996a, 4) Moreover, once the regions receive the nationally budgeted staffing allo- cations, they are responsible for determining how the staffing is distributed among individual facilities through a process described in the following sec- tion. Regional managers have the discretion to consider local operational The actual process is a little more complicated (P. Kovalick, air traffic briefing to committee, May 30, 1996). First, the staffing standards are applied on a regional basis to determine regional re- quirements for ATCS staff in centers, terminals, and automated flight service stations. Then staffing requirements for management and other controller positions for which staffing standards are not available are added and summed to a total for each region. When funding levels are identified, the budgeted amount is allocated in a lump sum to each region on the basis of its percentage of the total; allocations are adjusted when there are differences between staffing requirements estimated by the staffing standards and the actual funding to meet those requirements. In addition, regions can request changes to their allocations from headquarters both during annual review and on a continuing basis year-round. 5 The exception to this process is the recent handling of new hires. Instead of allocating the fund- ing for this purpose using the staffing standards allocation process, FAA headquarters provided the funds to locations that have had persistent staffing imbalances.

FAA Headquarters (HO) Develops Regional ATCS Staffing Requirements Applying the Staffing Standards FAA HO Adds Other Staffing Requirements to Develop Regional Totals + FAA HO Identifies National Funding Available for Air Traffic FAA HO Compares Budgeted Funds with Staffing Standard Estimates + F OAdjustsnal Totals ortionately FAA HO Provides Regions with Adjusted Lump Sum Staffing Allocations FAA HO Responds to Regional Requests for Changes to Staffing Allocations FIGURE 2-2 FAA headquarters staffing process.

CURRENT STAFFING PROCESS 29 requirements and special needs, which may result in facility staffing levels that differ from headquarters model-derived numbers (FAA 1996a, 5). Regional Staffing Process The second step in determining facility-level staffing allocations involves the air traffic division managers of FAA's nine administrative regions and the managers of the air traffic control facilities within these regions. In this step the local factors that affect the staffing requirements of individual facilities— factors not possible to incorporate into the national staffing process—can be taken into consideration. The actual processes used to estimate facility staffing requirements and make final manpower allocations vary substantially from region to region (see text boxes), but the activities and products common to all of them are shown in Figure 2-3. The process unfolds as follows. Typically, regional managers solicit facility staffing needs for the coming year from individual facility managers. In response, facility managers conduct needs analyses and WESTERN PACIFIC REGION STAFFING APPROACH The Western Pacific Region has developed a uniform position-based approach for estimating facility-level staffing requirements. Elements of the national model are used—selection of the 37th-busiest day for facility position staffing estimation and the 1.76 adjustment factor— but the national work-time and forecasting models are not. The region also has a formal process for reconciling differences between facility requests and available funding. The staffing process worksas follows. Each facility is asked to iden- tify its 37th-busiest air traffic day for the preceding year. Then, for that day, the facility records the number of positions open during each hour of operation and estimates the number of shifts for each position. The result is then multiplied by 1.76 to provide for 7-day coverage and off- position activities. The final number is the annual staffing requirement for that facility. Facilities can request increased staffing and provide the justification as part of their staffing request. The results for each facility are examined by a regional executive board. Eight senior managers review the numbers and examine other factors such as facility use of overtime. The board then makes the final staffing allocations to all of the facilities within the region.

30 AIR TRAFFIC CONTROL FACILITIES SOUTHWEST REGION STAFFING APPROACH The staffing needs of this region are reviewed at least annually. The air traffic manager of each facility is responsible for identifying die staffing needs of the facility and reporting them to the regional office. This in- formation is considered, as well as facility-specific information such as hours of operation and number of operational positions, in determin- ing the facility ATCS staffing requirements. The ATCS staffing allocations are distributed to field facilities after regional allocations are received from headquarters. The number of po- sitions allocated by headquarters has recently been less than the iden- tified staffing requirements. The Southwest Region Management Plan- ning Board is responsible for determining the facility allocations. The process of distributing ATCS positions to the field facilities consists of identifying areas in the staffing requirements where reductions will have the least impact on mission accomplishment. Changes in a facil- ity's allocation from the preceding year are based on such factors as actual and anticipated changes in traffic levels and equipment. justify requests for increases in future staffing on the basis of such factors as airport expansion, use of an airport as a hub by a new or existing airline, and projected increases in air traffic. Once the regions receive facility staffing estimates and regional staffing al- locations from FAA headquarters, they review the facility staffing plans and forecasts. The regions then develop staffing proposals for individual facilities within their respective regions taking into account the constraints of the ag- gregate budget allocations provided by FAA headquarters. (The regions may request changes in the allocations from headquarters during annual review and on a continuing basis year-round.) The facilities generally review and comment on the individual staffing proposals, and the regions make the final facility allocations. The last step is sometimes conducted through a formal process in which regional and selected facility managers participate and agree on an allocation. Without an equivalent mechanism for reconciling head- quarters model-based estimates with regional estimates of facility-level staffing requirements, it is not surprising that the two are often at variance. PROBLEMS IN APPLICATION OF STAFFING STANDARDS TO LOCAL FACILITIES Congress has asked whether FAA headquarters could provide more accurate staffing forecasts at the individual facility level. The current staffing stan-

CURRENT STAFFING PROCESS 31 Regions Solicit Facility Staffing Needs Needs Analysis and Justify Regions Receive Facility Staffing Regions Review Estimates and Facility Staffing Overall Regional __ Plans and Staffing Allocation Forecasts from Headquarters Regions Develop Staffing Proposal and Solicit Facility Review + Facilities Review Regional Staffing Allocation and Comment Back to Regions + Regions Finalize Facility Allocations and Forward Them to Facilities FIGURE 2-3 FAA regional staffing process. dards were not developed for this purpose, but could they be modified to meet this objective? To address this question, it is important to understand why the current FAA staffing standards do not provide highly accurate staffing estimates for individual facilities nor are likely to do so. To seek answers, the committee examined the documentation provided by FAA on the staffing standards, interviewed the contractors who developed them, and were briefed on recent independent reviews of selected staffing standards (ErgoTech, Inc. 1995; Cullinane 1996). The explanation falls into

32 AIR TRAFFIC CONTROL FACILITIES two general categories—the characteristics of the staffing standards them- selves and the staffing process of which the models are only one part. A more detailed discussion of these issues can be found in Appendix C. Model-Related Factors According to several of the contractors engaged by FAA to develop the latest round of staffing standards, the standards were designed to generate national estimates. The sampling design, data collection, and model development were geared to this purpose with important implications for the precision of facility-level staffing estimates. Level of Aggregation Analytic models are used at several points in the estimation of staffing re- quirements (Appendix B) 6 One of the crucial models that drives the staffing estimates—the controller work-time model—uses highly aggregated data to explain controller work time as a function of aircraft activity. Sampling schemes, including site selection and data collection, were geared to pro- ducing national estimates. The key data inputs to the models—sample data on controller work times and aircraft activity—were pooled across all of the sample sites to produce generic models for each major type of air traffic con- trol facility.7 The aggregation of the data raises two issues. The first is a statistical issue. Both of the independent reviewers questioned the validity of pooling data from a limited number of sites without testing to ensure that the sites are rep- resentative of the larger underlying population (Ergolech, Inc. 1995, 65; Cullinane 1996, 52). The second point relates more directly to the develop- ment of more accurate facility-level staffing estimates. Treatment of the data as a single population, even if found to be sound statistical practice, averages out facility differences (GAO 1988, 23; FAA 1996a, 4). This eliminates vari- ances that would have been desirable to explain in estimating differences in controller work time at the facility level. Some effort was made in the devel- First, work-time models are developed that relate the time controllers take to perform certain tasks to aircraft activity—the primary factor that drives the level of controller work. Second, a schedul- ing model is used to determine shift schedules, given facility scheduling constraints (e.g., shift starts, lunch breaks) and the number of controllers required during a 24-hr day. Finally, forecasting mod- els are developed that relate staffing requirements to expected levels of aircraft activity. Different work-time models were produced for different work load situations (e.g., single- and multiple-controller work-time models for TRACONs), but in each case the data input to the mod- els were pooled across all of the sites sampled.

CURRENT STAFFING PROCESS 33 opment of the staffing standard for TRACONs to examine the potential for developing separate work-time models for different subsets of the data (e.g., organized by sector size and tpe). The characteristics of the subgroups, however, were not sufficiently distinctive to justify development of separate models (STI 1991a, 36). Simple Relationships The current staffing standards, particularly the work-time models, are based on simple relationships that do not adequately explain the operating com- plexities and dynamics of the air traffic control system. Data on operational characteristics were collected at sample sites in the data-gathering phase of model development.8 These factors, however, were not included in any of the final work-time regression models. Measures of aircraft activity were found to be the best predictors of controller work time; measurements of opera- tional, complexities, either singly or as a group, did not provide additional ex- planatory power and hence were excluded in the final models (STI 1991a, 40; STI 1991b, 24). With a limited number of variables, it is not surprising that the work-time models are relatively poor predictors of the variation in observed control- ler work times. For example, the single-controller model developed for TRACONs explains only 46 percent of the variation in controller work time (ErgoTech, Inc. 1995, 25). Thus, an arbitrary, uniform allowance for operat- ing complexity or difficulty, called a difficulty factor,9 was added to the mod- els to compensate for their poor predictive capability. In effect, this factor ensures that staffing requirements are adequate to control peak work load 90 percent of the time.'° However, it does not help explain potentially important sources of variance across facilities. Adjustment Factors Most of the discussion so far has concerned the inadequacies of the current methods of estimating controller work times. Once the staffing requirements needed to cover the work times estimated by the models are calculated, the 8 Examples of these characteristics for TRACONs include, among others, sector type, number of radar sectors, runway configuration, number of primary runways, and airspace restrictions (STI 1991a, 37). 9 The difficulty factor was determined by (a) calculating the standard deviation of all unexplained and unmodeled differences between the model and the total observed work time and (b) multiply- ing the resulting standard deviation by a 90 percent confidence factor. 0 A 95 percent confidence factor is used for the ARTCC single-controller models (STI 1991b, 26).

34 AIR TRAFFIC CONTROL FACILITIES results are adjusted by a factor that nearly doubles the staffing numbers. The currently used 1.76 adjustment factor takes into account 7-day facility operation. It also provides time for off-position activities, such as leave, off- position training, physicals, and union activities. The current adjustment factor was not developed in a way that is con- sistent with the other estimates in the staffing standards. For example, staf- fing requirements are calculated by measuring controller work during busy traffic periods. By comparison, allowances for time off position are based on averages of systemwide data. The result is an upward bias in the staffing estimates for all facilities. Use of the adjustment factor, however, does not appear to introduce a bias that would affect staffing estimates for one facility differently from another—the primary concern of this study. More relevant to the facility staffing estimation issue, use of a single na- tional adjustment factor does not reflect operating differences across facilities. The validity of the adjustment factor for different types of facilities could be tested and the effects of using a different factor for different facility types ex- plored." This approach could lead to staffing estimates more closely matched to facility operating practices, but it raises equity issues. For example, a study of the New York TRACON (ERAT 1995) found that controllers at that facil- ity are available to work 1,529.7 hr out of a 2,087-hr work year. The equiva- lent national figure is 1,665 hr, or 9 percent more. If the average 1.76 multi- plier is adjusted upwards for certain facilities to reflect differences in controller availability, resulting in higher modeled staffing estimates for these facilities, other facilities could legitimately ask why they do not receive equal treatment. Accuracy of Air Traffic Forecasts Air traffic forecasts affect the development of more accurate facility-level staffing estimates in several ways. FAA's current staffing standards include forecasting models that are developed by pooling data on facility operational characteristics and peak-period traffic from a sample of facilities. The result- ing models—two for TRACONs and four for terminals—attempt to repre- sent more facility types than the work-time models but still may not ade- quately explain facility-level differences.'2 The exceptions are the ARTCCs and the AFSSs; each facility has its own forecasting model based on facility- level air traffic data. II FAA staff have looked at different scenarios for TRACONs using different assumptions about con- troller availability, time on position, inclusion of a difficulty factor and other allowances, and se- lection of different forecast days—a 50th percentile to a 95th percentile busiest day. The results were staffing estimates for TRACONs ranging from 4,000 to 7,500. 12 In addition, more verification is needed that the current models adequately represent the differ- ent facility types and that the results can be generalized to all facilities within a certain type.

CURRENT STAFFING PROCESS 35 The more critical problem occurs in forecasting future year air traffic. To predict the staffing required to meet future year traffic levels, baseline data on facility traffic for the 90th percentile or 37th-busiest day of the preceding year are gathered. The appropriate forecast factor from FAA's forecasting sys- tem is then applied. The latter is an interactive system that combines econo- metric and time series model results with aviation industry forecasts, expert opinion, and anticipated policy impacts to derive a set of FAA aviation fore- casts (FAA 1996b, VIII-6). Although FAA short-term forecasts have been quite accurate at the national level in recent years,13 changes in airline ser- vice or in local or regional economic conditions can affect the accuracy of staffing forecasts for specific facilities over the course of the year (FAA 1996a, 6). Staffing requirements are unlikely to reflect the changes until the subsequent year. The choice of the 37th-busiest day as the forecast target has also been questioned. Its use has been traced to the original 1961 staffing standard, but no scientific basis could be found (GAO 1988, 56). It appears to have been chosen to ensure that staffing would be adequate to handle peak conditions. A recent sensitivity analysis by FAA staff (FAA 1994) has shown that, at least for TRACONs, selectiOn of a 70th or a 90th percentile busiest day would have made little difference in staffing requirements at specific facilities.'4 More- over, it would not likely affect one facility forecast differently from another, the concern of this study. Example The New York TRACON provides an example of the difficulty of tailoring staffing standards to individual facilities. As part of an analysis of operational errors at the New York TRACON, a regional assessment team concluded that FAA's national staffing standard for TRACONs was not applicable to the New York facility and proposed an alternative model (ERAT 1995). Application of the Eastern Region Assessment Team (ERAT) model predicted controller staffing at levels approximately one-quarter higher than those predicted by the FAA staffing standard-244 versus 192 ATCSs.15 The reasons for the differences are useful in illustrating the difficulties of applying the national staffing standards reliably to local facilities. Three fac- 13 For example, FAA forecasts of domestic air traffic were within ± 1 percent of actual traffic for fiscal year 1995 (FAA 1996b, V1I1-1). 14 See Appendix C for a more detailed discussion. 15 The model used to derive the 192 staffing figure deviated from the typical TRACON staffing stan- dard because it used area rather than facility traffic counts to project estimated traffic on the 90th percentile busiest day as recommended by ERAT (ERAT 1995, 2-4).

36 AIR TRAFFIC CONTROL FACILITIES tors account for most of the differences. First, more controllers are some- times required for a sector than the national staffing standard assumes. The New York TRACON often operates some of its sectors with two controllers, whereas the FAA staffing standards assume that one-controller sectors are the norm. According to one manager at the facility, a reason for the differ- ence is the severely congested and constricted airspace in the New York area. The New York TRACON serves three major airports within a 16-km (10-mi) radius, a situation that places heavy demands on controllers but constrains more efficient organization of the airspace (ERAT 1995, 220).16 Second, the New York TRACON uses different shift schedules and longer breaks than are assumed in the national staffing standards. ERAT recom- mends using an adjustment factor rather than the staffing standard schedul- ing algorithm used by 'FAA headquarters, which results in a larger staffing number (FAA 1995, 10). Together, the first two factors account for about two thirds of the overall difference between the national and ERAT estimates of staffing requirements. Third, ERAT believes that the allowance for off-position activities, such as leave, training, and union activities, should be larger than that assumed in the national factor of 1.76 (ERAT 1995, 2-20).11 The more facility-sensitive ad- justment factor of 1.91 recommended by ERAT produced nearly a 10 percent increase in staffing requirements. The regional assessment team concluded that the national staffing stan- dard is not valid for the New York TRACON and recommended the addi- tional staffing projected by application of the ERAT model (ERAT 1995, 2-19-221). System-Related Factors Even if FAA headquarters staffing standards could be modified to provide more accurate facility staffing estimates, there are likely to be continuing dis- crepancies between FAA headquarters model estimates and regionally de- termined requirements because of the way the staffing standards are used in the current staffing decision process. 16 According to ERAT, most TRACONs service a primary airport centered within the geographical confines of allotted airspace. The New York TRACON has three major airports within a 16-km (10- mi) radius, and the primary airport is located at the extreme edge of the areas' airspace configura- tions. This Situation creates a heavy work load for controllers, particularly in periods of congested traffic, and precludes airspace optimization (ERAT 1995, 2-20). ° FAA staff, however, contend that the ERAT adjustment factor exceeds typical FAA policies. For example, it allows for 100 percent use of both sick leave and annual leave.

CURRENT STAFFING PROCESS 37 Use of Headquarters Staffing Standards As indicated at the outset of this chapter, the staffing standards are only one input to the process of estimating staffing requirements and making facility- level staffing decisions. Discrepancies between facility staffing estimates produced by FAA head- quarters and those determined by the regions reflect the different processes used by the FAA regions to determine facility staffing requirements. Within the aggregate staffing budget provided by FAA headquarters, the regions ex- ercise their authority to allocate staffing on the basis of their knowledge of operational circumstances at individual facilities. Moreover, each FAA region has its own way of developing staffing estimates and allocating funds for positions to its individual facilities. Regional practices could be made more uniform, but without a process for reconciling headquarters with regional estimates of individual facility staffing requirements, differences are likely to persist. External Factors Model-based estimates of staffing requirements are inherently imprecise. Staffing decisions depend on a variety of legitimate considerations that can- not readily be reflected in any model. For example, budgetary cutbacks can result in fewer staff than might be estimated using either the FAA head- quarters models or regional staffing estimation methods. Conversely, spe- cial congressional appropriations can result in placement of additional con- trollers at specific locations. FAA itself has handled its new hires in FY 1996 "off-line" (i.e., without using the staffing standards), allocating the addi- tional positions directly to those locations where ATCS positions have been difficult to fill. Because of relatively limited new hires, insufficient funds to move experienced controllers from one facility to another, and the contin- uing difficulty of moving controllers to high-cost and low-desirability loca- tions, discrepancies between forecast requirements and actual staffing are likely to continue. Expected changes in the air traffic control system itself, as described in the preceding chapter, may alter the tasks that ATCSs perform, with impli- cations for staffing levels and requirements. It is not possible to forecast when these changes will be made or how large an effect they may have on staffing requirements, but the coming changes raise the issue of whether a major investment in improving the current staffing standards is prudent at this time.

38 AIR TRAFFIC CONTROL FACILITIES Lack of Validation Measures Until an external criterion is developed against which estimates of staffing requirements can be compared, there is no objective basis on which Con- gress, FAA headquarters, or the FAA regions can determine whether a spc- cific facility has the optimum number of ATCSs, whatever staffing standard or staffing approach is used. The current FAA headquarters staffing stan- dards have not been validated in this sense, and such validation is not easy. As the New York TRACON example shows, estimates of staffing require- ments may differ depending on assumptions about local management prac- tices (sector staffing) and facility operating circumstances. These differences are not likely to be resolved without a way to relate different levels of staffing to performance measures, such as the cost and level of service provided. Ad- mittedly this is a difficult task, because these measures involve safety and ef- ficiency and value judgments about the appropriate level of each. Neverthe- less, until such criteria are developed, disagreements over appropriate levels of facility staffing are likely to continue. CONCLUSION The committee does not believe that the current staffing standards can be used to provide highly accurate estimates of staffing requirements for indi- vidual facilities. The headquarters models could be improved, but it is un- likely that they could be modified sufficiently to provide stand-alone esti- mates of individual facility staffing requirements. Even if a better model could be developed, the committee believes that the current staffing process, which combines quantitative estimates with expert judgment, has consider- able merit. In fact, it is doubtful that modeling alone could provide an acceptable basis for establishing facility-level requirements. The headquarters staffing process is based on empirically derived data sampled from air traffic control facilities. It uses mathematical models and quantitative data to arrive at staffing estimates. The regional staffing process is based on expert, but subjective, judgments. The process varies from one region to another, and there is no uniform method for collecting the data used in making judgments of need. But it reflects the knowledge and judg- ments of those closest to the facilities. The headquarters process is useful for developing national budget re- quests and allocating resources to the regions. The regional process uses ex- pert opinion and negotiation among those who are knowledgeable about individual facilities and their needs to make the final staffing decisions. Each approach has strengths and weaknesses. Each has different func- tions. They appear to complement one another, and when used together they

CURRENT STAFFING PROCESS 39 could provide the basis for developing a better approach to estimating staffing requirements for individual facilities, the subject of the next chapter. REFERENCES ABBREVIATIONS ERAT Eastern Region Assessment Team FAA Federal Aviation Administration GAO General Accounting Office STI Standards Technology, Inc. Cullinane, T.P. 1996. A Review of Staffing Standards Development Procedures for FAA Air Route Traffic Control Centers. DTFA01-93C-00067, Task 1. Washington Consulting Group; FAA, U.S. Department of Transportation, Jan. 25,57 pp. ERAT. 1995. New York TRACON Controller Staffing. FAA, U.S Department of Trans- portation, Jan. 13, 21 pp. ErgoTech, Inc. 1995. Review of the FAA Staffing Standards for Terminal Radar Ap- proach Control (TRACON) Facilities. West Lafayette, Ind.; FAA, U.S. Depart- ment of Transportation, Feb. 1,87 pp. FAA. 1994. TRACON Staffing Standard Review: Analysis of 90th Percentile Day. U.S. Department of Transportation, Sept. 29. FAA. 1995. Review of New York TRACON Controller Staffing. Final Report. U.S. De- partment of Transportation, Feb. 28. FAA. 1996a. Report to Congress: Air Traffic Controller Staffing Requirements. Report to the House Transportation Infrastructure Committee and the Senate Com- merce, Science, and Transportation Committee pursuant to Section 120 of Public Law 102-581. U.S. Department of TIansportation, March. FAA. 1996b. FAA Aviation Forecasts, Fiscal Years 1996-2007. FAA-APO-96-1. U.S. Department of Transportation, March. GAO. 1988. FAA Staffing: Improvements Needed in Estimating Air Traffic Controller Requirements. GAO-RCED-88-106. June, 66 pp. STI. 1991a. FAA Staffing Standards for Terminal Radar Approach Control Facilities. Technical Report. DTFAO1-88-Y-01016, Work Order 1. Rockville, Md., Nov. 15. STI. 1991b. FAA Staffing Standards for Air Route Traffic Control Centers. Validation and Revision Report. DTFAO1-88-Y-01016, Work OrderS. Rockville, Md., Nov. 15.

3 STRAT[6Y FOR AN IMPROV[D SIAFFIN6 PR0C[ss The congressional request for more accurate estimates of controller staffing at individual facilities is best addressed by focusing on the two-part process by which staffing requirements are determined rather than on the Federal Aviation Administration (FAA) headquarters staffing standards alone. The committee does not believe that the current staffing standards can be modi- fied to provide stand-alone estimates of individual facility staffing require- ments at the level of precision that the congressional request appears to imply. Furthermore, even the best of estimates are likely to deviate from ac- tual facility staffing. Factors external to the models, such as difficulties.in moving controllers to certain locations and differences in facility manage- ment practices (e.g., shift scheduling and sector staffing policies), are apt to produce discrepancies between modeled estimates and actual facility staffing. On the other hand, the committee believes that the current process of de- termining and allocating controller staffing, which involves a headquarters "top-down" component and a regional "bottom-up" component, has merit. If both parts of that process were improved and a mechanism were added for resolving differences between headquarters and independently derived regional estimates of staffing requirements for individual facilities, there would be greater likelihood of consensus between FAA headquarters and the regions on appropriate facility staffing levels. in the following secuon, the committee's concept for an improved staffing process is described. Each element of a strategy to realize these improvements is then discussed. 'Where appropriate, alternative approaches are considered, including staffing methods in related systems that might be applicable. The chapter ends with a summary of the key elements of a desirable staffing process. CONCEPT OF AN IMPROVED STAFFING PROCESS The prospect of generating more credible and widely accepted staffing num- bers for each facility would be enhanced by establishing a formal and uni- 40

STRATEGY FOR AN IMPROVED STAFFING PROCESS 41 form process—agreed upon by FAA headquarters, regional, and facility staff—for estimating, reviewing, and reconciling facility staffing estimates. The process should build on the strengths of the current staffing system. It would retain the current system of checks and balances—the bottom-up staffing approach of the FAA regions and their facilities, which reflects the knowledge and expert judgment of regional and facility managers, with the top-down headquarters process, which offers a consistent basis on which to estimate staffing requirements across facilities, it would add an oversight function to help reconcile differences in the staffing estimates now developed separately by FAA headquarters and by the regions. Finally, it would involve development of a systemwide data base on the operational and performance characteristics of individual facilities for use as an information system by an oversight team and by regional and facility staff. With summary information on how staffing levels correlate with the performance of the air traffic con- trol system, the team would have external benchmarks against which to compare and validate facility staffing estimates. Each element of the improved staffing process, which is displayed in Fig- ure 3-1, is addressed in the following sections. I FAA Headquarters 7I Model-Driven i Staffing Estimates Performance I Measures to I Oversight Team to I Validate Staffing __... Reconcile Facility Facility Staffing Estimates Staffing Differences Estimates [—Expert Judgment I Staffing Estimates r F4AA Regions I I with Facility Input I FIGURE 3-1 Elements of an improved staffing process.

42 AIR TRAFFIC CONTROL FACILITIES IMPROVING THE HEADQUARTERS STAFFING PROCESS After considering several alternatives, the committee concluded that an im- proved headquarters staffing process should start, as it does now, with ini- tial headquarters and regional estimates of staffing requirements for all air traffic control facilities. This conclusion was based on the analysis that fol- lows, which supports the role of and continuing need for headquarters staffing standards. Need for Staffing Standards As an alternative to the current model-driven headquarters staffing process, the committee considered the elimination of staffing standards and their re- placement with a system under which staffing estimates would be based on expert judgment. FAA technical staff, drawn from headquarters and the field, would develop estimates of sector staffing requirements for individual facil- ities on the basis of their knowledge of operational characteristics, controller capabilities, and procedural requirements. Knowledge of work measurement techniques would also be valuable. For example, estimates could be based on historical data about facility staffing requirements adjusted for expected changes (e.g., in equipment, traffic, staff turnover) for the forecast year. In the absence of an objective criterion for determining the "correct" or "right" level of facility staffing, the judgment of experts offers a plausible al- ternative. Expert judgment is commonly used in many other fields—such as medical diagnosis—as a basis for making decisions. It is particularly appro- priate for decision making under conditions of uncertainty.' A form of ex- pert judgment is being used now, although in an informal way, by many of the FAA regions to make facility staffing decisions. The idea would be to create a similar but more uniform process at the headquarters level.2 In the committee's judgment, replacement of the current staffing standard with expert judgment maybe feasible, although the cost could come close to FAA's current annual budget3 for staffing standards of approximately 'The methods and techniques that have been developed to support such analysis are referred to as decision analysis (TRB 1996, 117-118). In fact, FAA headquarters used such a position-based approach for estimating facility staffing re- quirements before the 1981 controllers strike (GAO 1988, 50). However, at that time it was heav- ily criticized for being too subjective; there was no validation process to ensure that facility man- agers were not inflating their staffing requests (GAO 1988, 50). Currently FAA deploys five or six staff members and one manager from the Engineering Analysis Branch plus one or two persons from traffic operations to support the development and mainte- nance of staffing standards. Annual contract funding for these purposes has ranged from about $200,000 in Fiscal Years 1993 and 1994 to $500,000 in Fiscal Years 1995 and 1996 and exceeded $1 million when the staffing standards were revised in the late 1980s (S. Helzer, memorandum pre- pared for meeting of committee, July 26, 1996).

STRATEGY FOR AN IMPROVED STAFFING PROCESS 43 $500,000 because of the substantial data collection required. However, user acceptance of the approach, specifically its subjectivity, was perceived to be a major drawback. The committee concluded that an empirically based, mathematical model provides a consistent, quantitative basis to complement the judgment of experts and should lead to more objective and uniform fa- cility staffing estimates than would exist in the absence of such a model. Improvements to the Current Staffing Standards Once the decision was made to retain a model, the discussion turned to what modeling approach would be most appropriate. As observed earlier, the cur- rent headquarters staffing standards do not necessarily provide highly accu- rate estimates of staffing requirements at the facility level. The data in the work-time models, which provide the basis for estimating the staffing re- quirements, are aggregated so that the facility staffing estimates simply re- flect differences in projected air traffic and not differences in operating com- plexities. Nor is it possible to validate the estimates without an external criterion. It may be possible, however, to improve the models sufficiently to provide more useful input for FAA headquarters initial staffing estimates and for subsequent oversight purposes. Short- Term Options In the short run, FAA headquarters should continue to use the current staffing standards and examine the potential for model refinements. The use- fulness of the staffing standards may be increased by making methodologi- cal improvements (e.g., larger sample sizes) that are known to enhance mea- surement precision in other contexts. Pilot studies offer a low-cost (i.e., within the current FAA budget for staffing standards of approximately $500,000) way to explore the feasibility and cost of refinements. Pilot studies should be launched to explore modifications to the current staffing standards. For example, data could be collected at more facilities, ex- panding the sample size for statistical testing. State-of-the-art clustering tech- niques4 could then be used on a trial basis to group facilities, or sectors Cluster analysis refers to a set of techniques or algorithms that can be used to separate a set of ob- jects (e.g., air traffic control facilities) into groups on the basis of some attributes or charactensucs of the objects. The result of clustering must be unique; each object must belong to one and only one group. Clustering is a scientific tool widely used in various fields, including engineenng, med- icine, biology, statistics, artificial intelligence, and marketing. Its general purpose is to reduce or or- ganize a large volume of data into smaller sets of data with recognizable patterns that are amenable to analysis. in the fields of statistics and applied mathematics, clustering is used to support hy- pothesis generation and testing, model fitting, or model development to predict requirements or characteristics of object groups. .

44 AIR TRAFFIC CONTROL FACILITIES within or among facilities, with similar characteristics as the basis for sup- porting development of separate models for these groups if statistically sig- nificant differences among clusters are found. The many factors that affect controller work time could be screened to identify those highly correlated with differences in work performance. These variables could then be in- cluded in the work-time models.5 Finally, systemwide allowances (for leave, time off position, and 7-day facility coverage), which are applied as an ad- justment factor to staffing estimates, should be reexamined. The use of a sin- gle national adjustment factor cannot reflect operating differences across fa- cilities. Moreover, this factor was not developed in a way that is consistent with the other estimates in the staffing standards. Controller staffing re- quirements are calculated for busy traffic days (the 90th percentile busiest day), but important elements of the adjustment factor, such as the allowances for time off position, are based on averages, which could result in an upward bias in staffing estimates. Efforts to develop more facility-sensitive staffing standards should be focused on facilities with the largest staffing requirements—the air route traffic control centers (ARTCCs), large terminal radar approach control (TRACON) facilities, and large air traffic control towers (ATCTs)—and on facil- ities where automated data are available to support model building.6 More limited data collection and simpler, position-based models would probably be more appropriate for the smaller TRACONs and ATCTs, which typically operate with fewer positions and more static staffing patterns. Pilot studies could also be conducted to examine the feasibility and cost of alternative modeling methods, should ner approaches prove desirable in the longer term. Long-Term Options FAA headquarters should consider a more sophisticated modeling approach under two circumstances—first, if the pilot studies indicate that refinements to the current staffing standards can only be accomplished at high cost (mil- lions of dollars in addition to FAA's current budget for staffing standards), Without a better understanding of how these factors affect controller work, however, the likeli- hood of their successful inclusion in the models is low. 6 Cluster analysis and addition of more factors that affect controller performance may prove diffi- cult for some of these facilities, however, because of the difficulty of characterizing and distin- guishing sectors and their staffing levels. For example, traffic characteristics and work load distri- bution among the sectors at ARTCCs are dynamic; sectors characterized as arrival sectors in one period may have a mix of overtraffic and arrival traffic in the next period with different implications for controller work load. Also, sector staffing is more variable at ARTCCs, where it can range from one to four persons per sector.

STRATEGY FOR AN IMPROVED STAFFING PROCESS 45 and second, if increased automation and procedural changes in the air traf- fic control system result in major changes in the nature of controllers' work that would likely render the current models obsolete. One method that warrants consideration is a task analysis—based model- ing approach. A major limitation of FAA's current staffing standards is their insensitivity to the time-varying work load factors7 associated with individ- ual sectors. A solution to this problem is to develop a modeling approach based on more detailed analyses of controller work tasks as they relate to work load variables (see text box). Such an approach would involve a de- tailed description of system functions and controller tasks and their interac- tion in a dynamic environment. Inherent in such an analysis is a dynamic model of air traffic control operations that can be reflected in a computer emulation.8 This advanced modeling system would be automated to accommodate the large volume of data processing required to represent sectors, traffic, environ- mental, and many other work load variables that controllers respond to as they maintain aircraft separation. The most advanced form of the model would be driven by detailed descriptions of operations derived from electronic records of radar tracks and computer messages. A task analysis—based modeling approach should provide a far more ade- quate and flexible modeling tool than the current staffing standards. Sup- ported by a better theoretical understanding of controller work functions and tasks and automated data inputs that closely relate work to work load, the model should more accurately reflect controller work load and associated staffing requirements. Changes occurring in the air traffic control system would be reflected in the simulation model by changed model parameters eliminating the need for a major recalibration of the model and extensive new data collection. Because many types of data driving the model would be au- tomatically collected, the model could be updated in real time, eliminating the need for the periodic model recalibration studies that FAA currently con- ducts. A task-based model would be costly to develop. Several million dollars would be required to establish the automated system and collect the initial empirical task analysis information. Once the model was up and running, the cost of maintaining and updating it would probably be commensurate with Time-varying factors that contribute to facility differences include traffic volume, mix of cruising and transitional flights, and mix of aircraft with varying operating speeds and performance charac- teristics. simulation model was developed for the FAA staffing standards for ATCTs. However, the model estimated only one of the work elements of ATCT controllers—controller-pilot radio communica- tion (Schmeidler and D'Avanzo 1994, 34). The task analysis—based model would be more com- prehensive and would be based on a theoretical understanding of how controller tasks are related to work load.

46 AIR TRAFFIC CONTROL FACILITIES TASK ANALYSIS—BASED MODELING APPROACH Development of the model would require the following steps: Detailed descriptions of controller operations at representative sites and development of a systematic classification of controller tasks, sub- tasks, and work elements by sector (i.e., position) based on the descrip- tive information would be required. Typical work elements might consist of air-ground communications, interphone communications, face-to-face communications, data entry, display viewing, and flight strip processing. The durations of these observable work elements would be measured by trained observers at facility sites according to a sam- pling plan. From these observations, the frequency of occurrence and duration of each task, subtask, and work element would be calculated. Software that encodes the controller task analysis classification scheme would have to be developed to relate traffic loads and other complexity factors quantitatively to sector staffing requirements. Algo- rithms that specify sector staffing size as a function of traffic and other complexities would be defined. The algorithms would be calibrated on the basis of data describing actual facility operations; comprehensive historical records of activities should be available at ARTCCs and many TRACONs. Finally, the model would be structured for operation on a workstation or high-performance personal computer platform. Adjustments of the staffing estimates would be required for off- position activities and forecast traffic levels. Of course, the model would be calibrated and field tested before adoption. the current FAA budget for staffing standards (on the order of $500,000 an- nually). However, application of the model would be limited to facilities where automated data are available—currently the ARTCCs and possibly some TRACONs. Investment in a more sophisticated model could only be justified if pilot studies indicate that the cost of refinements to the current staffing standards would approach the cost of the new modeling method or if changes in the air traffic control system increase the availability of automated data for model- ing purposes and alter the nature of controllers' work in ways requiring major recalibration of the current staffing standards. For all options, more attention should be paid to documentation of model development and key modeling assumptions than has been evident in the de- velopment of the current staffing standards. In addition, FAA headquarters

STRATEGY FOR AN IMPROVED STAFFING PROCESS 47 should present model-developed staffing estimates with statistical confidence intervals9 to define for users, such as the oversight team or congressional staff, the statistical accuracy of the estimates. IMPROVING THE REGIONAL STAFFING PROCESS Whatever changes are adopted at the headquarters level, the regional ap- proach to developing estimates of facility staffing requirements should be re- tained and strengthened by the adoption of a uniform approach to deter- mining facility staffing requirements across the regions. As discussed in the preceding chapter, each region uses its own staffing estimation and allocation process. Some use elements of the headquarters staffing standard. For example, the Western-Pacific Region asks each facility to provide information on actual staffing for a certain traffic load day (the 90th percentile busiest day of the preceding year) to establish baseline staffing requirements. Other regions simply request input from facility man- agers about staffing needs. Similarly, in making final facility staffing alloca- tions, some regions have a formal process in which regional and facility man- agers participate and agree upon numbers. Other regions simply make the final decision. FAA headquarters should take the lead in providing staff support to the regions and facilities to develop a uniform regional-level staffing approach and reporting format. Because facility managers and staff often differ in their understanding of such terms as "authorized staffing" and "funding levels," the first step, which has already begun, is to develop a common terminology. The next step is to share regional approaches to developing facility staffing estimates, perhaps at the annual resource conferences that are held with re- gional and headquarters personnel. The most desirable approach would be for headquarters and the regions, with input from the facilities, to reach con- sensus on a "best practice" approach and move toward its uniform adoption across the regions. DEVELOPING A HEADQUARTERS-LEVEL OVERSIGHT PROCESS The current staffing system has no formal mechanism for reconciling differ- ences between FAA headquarters and regional estimates of facility-level Confidence intervals measure the likelihood, frequently at a 95 percent level of confidence, that the predicted mean for a random sample will lie within the interval bounds. A large confidence interval indicates a wide variability around the mean.

48 AIR TRAFFIC CONTROL FACILITIES staffing requirements. In an improved staffing system, a headquarters-level oversight process should be established to review, explain, and reconcile dif- ferences between headquarters and regional estimates. An oversight process would improve the likelihood of agreement among those involved in mak- ing staffing decisions on appropriate staffing levels at individual facilities. The oversight process should reflect the expertise of those involved in making staffing decisions, including FAA regional and facility managers as well as headquarters technical staff. Organizationally, the oversight function could be carried out in several ways. It could be centralized at headquarters as a continuing oversight team of carefully chosen individuals with expert knowledge and breadth of experience to make judgments about appropriate facility staffing levels. Or a team of headquarters, regional, and facility-level representatives, serving for 1 year or for a longer period and supported by headquarters technical staff, could be formed for this purpose. The commit- tee believes that the organizational structure of the oversight function and its implementation are decisions best left to FAA's discretion. However it is structured, the functions of the oversight team should be clearly separated from the FAA headquarters process of determining initial facility staffing requirements. The oversight group should be responsible only for adjudicating differences between headquarters and regional esti- mates of facility staffing requirements. It would not necessarily review every facility staffing estimate every year. Instead, it would focus on reconciling dif- ferences where regional estimates fall outside the staffing confidence inter- vals established by the headquarters staffing process or where data on facil- ity performance indicate a possible staffing problem. The procedures used by an oversight group, including any models and modeling assumptions, should be based on good quantitative techniques and human resource management practices. They should be clearly explained and accessible to those in the regions and facilities who are involved in staffing decisions. And they should be uniformly applied across regions and facilities to promote efficient and consistent distribution of staffing resources. DEVELOPING PERFORMANCE MEASURES FOR VALIDATING FACILITY STAFFING ESTIMATES To assist the oversight group in carrying out its functions, FAA should de- velop an information system based on systemwide data about the operating and performance characteristics of air traffic control facilities. The system would include facility information. Hours of operation, type and volume of traffic, and staffing characteristics [e.g., number of full-performance-level (FPL) controllers on board,10 attrition rates, training pipeline] are examples IS This term refers to controllers who are qualified on all sectors and whose major responsibility is the separation and control of air traffic.

STRATEGY FOR AN IMPROVED STAFFING PROCESS 49 worth exploring for their links with staffing levels. It would also include per- formance measures. Sector staffing relative to aircraft activity levels, savings from reductions in air traffic delays attributable to air traffic controlservice, and use of overtime are examples worth pursuing.t' These data, some of which are already being collected by many of the FAA regions and by head- quarters, would provide a basis on which to compare staffing across similar facilities. They would also provide benchmarks of the relative costs and ser- vice implications of different staffing levels, ultimately providing a scientific basis for validating facility staffing estimates. LEARNING FROM REIATED SYSTEMS in searching for a better staffing process, the committee looked outside the FAA air traffic control system for approaches or methods of estimating staffing that FAA could adapt. Like the FAA air traffic control system, many other systems—the military, emergency services, fast food services, main- tenance and delivery, and transportation systems—have variable staffing demands. To qualify as comparable, a system had to have many of the following characteristics: A loosely structured and open- rather than close-ended system; A dynamic, rapidly changing operating environment that creates a fluc- tuating work load with peak periods; Operator tasks that are paced by external events and are responsive to changes in demand; Operator tasks that are standardized, repetitive, and of short duration; and Work that requires experienced staff and a work environment where the consequences of errors are serious. Systems that used a formal process or model to identify and project staffing needs were of special interest. The only systems found to be truly comparable are the air traffic con- trol systems of the military and of other countries. The review of these and other closely related systems, however, did not uncover any "off-the- Identifying appropriate measures, however, will not be easy. For example, high use of overtime, low use of annual leave, and high time on position could indicate a staffing shortage at a particular facility. But it could also indicate a problem in the mix of staffing; the overall level of staffing could be comparable with other similar facilities, but the number of FPLs could be lower, accounting for the use of compensatoly staffing measures.

50 AIR TRAFFIC CONTROL FACILITIES shelf" approaches or methods that FAA could adopt to improve staffing estimates.12 Two approaches were found that merit further consideration. First, the United States armed forces have developed a bottom-up staffing approach for military controllers. Certain aspects ofthe approach could be appropriate, particularly for the FAA regions as they explore development of more uni- form facility staffing methods. At least one of the services, the U.S. Air Force, had used a work-time modeling process similar to FAA's but had abandoned it for reasons of cost and complexity. Now the Air Force has developed a uni- form, position-based approach for individual facilities to estimate their staffing requirements and a formal oversight mechanism for reviewing indi- vidual facility staffing estimates (see text box). The Air Force approach might have most application for small FAA TRACONs and ATCTs where there is not much variability in sector staffing. The second approach involves the use of mathematical models to deter- mine optimal staffing and allocation of manpower given a level of demand, scheduling constraints, and service-level objectives. Such models have been developed for air traffic control facilities, airline operations, and package ex- press services, among others, to provide a more systematic basis for deter- mining staffing requirements and allocating personnel than a bottom-up es- timate can produce (see text box). Most of these models can be adapted to any schedule-driven environment for which work processes and perfor- mance or service standards can be defined. They work best in situations where they can take advantage of the flexibility offered by part-time workers and where work peaks can be smoothed to even out staffing needs. Staffing decision support systems of this type have limited application potential for FAA at this moment. However, they merit consideration in the future as the agency examines more flexible working arrangements13 and develops more external measures for assessing the effectiveness of staffing decisions. SUMMARY The congressional request for more accurate estimates of controller staffing at individual facilities can best be met through changes in the process for de- 12 A comprehensive search of the scientific and technical literature conducted for the committee by the Crew System Ergonomics Information Analysis Center—a Federal Information Analysis Cen- ter—found a surprisingly small number of relevant publications and reports. Either little research and development has been done on staffing standards or, if it has, it is not publicly available. 13 Currently, use of part-time controllers by FAA is veiy limited. According to FAA staff, availabil- iry is an issue; part-time controllers may only want to work certain hours and these times may not coincide with when they are needed. Currency is also a concem. To stay current, controllers must work on all positions. This requirement can limit deploying part-time controllers where they are needed.

STRATEGY FOR AN IMPROVED STAFFING PROCESS 51 UNITED STATES AIR FORCE STAFFING PROCESS FOR AIR TRAFFIC CONTROL: A BOTTOM-UP APPROACH The United States armed services use a standardized, bottom-up, position-based staffing approach and oversight process for estimating staffing requirements for air traffic control. The Air Force staffing process provides a good illustration of how the approach works. First, each tower and radar facility determines how many positions are required for how many hours—for example, four positions for 10 hr, four positions for 8 hr, and three positions for 6 hr—for weekdays and weekends during normal peacetime flying operations. Then the total number of position hours is divided by a manpower availability factor. This servicewide figure includes time for leave and other general ab- sences and a special adjustment for air traffic control for time off posi- tion for such activities as shift briefings, training, and flight physicals. Finally, extra allowances or variances are given for special circum- stances or changes in facility mission. An oversight process helps ensure that individual facilities are not overstaffed and that staffing resources are equitably distributed. The major command associated with each facility collects data on traffic counts and positions by facility type to provide benchmarks against which to evaluate individual facilities' staffing estimates. The major command also visits each facility to evaluate position staffing firsthand. The Air Force has fewer air traffic control facilities than FAA, and they handle a lower volume of traffic. Flight schedules are less pre- dictable—traffic peaks vary greatly from day to day except at training facilities—so staffing requirements are determined for average rather than peak traffic conditions. Controllers work more time on position than at FAA facilities, and there is less variability between facilities in position staffing. These distinctive features may make staffing esti- mates both easier to generate and more accurate. Nevertheless, the approach merits study by FAA, particularly for the smaller air traffic control facilities. termining staffing requirements rather than by focusing solely on the devel- opment of improved model-based staffing standards. The key elements in the strategy for achieving an improved staffing process, which builds on the current approach, follow. First, the headquarters staffing process should retain the use of a quanti- tative model, which provides a consistent method for estimating staffing re- quirements to complement the judgment of experts. Improvements in the

52 AIR TRAFFIC CONTROL FACILITIES AUTOMATED STAFFING PROCESS FOR AIRPORT OPERATIONS Determining staffing requirements and assigning the personnel needed to support airport operations is often a labor-intensive, manual process that calls for detailed analysis of flight schedules, employee availabil- ity, scheduling constraints, and service and cost considerations. Commercial organizations have developed computerized decision support systems to help the airlines determine the level of staffing re- quired to support scheduled flights. One such system generates staffing requirements and shift schedules automatically for ramp and passen- ger services. A staffing requirements model first generates staff work load re- quirements on the basis of flight schedules and other user-defined pa- rameters. The product is an automated manning chart that shows flights and associated staffing requirements within a 24-hr period for an average day. A scheduling model determines the shifts required to cover the work load. The model allows managers to balance staffing with service-level requirements for a specific flight schedule. The models are particularly useful for analyzing the effects on staffing requirements of changes in flight schedules, addition of flights or other changes to work load, changes in staffing policies and level of service assumptions, and use of part-time shifts and other methods of deploying personnel. If FAA were to define levels of service—either at headquarters or in conjunction with the regions—the agency could take advantage of such modeling capabilities to compare the cost and service implications of different staffing alternatives. current models to enhance their usefulness for initial staffing estimation and subsequent oversight purposes are desirable. Thus, in the short term, FAA should examine the potential for refinements to the current staffing stan- dards through low-cost pilot studies. In the longer term, if the cost of refinements to the current staffing standards is large or if changes in the air traffic control system result in major changes in controllers' work that would render the current models obsolete, consideration should be given to developing a more sophisticated model. Second, a uniform approach for estimating facility staffing requirements is needed at the regional level. FAA headquarters, in consultation with the regions and with input from the facilities, should identify and share infor- mation on best practices in the individual regions and in other systems, such as the U.S. armed services. Once consensus is reached on a preferred ap-

STRATEGY FOR AN IMPROVED STAFFING PROCESS 53 proach, FAA headquarters should establish it uniformly across the regions and provide training, if needed, in its use. Third, a headquarters-level oversight process should be established to re- view and reconcile differences between headquarters and regional estimates of facility-level staffing requirements. The oversight process should involve the expertise of those making facility staffing decisions, including FAA re- gional and facility managers as well as headquarters technical staff. FAA should determine how best to organize this function. The oversight process should help FAA reach facility staffing estimates that reflect a broad consensus abOut required levels of staffing at individual facilities. This would be an improvement on current practice. However, without some external measure of the "truth" of these estimates, controversy over their validityis likely to continue. The committee recognizes the diffi- culty of developing such a benchmark. But it urges as the fourth and final element in an improved staffing process that FAA headquarters together with the regions develop systemwide data for tracking the operating and per- formance characteristics of air traffic control facilities. In time these indica- tors could provide a yardstick for determining that the staffing process is producing the "correct" number of facility controllers. REFERENCES ABBREVIATIONS GAO General Accounting Office TRB Transportation Research Board GAO. 1988. FAA Staffing: Improvements Needed in Estimating Air Traffic Controller Re- quirements. GAO-RCED-88- 106. June, 66 pp. Schmeidler, N.F., andJ.J. D'Avanzo. 1994. Development of Staffing Standard.s for Air Traffic Control Functions in Tower Cabs. Technical Report. DTFA01-89-Y- 01037. Operational Technologies Services, Inc., Vienna, Va., and OMNI Engi- neering & Technology, Inc., McLean, Va., Sept. 21. TRB. 1996. Special Report 248: Shopping For Safety: Providing Consumer Automotive Safety Information. National Academy Press, Washington, D.C., 160 pp.

APPENDIX A CoNRfssIoNAL REQuEST fOR STAffING STANDARDS STUDY U S. Congress, House Committee on Appropriations, H. Rept. 104-177, on H.R. 2002, Department of Transportation and Related Agencies Appropria- tions Act, July 11, 1995. Staffing standards study—After many years of internal study, the FAA still does not have a complete understanding of how many controllers are re- quired at each of its facilities. The FAA's 1994 review indicated that almost thirty percent of the agency's field facilities had staffing imbalances of greater than ten percent, compared to the planning standard. Last year, the agency stated their planning standards could not be used for facility planning due to the unique needs of each facility. While acknowledging that some facilities have unique staffing needs, the [House] Committee [on Appropriations] believes the FAA needs a solid planning methodology on which to base its staffing, training, and facility al- location decisions. Without good planning tools, as the agency downsizes there is a higher likelihood of situations similar to the emergency situation which occurred earlier this year at the New York en route center requiring immediate staffing increases. This problem could become even more acute with a smaller and possibly less experienced controller workforce and little or no developmental pipeline for controllers. For these reasons, the [House] Committee [on Appropriations] directs FAA to study the development of a comprehensive methodology whereby the FAA could determine the required number of controllers at each of its facil- ities. This study is to be conducted by the National Academy of Sciences, and should be submitted to the House and Senate Committees on Appropriations no later than April 15, 1997. 54

APPENDIX B CALCULATION Of fACILITY STAffIN6 REQUIREMENTS A key function of staffing standards is to provide an objective method for determining staffing requirements. In this appendix, a description is pro- vided of the steps involved in the Federal Aviation Administration's (FAA's) method of calculating staffing requirements for four major categories of fa- cilities—air route traffic control centers (ARTCCs), terminal radar approach control (TRACON) facilities, air traffic control towers (ATCTs), and auto- mated flight service stations (AFSSs). Table B-i indicates, for each type of facility, the number of facilities and the number and percentage of on-board air traffic control specialists (ATCSs) covered by the staffing standards. OVERVIEW OF FAA STAFFING STANDARDS CHARACTERISTICS AND OPERATION FAA's staffing standards have certain common characteristics. First, they are based on observable and measurable work tasks and activities that create work, known as work load.' Second, because air traffic varies considerably throughout the day and the year, work times and work load typically are measured during busy times to ensure adequate controller coverage in times of peak activity. Third, the staffing standards are based on sample data, which are assumed to be representative of the underlying population, so they are applied to all facilities in a class. Finally, the staffing standards reflect cur- rent facility operating procedures and constraints; they do not address the question of how facilities could be most efficiently staffed. Work is expressed as some measure of the activity of a person. An example of work is the com- munication of aircraft position information by a controller and a pilot. Work load is a demand or requirement for work that is generated by a system or process. An example of work load is the ap- pearance of aircraft on a radar screen that requires a controller to perform the task of maintaining separation of the aircraft. In this case work load increases as a function of the number of aircraft and other variables. 55

56 AIR TRAFFIC CONTROL FACILITIES TABLE B-i AIR TRAFFIC CONTROL FACILITIES AND STAFFING COVERED BY STAFFING STANDARDS BY MAJOR TYPE OF FACILITY (P. K0vALICK, AIR TRAFFIC BRIEFING TO COMMITTEE, MAY 30, 1996) NUMBER OF FACILITIES COVERED BY FAA STAFFING ON-BOARD ATCS STAFP FACILITY TYPE TOTAL STANDARD NUMBER PERCENTAGE ARTCC 24 201, 6,353 38 ATCT 1 379c 4271d 25 TRACON 1 3787d 22 AFSS 91 60 2470 15 Total 588 459 16,881 100 NoTE: Facility data are for federal fiscal year 1995. On-bOard staffing figures are as of March 1996. ATCS = air traffic control specialists; ARTCC = air route traffic control centers; ATCT = air traffic contrdl towers; TRACON = terminal radar approach control facilities; AFSS = automated flight service stations. That portion of the ATCS work force covered by staffing standards. bDoes not include the Anchorage ARTCC and the off-shore facilities at Sanjuan, Honolulu, and Guam. CThere are 28 stand-alone TRACONs and 202 stand-alone ATCTs. The remaining 149 facilities are mixed, that is, the ATCT and the TRACON are located at the same facility. For mixed facilities, the staffing model relevant to each facility type is applied and summed to determine the staffing requirements for the facility as a whole. The staffing standards are not applied to the 94 contracted ATCTs. dfiased on an estimate that ATCTs represent 53 percent, and TRACONs 47 percent, of the ATCS work force for these types of facilities. Staffing standards are defined by FAA as mathematical models incorpo- rating one or more equations that operate on compilations of work times to compute the number of personnel required to perform a set of tasks (S. Helzer, air traffic briefing to committee, May 30, 1996). FAA's staffing stan- dards incorporate three analytical models—a work-time model, a schedul- ing model, and a forecasting model—which are applied to estimate annual facility staffing requirements. Although the specific data and statistical analysis methods differ depend- ing on the type of facility, the general approach for calculating staffing re- quirements is similar (Figure B-i). The first step is to determine the daily ATCS staffing needed to cover observed levels of work load (first five boxes of Figure B-i). To accomplish this, appropriate measurement methodologies and data measures are identified, the data are collected from a sample of sites, models are developed relating work time to work load levels, and the mod- els are applied to determine ATCS staffing required to handle facility traffic for each interval of a day. The next step is to adjust daily ATCS staffing re- FIGURE B-i Flowchart of calculation of facility staffing requirements using FAA staffing standards. Boxes without shading represent the steps involved in estimating staffing requirements at the applicable stage in the process. Lightly shaded boxes represent model inputs. Darkly shaded boxes represent model outputs.

Identification of Measurement Methodology and Data Measures I Data Collection Development of ATCS Work- Time Models Application of Work-Time Models to Facility 24-hour Traffic Requirements for Each Time Frame in 24- hour Day Detailed Facilit Traffic Activity Traffic Count Computer Program Number I Shifts, Shift I Lengths, Shift I Start Times, I I Break I Requirements Application of Development of Scheduling Scheduling 1 Algorithm Model Staffing Application of Forecasting Models 90% ~Da Facility Fore Facility Staffing Adjustment for Off-Position Activities and 7- day Covera e Annual Facility Staffing Requirement

58 AIR TRAFFIC CONTROL FACILITIES quirements to reflect scheduling constraints by applying a scheduling algo- rithm (Boxes 6 and 7 of Figure B-i). The final step is to estimate future year annual staffing requirements by developing and applying a forecasting model and adjusting the results to account for 7-day facility operation and al- lowances for ATCS leave and other administrative functions (Boxes 8 through iO of Figure B-i). The staffing standards are revised periodically. New forecasting models are developed annually on the basis of updated information provided by the facilities and by FAA's forecasting system. Review of the methodologies used for work measurement and the types of data collected occurs less often— approximately every 5 to 6 years. The following sections elaborate on the generic steps described in Figure B-i, drawing on information provided from various contractor reports (TAI 1985; STI 199ia; STI 1991b; STI 1994; Schmeidler and D'Avanzo 1994) ex- plaining development of the staffing standards methodologies and from briefings to the study committee by FAA staff (S. Helzer and P. Kovalick, May 30, 1996). Table B-2 provides more detail on the key methodological ap- proaches and modeling assumptions underlying the staffing standards for each major type of facility. STEPS INVOLVED IN CALCULATING ANNUAL FACILITY STAFFING REQUIREMENTS Determination of Daily ATCS Facility Staffing Requirements Step 1: Identification of Measurement Methodology and Data Measures Determination of an appropriate measurement methodology is the first step in the development of a staffing standard. The project teams began by visit- ing examples of air traffic control facilities targeted for staffing standards de- velopment to familiarize themselves with the work environment and ATCS responsibilities, positions, and functions. Time study and work sampling—both well-known industrial engineering methodologies—were selected as the most appropriate methodologies for mea- suring ATCS work. Time study refers to a work measurement technique of establishing a required time to perform a given task on the basis of measure- ment of the actual work done to complete the task. Time study is appropriate in measuring work that involves highly repetitive tasks of short duration, such as controlling aircraft. Work sampling involves periodic observations of work to produce estimates of the proportion of time workers are observed perform- ing tasks corresponding to the different work tasks that make up a job. Work sampling is one of the simplest work measurement techniques, yet it can be adapted to analyze a sophisticated work system such as that of the ATCS.

TABLE B-2 SUMMARY OF METHODOLOGIES AND MODELING ASSUMPTIONS USED IN THE DEVELOPMENT OF FAA STAFFING STANDARDS FOR MAJOR TYPES OF FACILITIES FACILITY TYPE NUMBER OF SAMPLE FACILITIES TASK ANALYSIS METHODOLOGY AND SAMPLING POSITIONS INTERVAL COVERED WORK-TIME MODEL ANALYSIS METHOD MODEL ADJUSTMENTS To DETERMINE STAFFING REQUIREMENTS FORECASTING MODEL ANALYSIS METHOD Air route 1985 study: six Timestudy; Radar, handoff, Linear regression Constant for Linear regression traffic ARTCCs 15-n-Lin coordinator, Dependent variable: "look at Dependent variable: control 1991 revalidation intervals and flight data total work in minutes display" time 24-hr controller centers study: three Independent "Difficulty" staffing requirement (ARTCCs) ARTCCs not variables:" number of factor, 90 Independent variable: visited in sector aircraft percent number of facility 1985 study minutes during a 15- confidence operations in 24-hr min interval; number level except period of aircraft leaving a for 1- Note: forecasting sector during a 15- controller model is developed for min interval; number model, which each ARTCC • of aircraft remaining used 95 in a sector during percent the entire 15-min confidence interval; maximum level instantaneous number 1.76 adjustment • of aircraft in a factor for 7- sector during a 15- day coverage min interval and off- position activities (continued on next page)

TABLE B-2 (CONTINUED) MODEL TASK ANALYSIS ADJUSTMENTS To METHODOLOGY DETERMINE NUMBER OF AND SAMPLING POSITIONS WORK-TIME MODEL STAFFING FORECASTING MODEL FACILITY TYPE SAMPLE FACILITIES INTERVAL COVERED ANALYSIS METHOD REQUIREMENTS ANALYSIS METHOD 2.5 controllers for each manual/oceanic sector for a 24-hr day Terminal 1991 study: 35 Time study; Radar, handoff Linear regression Constant for Linear regression radar TRACONs, 15-min coordinator, Dependent variable: "look at Dependent variable: approach partially random intervals and flight data total work in minutes scope" time annual controller control sample Independent variables: "Difficulty" staffing requirement facilities stratified by total number of factor, 90 Independent variables: (TRACONs) number of commercial and percent number of active sectors military aircraft in confidence sectors; peak 1995 revalidation a sector during a level, for all variability study: 7 15-min interval; but the flight (standard deviation TRACONs, all total number of data position of hourly traffic ARTS III general aviation 1.76 adjustment counts); traffic TRACON sites aircraft in a sector factor for 7- count for the 37th- during a 15-min day coverage busiest day in interval; and off- thousands total number of position Note: forecasting aircraft handled by activities models are developed

a facility during a 90th percentile for two types of 15-min interval busiest day TRACONs—ARTS II (flight data model traffic and ARTS III only) forecast facilities Air traffic 64 ATCTs for work Time study Local control, Simulation model based Constants for Linear regression control sampling and work ground control, on time study to other work not Dependent variable: towers 26 ATCTs for time sampling; and clearance estimate work time in accounted for annual controller (ATCTs) study 15-min delivery minutes for the by the staffing requirement 392 ATCTs for intervals primary work activity simulation Independent variables: hourly airport of ATCT controllers— model hours of operation traffic separation of aircraft 1.76 for 7-day (except for Terminal operation counts through radio coverage and Levels IV and V); comunication with off-peak IFR operations for pilots position 37th-busiest day; Average or lowest work activities VFR operations times in minutes for Additional (i.e., airport most other work staffing: operations - IFR activities based on three operations) for 37th- work sampling data controllers busiest day plus adjustments for for nonradar Note: forecasting multiple controller or approach models are developed combined controller control for four levels of positions facilities; two terminals on the basis controllers for of density of air limited radar traffic approach facilities where IFR - VFR ~t 100 (continued on next page)

TABLE 6-2 (CONTINUED) TASK ANALYSIS METHODOLOGY NUMBER OF AND SAMPLING FACILITY TYPE SAMPLE FACILITIES INTERVAL POSITIONS COVERED WORK-TIME MODEL ANALYSIS METHOD MODEL - ADJUSTMENTS To DETERMINE STAFFING REQUIREMENTS FORECASTING MODEL ANALYSIS METHOD 90th percentile busiest day traffic forecast Automated 1994 study: nine Time study; In-flight, EFAS, Work-time equations are Constants for Linear regression flight AFSSs, non- 1-hr preflight, developed to calculate such work Dependent variable: service random selection intervals data, total work in minutes elements as number of controllers stations process broadcast, and by position for a 1-hr "review required for the 90th (AFSSs) weather time period display," percentile busiest day observation For tasks with known "pre-position Independent variables: work load drivers, briefs," and number of pilot time to perform each "related briefings provided task is calculated activity" during a 24-hr time using either standard "Difficulty" period (90th time analysis factor, 90 percentile day); (dividing total time percent number of flight expended on a task by confidence plans received the total number of level, except during a 24-hr time. task observations) or for data, period (90th regression analysis broadcasting, percentile day)

where the dependent and weather variable is typically observation total work in minutes positions and the independent Leveling variable is some analysis to measure of activity equalize For tasks without performance of readily identifiable noncritical drivers, a constant tasks among time is used, defined time study as the average intervals percentage of time 1.76 adjustment expended on these factor for 7-day tasks converted into coverage and off- minutes from the 1-hr position activities time study samples . 90th percentile busiest day traffic forecast NOTE: ARTS = automated radar terminal system; IFR = Instrument Flight Rules; \'FR = Visual Flight Rules; EFAS = Enroute Flight Advisory Service. SNot all variables appear in all staffing model equations.

64 AIR TRAFFIC CONTROL FACILITIES Application of these methodologies requires defining small, discrete divi- sions of work, called work elements. The appropriate unit of analysis was deemed to be a sector in an air traffic control facility. Thus, for example, the work activities of ATCSs at TRACONs were classified into 21 work elements for possible data collection for the radar, handoff, and coordinator positions. Such activities included receiving a communication from or transmitting one to an aircraft, communicating by voice within a facility, and processing flight strips (511 1991a, 10).2 The appropriate duration of a work sample must also be determined. The observation period should include a good portion of the tasks performed. For ARTCCs, TRACONs, and ATCTs, a 15-min observation period was se- lected, because this generally represents the maximum time an aircraft is in a given sector for which an ATCS has responsibility. For AFSSs, a 1-hr obser- vation period was determined to be more suitable, because it included a good portion of the tasks performed by ATCSs in these facilities. Next, appropriate work load measures must be defined, and collected for the same observation interval as the work elements. The primary factor that generates controller work is aircraft activity. To illustrate, the following measures of aircraft activity were identified for possible data collection at TRACONs: the number of aircraft in the sector at the beginning of the 15-min observation period, the number at the end of the period, and the number acquired by the sector during the period (STI 199 la, 11). In addition, "complexity factors," which make one facility different or more complex than another, such as airspace restrictions and runway configuration, were identified for data collection (STI 1991a, 12). Complexity factors provide one explanation for why controllers at different facilities handle varying amounts of traffic in a specified period of time. Finally, pilot tests were conducted to validate the selected measurement methodologies and data collection strategies. For example, for the TRA- CONs, validation included a comparison of the work load information col- lected manually by the study team with automated data on air traffic levels available from two of the pilot facilities (STI 1991a, 17). A comparison of work data gathered by different observers was also made to ensure consis- tency and uniformity in data collection (STI 1991a, 17). Step 2: Data Collection Sampling Strategies and Site Selection Once the measurement method- ology has been selected, full-scale data collection can begin. Sampling strate- 2 To facilitate collection of ATCS work tasks, the project team used an automated hand-held event recorder—an OS-3 Event Recorder—specifically designed to collect time study data. Each key on the device is programmed to collect data for a specific work element. A discussion of the adequacy of these reliability tests is contained in Appendix C.

CALCULATION OF FACILITY STAFFING REQUIREMENTS 65 gies must be determined for site selection and work measurement. As noted in Chapter 2, FAA contractors designed their sampling strategies for devel- opment of national-level staffing standards. Development of facility-level staffing standards would have required larger sample sizes, increasing the cost of data collection. Facilities were selected for data collection from stratified samples for TRACONs and ATCTs. Some facilities in the sample groups were chosen randomly; others were identified by the FAA regions for inclusion in the samples. For ARTCCs and AFSSs, sites were selected to provide "a repre- sentative sample." The appropriate number of the work samples to be collected at each se- lected site was also determined. The reports vary in their specificity con- cerning both the determination and the adequacy of the work sample size for statistical testing. Collection and Summary of Work Data Work samples were collected primarily during periods of heavy traffic to ensure that ATCS staffing is ade- quate to cover traffic peaks. The data were collected for each sector and for single and multiple staffing of these sectors. In addition, the data collection team took manual counts of aircraft activity corresponding to the time-study samples. Finally, the team collected other data about the facilities, including complexity factors and shift schedules. Once collected, the work sample data were summarized for each time pe- riod, errors in data input were corrected, and data outliers were dropped.5 Data collected on work load counts and, in some cases, complexity factors were then added for each time period. In a final step, the data were divided for analysis into data bases by ATCS position. These data provide the inputs to the work-time models described next. Step 3: Development of ATCS Work-Time Models The choice of model form depends on an analysis of the relationship between controller work time and measures of work load.6 Work-Time Models for ARTCCs, TRACONs, and AFSSs For each of these major types of facilities, a linear relationship was observed between ATCS Three ARTCCs that had not been included in the previous ARTCC staffing standard study were selected (STI 1991b, 10). Initially, one AFSS was selected from each of the eight regions in the con- tinental United States (STI 1994, 17). 5 For example, in developing the staffing standard for TRACONs, observations with very high work values corresponding to very low aircraft activity periods were eliminated. The concern was that these observations were not indicative of controller work, particularly when the purpose was to measure work times during busy periods, and that they could skew the data if they were not re- moved (STI 1991a, 31). 6 Typically, correlation matrices and scatter plots were developed to determine the relationship be- tween controller work time and work load for a particular work element.

66 AIR TRAFFIC CONTROL FACILITIES work time and work load for most of the key work elements. Thus, linear regression was deemed to be an appropriate model form. Models were developed for different ATCS positions (e.g., radar con- trollers, flight data controllers) from simple linear relationships between work time and work load, pooling the data on each from the sample sites. For some facilities like TRACONs, subsets of the data, grouped by such fac- tors as sector type and facility type, were analyzed to determine whether sep- arate models for each subset were justified. Significant differences were not found across the subsets, so the data were combined and treated as a single statistical population. The exception is the ARTCCs, for which separate mod- els were developed by sector type (i.e., high- and low-altitude sectors). Complexity factors were also analyzed for potential inclusion as inde- pendent variables in the work-time regression models. However, the com- plexity factors did not significantly improve the models' ability to explain the observed variation in ATCS work time, either singly or as a group. Hence, these factors were not included in the final models. Because they are based on aggregated data and simple relationships be- tween ATCS work time and work load, the models do not explain a high frac- tion of the variation in ATCS work time observed during data collection.7 Thus, an arbitiary allowance for complexity or difficulty, a "difficulty factor," is added to most of the models8 to compensate for the poor predictive capa- bility of the models. The addition of the difficulty factor ensures that the models predict controller work times adequate to handle peak work load 90 percent of the time.9 Tables B-3 through B-5 show the actual work-time regression models for each of the ATCS positions at TRACONs. Although not shown, similar types of equations were developed for the ARTCCs and for many of the positions at the AFSSs. Work-Time Models for ATCTs Analysis of the relationship between radio communication activity—one of the primary work tasks of ATCT con- trollers—and work load did not indicate a linear pattern. Instead, the data 7 For example, the single-controller model for TRACONs explained 46 percent of the variation in the examined data predicting total controller work time (ErgoTech, Inc. 1995, 25, 60). Similar in- formation was not available for the multiple-controller model or for the models developed for the ARTCCs or the AFSSs in the written documentation provided by the contractors. 8 A difficulty factor is not added to the work time models for the flight data position, because it is based on predicting average rather than peak staffing requirements. Thus, the flight data peak work load can be distributed through lighter work load periods. The difficulty factor is determined (a) by calculating the standard deviation of all unexplained and unmodeled differences between the model and the total observed work time and (b) by multiply- ing the resulting standard deviation by a confidence factor. For example, the standard deviation of the differences between the model and observed work time is 1.92 min for the single-controller model for TRACONs. A 90th percentile confidence factor is 1.282. Multiplying 1.92 by 1.282 yields a difficulty factor of 2.4614 mm, which is added to the model (STI 1991a, 41). A 95th percentile confidence factor is used for the ARTCC one-controller models (STI 1991b, 26).

TABLE B-3 SINGLE-CONTROLLER WORK-TIME MODEL FOR TRACONs (STI 1991a, B-7) ELEMENT INTERCEPT COMILTOT GATOT Look at scope/keyboard and trackball 0.5612 0.0144 0.0232 Combined for process flight strip 0.4925 - 0.0316 Combined for communicate with aircraft - 0.5096 0.5480 Combined for related activity 0.0976 - - Combined for interphone communication 0.4460 - 0.0313 Combined for within facility communication 0.1291 0.0066 0.0160 Total 1.7264 0.5296 0.6501 Look at scope 1.0000 Variability . 2.4614 Total 5.1878 NOTE: TW1 = 5.1878 + (0.5296WCOMILTOT) + (0.6501*GATOT) where TW1 = total work in minutes for a single radar controller on a sector for a 15-min interval, COMILTOT = total number of commercial and military aircraft in a sector during a 15-min interval, and GATOT = total number of general aviatidn aircraft in a sector during a 15-nun interval. TABLE B-4 MULTIPLE-CONTROLLER WORK-TIME MODEL FOR TRACONs (STI 1991a, B-B) ELEMENT . INTERCEPT COMILTOT GATOT Look at scope/keyboard and trackball 0.2498 - - Combined for process flight strip 0.4464 . -0.0 136 - Combined for communicate with aircraft - 0.4696 0.5035 Combined for related activity 0.0074 - - Combined for interphone communication 0.0310 - - Combined for within facility communication 0.3021 - - Total 1.0367 0.4560 0.5035 Look at scope 1.0000 Variability 2.2307 Total 4.2674 NOTE: TW2 = 4.2674 + (0.4560*COMILTOT) + (0.5035*GATOT) where TW2 = total work in minutes for a radar controller in a two-controller sector situation for a 15-min interval, COMILTOT = total number of commercial and military aircraft in a sector during a 15-min interval, and GATOT = total number of general aviation aircraft ins sector during a 15-min interval.

68 AIR TRAFFIC CONTROL FACILITIES TAB[E B-5 FLIGHT DATA CONTROLLER WORK-TIME MODEL FOR TRACONs (STI 1991a, 13-10) ELEMENT INTERCEPT FACAC Combined for keyboard activity 0.6313 - Combined for walk 0.6105 0.0257 Combined for distribute strips 0.6494 0.0542 Combined for process strips 5.4306 0.0827 Voice communication 0.8873 - Related activity 0.9099 - Total 9.1190 0.1626 NOTE: TWFD = 9.1190 + (0.1626*FACAC) where TWFD = total work in minutes for the flight data position in a facility for a 15-min interval and FACAC = total number of aircraft handled by a facility during a 15-min interval. This facility count rep- resents the sum of the arrivals to and departures from the facility. Because the flight data position's work load occurs 15 to 30 mm before an aircraft's arrival at a facility, the flight data model for a current interval is applied using facility counts from two intervals later. indicated that radio communication is a random activity, the essence of which is difficult to capture in a closed mathematical form (Schmeidler and D'Avanzo 1994, 34)10 Thus, simulation rather than linear regression was se- lected as the appropriate model form. Simulation is appropriate when activ- ities are so-complex that they cannot easily be described using traditional mathematical formulations. Some, but not all, complexity factors were included in the ATCT simu- lation model. In addition, algorithms were developed to specify time al- lowances for controller activities other than radio communication." A pro- iS Radio communication between the ATCT controller and aircraft pilots is a sequential activity. The controller Cannot engage in multiple communications simultaneously, thus causing queuing. Radio communication is a random activity because the time required for individual communications varies unpredictably within as well as across ATCT positions (Schmeidler and D'Avanzo 1994, 34). Ii Work times for these activities were determined by analyzing the work sampling data. More specifically, the number of minutes spent on each work element was computed by dividing the number of observations for the selected work element by the total number of observations for the 15-min observation period, and then multiplying the product by 15 mm (Schmeidler and D'Avanzo 1994,51). The work sampling data were then grouped by amount of aircraft activity by 15-mm in- terval and by ATCT control functions. For work elements for which there was no obvious correla- tion with work load (e.g., miscellaneous work time for the local control position), work time was estimated as an average, excluding the highest and lowest counts (Schmeidler and D'Avanzo 1994, 54). For work elements more highly correlated with work load (i.e., looking and scanning work time for the local control position and miscellaneous work times for the ground control and clear- ance delivery positions), the work time was estimated for peak traffic conditions (Schmeidler and D'Avanzo 1994,55). Other allowances were added for coordination if the work load reaches a level that is more than one controller can handle in the 15-min interval (Schmeidler and D'Avanzo 1994, 55-56).

CALCULATION OF FACILITY STAFFING REQUIREMENTS 69 cedure similar to that for the other staffing standards was used to pool the work and work load data from the sample sites to develop the time al- lowances for work not accounted for by the simulation model. Step 4: Application of Work-Time Models Because air traffic control work load fluctuates, a major purpose of the work- time models is to ensure adequate staffing for work load peaks. Before daily ATCS staffing requirements can be determined, detailed data on facility traf- fic by sector must be obtained for each time interval of a 24-hr day. Detailed automated data on daily traffic activity were available for the ARTCCs, for many TRACONs,12 and for some positions at AFSSs. Most ATCTs manually track work load data in 1-hr intervals.'3 Once the work load data are available, the work-time models are applied to the work load measures at the sector level for each time interval of the day, and the results are adjusted to provide integer staffing requirements.'4 Using the TRACONs as an illustration, if there was any aircraft activity during a 15- min interval, the single-controller model is applied to determine whether one radar controller is sufficient to staff the position. If the total work time cal- culated using this model is greater than 15 mm, the multiple-controller model is applied. If the work time using this model is greater than 15 mm, three controllers are needed, the maximum number that can support one radar position (STI 1991a, 50-5 1). The process is repeated for each time interval of a day. Step 5: Daily ATCS Staffing Requirements This is the output of the preceding four items. Application of the work-time models provides ATCS staffing requirements for each time interval of a 24-hr day and a daily total. 2 Detailed information on aircraft activity for each sector in a TRACON is available from all auto- mated radar terminal system (ARTS Ill) facilities with the exception of the Chicago O'Hare TRACON (STI 1991a, 16-17). 13 An algorithm was developed to approximate work load distribution during the hour so that the work load data could be matched with the work time data that were collected for 15-min intervals (Schmeidler and D'Avanzo 1994, 32). 11 The exception is the flight data position. The flight data model is applied to facilitywide rather than sector-level work load data. Furthermore, the flight data position's work load occurs 15 to 30 min before an aircraft's arrival in the facility's airspace, which must be taken into account in match- ing work time with work load when the model is applied.

70 AIR TRAFFIC CONTROL FACILITIES Adjustments for Scheduling Step 1: Development of Scheduling Model Air traffic control facilities require several shifts to cover peaks in work load and around-the-clock operation. Thus, shift scheduling is important to en- sure adequate daily staffing to meet peaks in activity and to allow for meals and breaks. A scheduling model was developed, which converts ATCS interval staffing requirements to daily requirements subject to shift scheduling constraints. There are two inputs to the model: (a) scheduling parameters, which include such items as the number of contiguous time periods that make up a shift, the number of periods allowed for meals, and limits on shift start times; and (b) the input from the preceding step, which provides information about ATCS staffing requirements for each interval during facility operation. Step 2: Application of Scheduling Algorithm The scheduling parameters for the scheduling model come from two sources. Generally, the shift start times are selected by FAA on the basis of analysis of the data collected at the sample sites; they represent the most commonly staffed shifts among the facilities studied. The other data on shift length and allowances for meals, breaks, and position-related briefings reflect FAA head- quarters policies and FAA bargaining unit contracts. None of the scheduling parameters are facility specific. Once the scheduling parameters have been specified and the staffing re- quirements considered, an algorithm is applied to determine an appropriate staffing schedule. Step 3: Daily Facility Staffing Model outputs include a shift schedule and the total number of ATCS staff required during a facility operating day. Estimating Annual Facility Staffing Requirements Step 1: Development of Forecasting Models A major use of the staffing standards is to estimate future staffing require- ments. Forecasting models were developed to assist in projecting facility

CALCULATION OF FACILITY STAFFING REQUIREMENTS 71 staffing requirements. Using data on daily schedule-adjusted staffing re- quirements as the dependent variable and on daily facility traffic counts and other measures of traffic activity as the independent variables, a regression line is fitted. Separate forecasting models were developed for two types of TRACONs, four types of ATCTs, and for each ARTCC and AFSS.15 Step 2: Application of Forecasting Models Future year staffing is estimated in several steps. First, traffic activity counts for the 90th percentile busiest day (the 37th-busiest day) are determined for the preceding year. Then traffic is forecast for the current or any future year by multiplying the activity counts for the preceding year 90th percentile busiest day by the appropriate forecast factor from FAA's forecasting sys- tem.16 Finally, traffic forecasts and other relevant activity data are inserted into the forecasting model to forecast staffing levels for the 90th percentile target day. Step 3: 90th Percentile Busiest Day Facility Staffing Requirement This is the output of the preceding step. Application of the forecasting model provides a projected ATCS facility staffing requirement for the 90th per- centile busiest day of the forecast year. Step 4: Annual Facility Staffing Requirement The final step is to adjust the estimated daily facility staffing requirement to an annual total, taking into account 7-day facility operation and the time that ATCSs are not available for controlling traffic. The first adjustment for 7-day operation is handled by multiplying the daily staffing requirements, which are based on a 5-day workweek, by 7/5 or 1.4 to provide adequate staffing for a 7-day workweek. No adjustment is made for weekend versus week- day staffing requirements, although weekend traffic typically is lower than weekday traffic. The second adjustment, for time off position, includes such activities as leave, training, annual physicals, and union meetings. Annual hours for off- 15 The two types of TRACONs include ARTS Ill facilities, from which automated measures of traf- fic activity can be retrieved, and ARTS!! facilities, from which they cannot. The four levels of ATCTs are differentiated by the density of air traffic controlled. 6 FAA provides facility-level forecasts of aircraft handled at terminals, ARTCCs, and AFSSs as part of its forecasting system (FAA 1996, VIlI-9).

72 AIR TRAFFIC CONTROL FACILITIES position activities represent facilitywide averages. The current adjustment factor, calculated by dividing the total hours per year (2,087) by the total hours available for ATCS activity, is 1.259. The total adjustment factor-1.76—is the product of the two preceding adjustment factors. The facility staffing requirements for the 90th percentile forecast day are multiplied by 1.76 to determine the final annual facility staffing number. For illustrative purposes, the following equations give the forecasting models for estimating future year staffing requirements at TRACONs (data are from FAA and are as of December 15, 1995). For ARTS III facilities; C = 1(1.0431*S) + (0.0696*V) + (6.3447*F)1*1.76 For non—ARTS III facilities, C= 1(1.0065*5) + (0.0204*V) + (7.1508*F)1*1.76 where C = number of controllers (annual requirement), S = number of active sectors, V = peak variability (staiidard deviation of hourly traffic counts), and F = traffic count for 37th-busiest day, in thousands. REFERENCES ABBREVIATIONS FAA Federal Aviation Administration STI Standard Technology, Inc. TAI Technology Applications, Inc. - FAA. 1996. FAA Aviation Forecasts, Fiscal Years 1996-2007. FAA-APO-96-1. U.S. De- partment of Transportation, March. ErgoTech, Inc. 1995. Review of the FAA Staffing Standards for Terminal Radar Ap- proach Control (TRACON) Facilities. West Lafayette, Ind., Feb. 1, 87 pp. Schmeidler, N.F., andJ.J. D'Avanzo. 1994. Development of Staffing Standards for Air Traffic Control Functions in Tower Cabs. Technical Report. DTFA01-89- Y-01037. Operational Technologies Services, Inc., Vienna, Va., and OMNI En- gineering & Technology, Inc., McLean, Va., Sept. 21. STI. 1991a. FAA Staffing Standards for Terminal Radar Approach Control Facilities. Technical Report. DTFAO1-88-01016, Work Order 1. Rockville, Md., Nov. 15. STI. 1991b. FAA Staffing Standards for Air Route Traffic Control Centers. Validation and Revision Report. DTFA01-88-Y-01016, Work Order 5. Rockville, Md., Nov. 15. STI. 1994. FAA Staffing Standards for Automated Flight Service Stations. Technical Re- port. DTFA01-87-Y-01016. Rockville, Md., Nov. 15. TAI. 1985. Development of Air Route Traffic Control Center Staffing Standards. DTFA01-83-Y-30548, Task Order 14, Deliverable 8. Falls Church, Va., Sept. 6.

APPENDIX C RvIw of SHORTCOMINGS IN CURRENT STAffING STANDARDS fOR APPLICATION TO INDIVIDUAL fACILITIES A more detailed review of Federal Aviation Administration (FAA) head- quarters staffing standards from the perspective of their application to indi- vidual facilities is provided in this appendix. To conduct this review, the committee examined the numerous reports provided by FAA that docu- mented development or revision of staffing standards for each of the major categories of air traffic control facilities.' The contractors who developed these reports were also interviewed.2 Finally, the committee was briefed on recent reviews of the staffing standards for ARTCCs and TRACONs prepared for FAA by industrial engineering experts (ErgoTech, Inc. 1994; ErgoTech, Inc. 1995; Cullinane 1996). One of the difficulties in reviewing the staffing standards was the lack of adequate documentation in many of the reports on data collection ap proaches, model-building efforts, and statistical testing of model results. Without more detail, itwas often difficult.to determine whether appropriate data collection and analytic procedures were followed. In fairness, the reports were prepared for FAA's in-house use. In the future, however, more atten- tion should be given to documenting their development, particularly if These reports include the staffing standards for air route traffic control centers (ARTCCs) (TAI 1985; STI 1991a), for terminal radar approach control (TRACON) facilities (STI 1991b; FAA 1995), for air traffic control towers (ATCTs) (Schmeidler and D'Avanzo 1994); and for automated flight service stations (AFSSs) (STI 1994). 2 Staff met with Neal Schmeidler, president of Omni Engineering & Technology, Inc., who was prin- cipal investigator of the terminal staffing standards report (Schmeidler and D'Avanzo 1994) and was involved in the original staffing standards study for ARTCCs (TAI 1985); John D'Avanzo, also of Omni Engineering & Technology, Inc., who was also involved in the terminal staffing standards report; and Michael Watson, formerly of STI Technology, Inc., part of the team that developed the staffing standards for TRACONs (STI 1991b). Allen Benn, who conducted much of the statistical analysis for many of the staffing standards, was contacted in a telephone interview. 3 In 1993 FAA established a committee, chaired by the Deputy Administrator, to review the pur- pose and role of staffing standards in the agency. The Staffing Standards Review Committee en- gaged outside experts to evaluate selected staffing standards and alternative work measurement methodologies. 73

74 AIR TRAFFIC CONTROL FACILITIES staffing models are to play a role in a strengthened headquarters-regional staffing process. Better documentation should increase understanding and use of the models and should assist verification and reviews of methodolog- ical approaches. Another critical problem was the lack of any external measure of "truth" against which to test the validity of model-predicted estimates of facility staffing requirements. For a model to have validity in the scientific sense, pre- dictions of the model must be compared, directly or indirectly, with an ex- ternal measure of the "correct" level of staffing.4 FAA, however, has no for- mal performance criteria related to staffing or any systematic way to measure air traffic control system performance that would allow direct or indirect5 ob- servation of the correct level of staffing. On-board facility staffing is fre- quently compared with facility estimates predicted by the staffing standards, but this is not a good test of the validity of the estimates. Actual staffing does not necessarily reflect the correct level of staffing. in the absence of formal validation criteria, the committee examined how the staffing standards were developed and are currently used as a way of ad- dressing the issue of using the models to provide facility-level staffing esti- mates. Three topics are covered in the sections that follow: data collection, modeling, and application of the staffing standards. DATA COLLECTION The most critical ingredients of a staffing standard are quantitative, empiri- cal measurements of work and work load. Statistically unreliable or invalid measurements of these parameters can defeat even the most sophisticated mathematical models. The staffing standards reports often refer to validation of the staffing standards. However, as noted in the review of staffing standards for TRACONs (Ergolech, Inc. 1995, 41), the effort to compare model-based staffing estimates with actual staffing at specific facilities is not a validation of the staffing standard, but rather "a study of the reliability of these standards ... the model-predicted data was compared to data collected at those TRACONs that were used for the purpose of model development." "One would need to visit other TRACON facilities which were not used for the model development purposes, make observations about actual activity levels, and correlate these with the staffing model prediction data" (ErgoTech, Inc. 1995, 70). Even if this advice were taken, however, the comparison would not provide a true measure of the correct level of staffing. Model results could be correlated with well-defined quantitative performance measures from which one could infer the consequences of inadequate or excess staffing. For example, a high, sta- tistically significant correlation between model estimates that predict a shortage of staff and related performance measures (e.g., increased delay times for aircraft) and staffing compensations (e.g., more time on position, greater use of overtime, less use of annual leave) would be indirect evidence of the validity of the modeled estimates. As discussed in Chapter 3, however, establishing the links between staffing levels and performance measures and compensatory adjustments is not an easy task.

SHORTCOMINGS IN CURRENT STANDARDS 75 FAA selected time study and work sampling as its primary methods of col- lecting work data.6 These are well-known industrial engineering method- ologies for studying worker performance. The agency's choice was a good one. Work sampling is relatively simple to do, and time study measures are empirical and relatively inexpensive to collect. Nevertheless, as will be dis- cussed, lack of a more solid theoretical basis to guide decisions about data collection and apparent weaknesses in the application of these methods raise questions about the accuracy of the data. General Methodological Approach Detailed controller performance data were collected without an adequate theoretical basis that would have provided a context and a rationale for their collection. When studying human performance in complex systems, it is customary to develop a detailed task analysis of the functions, tasks, and work elements required of system personnel (Van Cott and Paramore 1988; Cullinane 1996, 44).7 Such analysis serves several purposes. First, it links human perfor- mance to system performance. The linkage provides a basis for interpreting how specific work activities might be affected by the operational and de- sign demands of the system and by fatigue, stress, and other performance- shaping factors; thus it provides the rationale for selection of specific work elements for study. Second, it helps establish preliminary estimates of the time required to perform tasks and related work elements. Finally, the con- trolled vocabulary used to develop a task analysis provides a uniform set of work element descriptors, which introduces consistency and uniformity into the actual data collection process. FAA's approach to data collection was based largely on observation of controllers at work in different operating environments in different types of facilities. The work teams then divided the controller's primary function- 6 Early in the development of staffing standards FAA decided to use measures of controller behav- ior that could be identified and recorded by trained observers. These observations are the basic data on which staffing standards are developed. In 1994 FAA asked a contractor to examine the feasi- bility of using cognitive engineering methods (i.e., measurements of mental activities) to comple- ment the prediction of staffing requirements based on observable work measurement for the con- trollers at TRACONs. The study concluded that "the cognitive engineering techniques in their present form are not well suited for setting standards for TRACON controllers" and that "these con- troller tasks contain greater complexity than cognitive engineering methods can presently model" (ErgoTech, Inc. 1994, 3). A work task consists of a series of work elements, such as looking at a radarscope, processing a flight strip, using a keyboard, or communicating with an aircraft by voice. Tasks have identifiable goals, such as to provide clearance, that fulfill some system function that the controller is designated to carry out.

76 AIR TRAFFIC CONTROL FACILITIES separation of aircraft—into small, discrete divisions of work, called work el- ements (STI 1991b, 10). A work element is among the briefest activities that can be detected, discriminated from other work elements, and measured in duration by trained observers. Without a task analysis to link these work elements to broader work tasks and system functions, however, factors that might have affected work time, such as fatigue, stress under sustained activity levels, and errors and error recovery, could have been overlooked. Sampling Issues Sampling design and adequacy of data collected for analysis are impor- tant issues that could affect the adequacy of FAA's staffing standards esti- mates for individual facilities. According to FAA's contractors, site selection and sampling plans were geared to the development of national staffing standards. Site Selection Site selection methods did not always appear to follow good statistical prac- tice. For example, agency plans to conduct random and stratified sampling of facility sites were sometimes adjusted when there were requests from the regions to include sites that had not been randomly selected. The effects of these changes on the validity of the data collected are not well documented. In at least one case—the TRACON staffing standard—no significant differ- ences between randomly and nonrandomly selected study sites were found, although this finding had not been reported in the written documentation (ErgoTech, Inc. 1995, 30). Position Sampling The project teams determined the positions and staffing combinations to be observed from visits to the actual facilities and pilot tests. It is not always clear whether adequate samples were collected for each situation or, when the data on sample sizes are provided, whether the samples are adequate for statistical testing. For example, in the most recent revision of the ARTCC staffing standards, the project team decided to collect data on sectors staffed with one, two, three, and four controllers to accurately reflect ARTCC oper- ations (STI 1991a, 7). However, it appears that no sample data were collected for the high-altitude one-controller position, a limited number were collected for the high-altitude three-controller position, and only one sample was

SHORTCOMINGS IN CURRENT STANDARDS 77 collected for the four-controller position (Cullinane 1996, 18).8 The effect of undersampling of these positions on the validity of the data cannot be determined. Respondent Selection for Observation The staffing standards reports are silent on how air traffic control specialists (ATCSs) were selected for observation. Either all or some ATCSs on duty may have been observed. Any form of selection other than random sampling could introduce bias. For example, highly experienced ATCSs, who might be expected to work more efficiently than those with less experience, could have shorter and less variable work element times. The documentation does not indicate how respondent selection was handled. Work Sampling Different work sampling strategies were used for certain of the staffing stan- dards, which can produce data that vary in quality and accuracy (Meister 1985, 310-312). For example, random time interval sampling in 15-mm busy time periods (using a Silent Reminder for period selection) was used in the data collection for the ATCTs.9 Continuous sampling (using the OS-3 Recorder'°) in 15-min busy periods was used in data collection for the TRACONs. The effect of using different sampling methods on the consistency of the data is not known. The work element descriptors used in the initial and in the revision stud- ies for the same type of facility differed for at least one staffing standard— the ARTCCs. New work elements were added, and some were combined. 8 The practical difficulties of obtaining these data should be noted. The ability to observe different position configurations as well as many of the factors that shape controller work and work times are subject to what happens while the data collection team is on site. If a particular combination of factors is of interest, the data collectors.may need to stay on site until it can be observed. The oc- currence is often difficult to predict, and waiting for an occurrence clearly has cost implications. The Silent Reminder emits a low-intensity vibration (to avoid distracting the work force) at ran- dom times for a designated duration. When the signal occurs, the analyst records what he or she observed on a work sample form containing a listing of the work elements identified during the pilot test phase of the study (Schmeidler and D'Avanzo 1994, 26). A different approach was used to measure voice communication activities. Time study measurements were made of all recorded voice communications using voice tape playback equipment obtained from 25 facilities where work sampling was conducted (Schmeidler and D'Avanzo 1994, 30). 10 The 05-3 Event Recorder is a hand-held device—a small computer specifically designed to collect time study data. Each key on the device is programmed to collect data for a particular work element. The observer simply presses the key corresponding to the work element that occurs. The 05-3 keeps track of each work element and its associated time in fractions of seconds (STI 1991b, 10).

78 AIR TRAFFIC CONTROL FACILITIES The net effects of these changes on the consistency of the data cannot be assessed. Work Load Sampling Aircraft activity is the key factor that determines ATCS work load and the time required to perform work. Detailed data on air traffic counts are needed to operate the staffing standards. The availability and accuracy of these data are an issue. For example, the ARTCCs, some of the TRACONs, and the AFSSs have continuous, detailed, automated data on aircraft activity by sec- tor. Special programs are required to download the information from the TRACONs and AFSSs for modeling purposes. Traffic counts at the ATCTs had to be tabulated manually. The adequacy of the data can be a problem even with automated data. For example, facility-level staffing requirements for the ARTS III TRACONs were determined by using work load data from a single busy day (STI 199 lb. 65). Because detailed automated work load data could not be retrieved for the ARTS 11 TRACONs, the ARTS III data were used, adjusting for differences in operating hours. For the ATCTs, the work load data were collected on an hourly basis, but the work data were collected in 15-min samples. An algo- rithm was developed to approximate work load distribution during the hour so that the work load data could be matched with the work time data that were collected for 15-min intervals (Schmeidler and D'Avanzo 1994, 32). The lack of close coupling between work and work load variables could affect the accuracy of the work time estimates. Selection, Training, and Reliability of Data Collectors The collection of data on ATCS performance in the noisy environment of an air traffic control facility is a nontrivial task that calls for a high level of proficiency. Therefore it is customary to carefully select, thoroughly train; and formally test the reliability of all observers before proceeding with a data collection program. The agency's account of the way in which observer-analysts were selected, trained, and tested is incomplete. Criteria for the selection of observer- analysts are vague. Evidence for how observers were trained is scant. Formal studies of inter- and intraobserver reliability, commonly conducted before large-scale data collection commences, are not reported. The staffing stan- dards reports contain little information on the techniques used for verifying the reliability of data collected by multiple analyst-observers. For example, one report states:

SHORTCOMINGS IN CURRENT STANDARDS 79 To ensure consistency and uniformity in data collection among the 05-3 observers, a comparison of observers' data was made. Two 05-3 observers monitored one radar po- sition simultaneously. The data were analyzed, results were discussed, and adjustments were made as necessary to minimize the variability among the 05-3 observers. (STI 1991b, 17) An adequate test of observer reliability would have calibrated the data col- lected by all of the analysts in a comprehensive test of inter- and intra- observer reliability. To the committee's knowledge, this has not been done in the development of any of the staffing standards. Areas for Improvement Weaknesses in the way the agency has collected work data may have affected their overall accuracy. However, there is no reason to suspect that these inaccuracies would have biased specific facility staffing estimates, that is, biased staffing estimates for one facility but not for another; they would have had a similar effect on the data collection at all of the sample sites. If a facil- ity bias exists, it is more likely due to work load and complexity data that were not collected or that were collected but not used in the models, a topic covered in the following section. Nevertheless, several improvements could be made to enhance the qual- ity of the data in future data collection efforts, particularly to provide more accurate facility-level data. First, a more solid theoretical framework for measurement could be obtained from a systematic top-down analysis of controller tasks. In Chapter 3, development of an automated task analysis— based model, which could provide this framework, is discussed as a possible approach. The committee examined but dismissed the use of a simulator as a way of obtaining a better understanding of the air traffic control work environment and controller tasks. A simulator is available at the FAA Tech Center in At- lantic City, but concerns about its cost and fidelity caused it to be rejected. Configuring the simulator to represent the main types of air traffic control facilities would be time-consuming and expensive. Assembling appropriate controller subjects, particularly if multiple staffing groups were to be simu- lated, would not be easy. Finally, the data gathered from a simulator would have to be validated with data collected at real facilities. Increasing the number of sites represented and the amount of work and work load data collected could improve the accuracy of facility-level staffing estimates, although at an increased cost of data collection. With more data from more sites, facilities—or sectors within or among facilities—with like characteristics could be grouped in subcategories for statistical testing and potential model development. FAA has estimated the contractor costs of

80 AIR TRAFFIC CONTROL FACILITIES gathering work samples (100 samples) at one site at $16,600. Inclusion of FAA staff time and travel costs raises the cost to more than $31,000 per site. This option requires a greater investment in data collection but offers no guarantee that significant differences between facilities or sectors can be found or that more facility-sensitive models can be developed. MODEliNG Analytic models are used in three places in the development of FAA's staffing standards (Figure B-i). Issues concerning the statistical validity of the mod- els are addressed in this section, particularly as they relate to the use of the models to predict staffing requirements for individual facilities. Choice of Model Form With the exception of the simulation model developed to explain controller work time at ATCTs, the work-time and forecasting models are based on lin- ear regression techniques. Linear regression is appropriate when the rela- tionship between the variables of interest, such as controller work time and work load variables, is linear and the probability of expected responses— controller work time in the work-time models and controller staffing in the forecasting models—is normally distributed around the work load variables. The decision to use linear regression for the TRACON and ARTCC work- time models was based on analyses of time-study samples. Correlation ma- trices and scatter plots indicated a positive and linear relationship between controller work time and work load (STI 1991a, 34; STI 1991b, 20)." Ac- cording to the ErgoTech review, nonlinear model forms were tried where the data appeared to warrant such an approach,12 but the added complexity of using nonlinear models did not appear to offer a significant improvement in explaining work time variability (ErgoTech, Inc. 1995, 26). Selection of an appropriate model is critical to the validity of the staffing standards ap- proach. More documentation of the results obtained with nonlinear func- tions and. statistical tests of the appropriateness of linear regression tech- niques would provide greater confidence that the correct model form has been selected. "The scatter plots might have looked different if data outliers had not been removed from the data base. Observations with the highest work time, calculated by dividing total productive work min- utes by the total number of aircraft worked for a 15-min period, were eliminated. The rationale for eliminating outliers is that they correspond to periods of very low aircraft traffic and thus are not representative of controller work during busy periods (STI 1991a, 31; STI 1991b, 19). 12 Model forms tried included generalized polynomial, exponential, and logarithmic functions (Ergotech, Inc. 1995, 26).

SHORTCOMINGS IN CURRENT STANDARDS 81 Representativeness of the Models One of the primary reasons why FAA's staffing standards may not be highly accurate predictors of staffing requirements for individual facilities lies in the treatment of the data inputs to the models. The key data in the work-time models—controller work times and aircraft activity measures—are treated as one statistical population. The work-time equations are estimated by using the pooled data without reporting whether statistical tests were conducted to determine whether significant differences exist in the data from the vari- ous sample sites. In the one instance where such tests were reported—for the AFSS staffing standard—the wrong test was applied (STI 1994, 26).13 Both of the outside reviews (ErgoTech, Inc. 1995, 65; Cullinane 1996, 52) ques- tioned the assumption that "one model fits all" without further verification that the data drawn from a limited number of sample sites are representative of all facilities.'4 Some effort was made to analyze subsets of the data (e.g., by sector size and type), particularly for TRACONs, as candidates for separate work load models (STI 1991a, 36). According to the technical experts, the characteris- tics of the subgroups were not sufficiently distinctive to warrant separate models (STI 1991b, 36). The decision to combine and analyze model input data as one statistical population eliminated the very source of variance among facilities that would have been desirable to capture in estimating staffing requirements for individual facilities. Similar concerns apply to the forecasting models. Each ARTCC and each AFSS has its own model based on facility-level air traffic forecasts, but the forecasting models for other types of facilities are developed by pooling the requisite data on operational characteristics and peak-period traffic across a range of facilities. Forecasting models were developed for two types of TRACONs and four types of ATCTs (Table B-2). Verification is needed to ensure that the models adequately represent the different facility types and that the results can be generalized. 13 Regression analysis was used to determine whether significant site variations existed. The re- gression approach indicates only that there is a positive correlation between the total time expended on each task (the dependent variable) and the total number of task occurrences (the independent variable) for all samples for all of the sites (STI 1994, 26). It says nothing about site variability. The appropriate test for site variation is an analysis of variance or means analysis to determine whether the mea.n time to perform a given task at one site is statistically different from the mean time to perform the same task at other sites. 14 Cullinane points out, for example, that the controller work load model for the ARTCCs was de- veloped using the data from three ARTCCs from the 1991 study and six ARTCCs from the 1985 study without any verification that all of the ARTCCs are similar enough to uniformly apply the models. He concluded that "without this evidence, it is inappropriate to apply the staffing standards models based upon the limited sample to the larger population" (Cullinane 1996, 52).

82 AIR TRAFFIC CONTROL FACILITIES Predictive Capability of the Models Modeling and quantifying the factors that affect controller work are chal- lenges that the current staffing standards have not fully addressed. During the data collection phase, much information was collected on the complex- ity of air traffic operations, as well as on the volume and density of air traf- fic, as factors affecting controller work (STI 1991a, 25; STI 1991b, 8-9). The complexity factors were not included in the final work-time models because it was found that such factors, singly or as a group, only affected the models to a small degree (STI 1991b, 22).15 The best predictors of total controller work time, at least for the TRACONS and ARTCCs, were measures of aircraft activity (STI 1991a, 39; STI 1991b, 23). With a limited number of variables, the work-time models are poor pre- dictors of variations in observed controller work time. For example, the single-controller work-time model for TRACONs explains only 46 percent of the variation in observed controller work time (ErgoTech, Inc. 1995, 25).16 To account for this large unexplained and unmodeled variation in controller work, an adjustment or "difficulty factor" was added to the work-time equa- tions to avoid understating the work time needed by controllers to separate aircraft under varying levels of difficulty. This arbitrary adjustment factor ac- counts for a sizeable amount of measurable work time in many of the work- time equations (Cullinane 1996, 17).17 The elimination of complexity factors and the substitution of a substantial and uniform difficulty factor help ex- plain why the models are not good predictors of differences in controller work time at different facilities (TRACONs) or for different sectors within these facilities (ARTCCs). The forecasting models appear to be better predictors of observed varia- tions in staffing requirements than the work-time models. The forecasting equations for the four types of ATCT facilities, for example, explain between 63 and 97 percent of the variation in the observed data (Schmeidler and D'Avanzo 1994, 63). Similar estimates are not reported for the TRACON forecasting models. 11 The effect of some of the complexity factors may have been captured indirectly in the time study data. For example, a controller working in a low-altitude arrival and departure sector may have taken longer to perform the same task than a controller working in a high-altitude sector. 16 Similar numbers were not reported for the multiple-controller regression model for TRACONs or for the work-time models developed for the ARTCCs. 17 For the single- and multiple-controller work-time models for the TRACONs, the difficulty factor accounts for 2.46 and 2.23 min relative to measured work of 1.726 and 1.0367 mm, respectively, for a 15-min observed work interval (STI 1991a, B-7-13-8). For the ARTCCs, the difficulty factor accounts for 2.53 and 2.46 min relative to measured work of 2.66 and 2.49 min for the two- controller, low- and high-altitude sector work-time models, respectively (STI 1991b, C-1—C-2), and 3.64 and 2.25 min relative to measured work of 6.96 and 1.68 min for the one-controller and three-controller work-time models, respectively (STI 1991b, D-1, E-1).

SHORTCOMINGS IN CURRENT STANDARDS 83 Sensitivity Estimates Both the work-time and the forecasting models contain numerous adjust- ment factors and assumptions that are potential sources of error. Among the more important are the choice of the 90th percentile busiest day and the 1.76 adjustment factor in the forecasting models. FAA has selected the 90th percentile busiest day (the 37th-busiest day) as the target for forecasting staffing requirements to ensure that the number of controllers is adequate to cover air traffic for 90 percent of all days; overtime is used, if necessary, to support the extra staffing needed on the 10 percent of the days when air traffic is higher. The agency has conducted a sensitivity analysis indicating that, for TRACONs, selection of the 70th rather than the 90th percentile day would have only a modest effect on staffing requirements at individual facilities (FAA 1994a).18 A similar type of analysis could be con- ducted for the ARTCCs, the ATCTs, and the AFSSs. However, it is not ap- parent that selection of a different percentile day would affect one facility staffing estimate differently from another; rather it would affect the aggregate estimates. Assumptions about facility operating practices also can have a large effect on the accuracy of the staffing forecasts for individual facilities. Currently, the staffing standards apply a uniform adjustment factor that nearly doubles staffing estimates. The 1.76 factor, which represents a systemwide average, takes into account 7-day facility operation and time for off-position activi- ties. The case study on staffing requirements for the New York TRACON (ERAT 1995) discussed in Chapter 2 illustrates how different assumptions about the amount of time off position can result in different estimates of staffing requirements. Specifically, the staffing model developed by the East- ern Region Assessment Team (ERAT) incorporated more off-position time for the New York TRACON than was assumed in the 1.76 factor used in the headquarters staffing standard. The result was a higher adjustment factor, hence staffing number, for the New York TRACON using the locally sensi- tive ERAT model. Of course, applying the higher New York adjustment fac- tor to all TRACONs would probably not be acceptable from a budgeting standpoint. One other adjustment—for weekend versus weekday traffic—is a prob- lem because of its exclusion rather than its inclusion in the application of the staffing standards. FAA data on traffic activity indicate that, for many facili- ties, air traffic is lighter on the weekends. However, the current staffing stan- IS The analysis indicated an increase of one controller assuming a 90th percentile day rather than a 70th percentile day at the Birmingham, Dallas, and Dayton TRACONs. It indicated an increase of two controllers for the Boston, Oakland, Philadelphia, and St. Louis TRACONs. Of course, with more than 17,000 controllers nationwide, small differences for each facility can make a significant difference in the aggregate.

84 AIR TRAFFIC CONTROL FACILITIES dards do not distinguish between weekdays and weekends in applying the work-time models and hence may overstate staffing requirements at certain facilities. Areas for Improvement Perhaps the most promising area for improving the staffing standards for ap- plication to individual facilities is to explore the development of work-time models that more adequately represent the different facility types, or the dif- ferent types of sectors within or among facilities. This could be accomplished in two ways. First, more data on work times and aircraft activity levels could be collected at more facilities, which would expand the sample size for sta- tistical testing. State-of-the-art clustering techniques could then be used on a trial basis to group like facilities, or like sectors within or among facilities, to support separate model development for these clusters if statistical testing shows significant differences between the groups. More representative work- time models would reduce model variance, increase model validity, and im- prove model predictive capability (ErgoTech, Inc. 1995, 5). Of course, more data collection would be costly, and there is no guarantee that the additional data would better discriminate among facilities or sectors. Second, more effort to define, operationalize, and include complexity fac- tors in the work-time models could improve the ability of the models to ex- plain variances in controller work time across facilities and sectors. The more that differences in controller performance can be explained by the models, the less the need for arbitrary adjustment factors and the greater the validity of the models. However, previous efforts to measure and model the effects of complexity factors on controller performance have not proved successful, so the payoff of this option is uncertain. Another area that warrants investigation is the choice of assumptions and adjustment factors built into the models. The validity of these assumptions (e.g., numbers of shifts, amount of time off position) should be tested, the ef- fects of using different assumptions examined, and the potential for using different assumptions and adjustment factors for different types of facilities considered. Finally, better model validation procedures should be developed. Exter- nal performance measures should be developed on the basis of information about the operating and performance characteristics of individual facilities. For example, level of service indicators, such as the amount of traffic throughput or delays attributable to air traffic control service, are worth ex- ploring for their links with staffing levels. These data can begin to provide a standard against which the validity of modeled staffing estimates can be com- pared. In all cases the staffing models should be validated at the level at which

SHORTCOMINGS IN CURRENT STANDARDS 85 they are being used. If the models are used to provide regional estimates for budgeting purposes, their validity should be tested at the regional level of aggregation. If they are intended to predict individual facility staffing requirements, they should be validated facility by facility. APPUCATION OF THE STAFFING STANDARDS FAA has indicated to Congress that the primary use of the staffing standards is for budgeting and resource allocation purposes at the national and regional levels (FAA 1996, 4; FAA 1994b, 2). Discrepancies between FAA headquar- ters model estimates and regionally determined staffing requirements for in- dividual facilities reflect use of the staffing standards in the current staffing decision process. - Procedures for Estimating Local Staffing Requirements Large discrepancies between FAA headquarters and regional staffing esti- mates for individual facilities reflect different processes used by the regions to determine facility staffing requirements. FAA headquarters produces mod- eled staffing estimates for each facility, and the FAA regions produce their own estimates of facility staffing requirements. The regions stay within the aggregate funding allocations provided by FAA headquarters, but they adjust facility-level staffing estimates and make final staffing allocations on the basis of operational circumstances at individual facilities within their respective jurisdictions. Moreover, each FAA region has its own way of estimating staffing requirements and allocating funds for positions to individual facilities. For example, the Western Pacific Region has developed a "bottom-up," position-based approach to developing estimates of facility-level staffing requirements. Elements of the FAA headquarters models are used, such as selection of the 37th-busiest day for position staffing determination, but the work-time and forecasting models are not. For other facilities, such as the New York TRACON, special models have been developed to reflect local conditions (ERAT 1995). Thus, there is no apparent consistency in staffing approaches across the regions. Other Factors Affecting Local Staffing Estimates FAA headquarters model-driven estimates of staffing requirements for indi- vidual facilities are likely to remain imprecise because staffing decisions are

86 AIR TRAFFIC CONTROL FACILITIES not exclusively driven by the staffing standards. Actual staffing depends on a variety of other considerations. For example, budgetary cutbacks can re- sult in fewer staff than might be estimated using either the FAA headquar- ters models or regional staffing determination methods. Conversely, special congressional appropriations and directives can result in placement of addi- tional controllers at specific locations. Finally, because of limited new hires, insufficient funds to move experienced controllers from one facility to an- other, and the continuing difficulty of inducing controllers to relocate to high-cost and low-desirability locations, imbalances between forecast and actual staffing at some facilities are likely to persist. Areas for Improvement Centralized, model-driven estimates of facility staffing requirements are best used to complement the judgment of experts, who can also consider local op- erating circumstances that influence staffing needs at individual facilities. There are many legitimate considerations affecting staffing decisions that are not accounted for in the models. That being said, consensus on appropriate staffing levels for individual facil- ities could be improved by adoption of a consistent approach across the FAA regions for determining staffing requirements and making final allocation de- cisions. FAA headquarters should take the lead in identifying regional "best practice" staffing methods and should encourage sharing of this information across the regions. SUMMARY This review of FAA's current staffing standards suggests several problem areas that could affect the adequacy of the models for estimating staffing re- quirements at individual facilities. First, an adequate theoretical framework for data collection is lacking, and there is a potential for sampling errors and human error in reliably measuring controller work. These problems could affect the accuracy of key data inputs to the staffing standards. Second, pool- ing of the data on work and work load variables without verifying that the sample data are representative of the underlying population of facilities raises statistical concerns. It also eliminates potentially significant differences be- tween facilities, which could improve the precision of facility estimates or support development of separate models for clusters of like facilities or sec- tors. Third, use of numerous and, in some cases, arbitrarily determined ad- justment factors and assumptions is a potential source of error, particularly for estimating facility-specific staffing requirements. Fourth, inadequate val-

SHORTCOMINGS IN CURRENT STANDARDS 87 idation procedures, particularly lack of an external standard against which to compare modeled estimates, make it impossible to test the correctness of the estimates provided by the staffing standards. Finally, use of alternative methods by the FAA regions for estimating staffing requirements for facili- ties in their regions and factors external to any staffing model are likely to re- sult in continuing discrepancies between FAA headquarters-generated and regional estimates of staffing requirements for some facilities. Several improvements are suggested that could make the FAA headquar- ters staffing standards better predictors of local facility staffing requirements. They include a more solid theoretical framework to guide data collection; better selection, training, and testing of the reliability of data collectors; re- estimation of work-time models to include more of the complexity factors that explain differences between facilities; larger sample sizes, increasing the amount of measurement data, to support analysis of the potential for devel- oping different models for different types of facilities or sectors; conduct of sensitivity analyses to determine the effects on staffing estimates of key model assumptions and adjustment factors; development of appropriate measures for model validation; and development of a uniform approach across the FAA regions for determining staffing requirements for individual facilities. REFERENCES ABBREVIATIONS ERAT Eastern Region Assessment Team FAA Federal Aviation Administration STI Standards Technology, Inc. TAI Technology Applications, Inc. Cullinane, T.P. 1996. A Review of Staffing Standards Development Procedures for FAA Air Route Traffic Control Centers. DTFAO1-93-C-00067, Task 1. Washington Consulting Group; FAA, U.S. Department of Transportation, Jan. 25, 57 pp. EP.AT. 1995. New York TRACON Controller Staffing. FAA, U.S Department of Trans- portation,Jan. 13, 21 pp. ErgoTech, Inc. 1994. Evaluation of the Applicability of Cognitive Engineering Methods to the Derivation of Staffing Standards for Air Traffic Controllers: A Pilot Study. West Lafayette, Ind.; FAA, U.S. Department of Transportation, Nov. 13,56 pp. ErgoTech, Inc. 1995. Review of the FAA Staffing Standards for Terminal Radar Ap- proach Control (TRACON) Facilities. West Lafayette, Ind.; FAA, U.S. Depart- ment of Transportation, Feb. 1,87 pp. FAA. 1994a. TRACON Staffing Standard Review: Analysis of 90th Percentile Day. U.S. Department of Transportation, Sept. 29. FAA. 1994b. Report to Congress: Controller Staffing Requirements. Report to the House and Senate Appropriations Committees pursuant to House Report 102-156, House Report 102-639, and Senate Report 102-351. U.S. Department of Trans- portation, April.

88 AIR TRAFFIC CONTROL FACILITIES FAA. 1995. Revalidation Study of FAA Staffing Standards for Terminal Radar Approach Control Facilities: Data Analysis and Technical Report. Management Engineer- ing Branch, Jan. 6. FAA. 1996. Report to Congress: Air Traffic Controller Staffing Requirements. Report to the House Transportation Infrastructure Committee and the Senate Com- merce, Science, and Transportation Committee pursuant to Section 120 of Public Law 102-581. U.S. Department of Transportation, March. Meister, D. 1985. Behavioral Analysis and Measurement Methods. John Wiley and Sons, New York. Schmeidler, N.F., andJj. D'Avanzo. 1994. Development of Staffing Standards for Air Traffic Control Functions in Tower Cabs. Technical Report. DTFA01-89- Y-01037. Operational Technologies Services, Inc., Vienna, Va., and OMNI En- gineering & Technology, Inc., McLean, Va., Sept. 21. STI. 1991a. FAA Staffing StandardsforAirRoute Traffic Control Centers. Validation and Revision Report. DTFA01-88-Y-01016, Work Order 5. Rockville, Md., Nov. 15. STI. 1991b. FAA Staffing Standards for Terminal Radar Approach Control Facilities. Technical Report. DTFA01-88-Y-01016, Work Order 1. Rockville, Md., Nov. 15. STI. 1994. FAA Staffing Standards for Automated Flight Service Stations. Technical Re- port. DTFAO1-87-Y-01016. Rockville, Md., Nov. 15. TAI. 1985. Development of Air Route Traffic Control Center Staffing Standards. Con- tract DTFAO1-83-Y-30548. Falls Church, Va., Sept. 6. Van Cott, H., and B. Paramore. 1988. Task Analysis (Chapter 7.3). In The job Analy- sis Handbook for Business, Industry, and Government (S. Gael, ed), John Wiley and Sons, New York.

STUDY COAAITTIE BIOGRAPHICAL INfORMATION AARON COHEN, Chairman, is Zachry Professor of Engineering in the Col- lege of Engineering at Texas A&M University. Mr. Cohen received a bach- elor's degree from Texas A&M University and a master's degree from the Stevens Institute of Technology. He spent 30 years with the National Aero- nautics and Space Administration (NASA). He held numerous positions at NASA's Johnson Space Center: Director of the Space Center, Director of Research and Engineering, Manager of the Space Shuttle Orbiter Project, and Manager of the Apollo Command and Service Modules. When he left NASA, he was serving as Acting Deputy Administrator. Before coming to NASA, he worked for General Dynamics Corporation and RCA and served in the United States Army. Mr. Cohen is an honorary fellow of the Ameri- can Institute of Aeronautics and Astronautics, a fellow of the American Astronautical Society and the American Society of Mechanical Engineers, and a member of the International Academy of Astronautics. A member of the National Academy of Engineering since 1988, he recently served as chairman of the Transportation Research Board's (TRB's) Committee for Assessment of Capacity and Demand for the National Advanced Driving Simulator. CHARLES B. AALFS retired in 1994 from a 25-year career with the Federal Aviation Administration (FAA) Air Traffic Control Service. An expert in automation of air traffic systems, Mr. Aalfs worked in numerous super- visory and operational positions in air traffic control. His experience covers several air traffic control environments, including terminal radar approach control (TRACON) facilities and air traffic control towers. Most recently, he was Air Traffic Control Manager for the Southern California TRACON. Before that he worked for the FAA Western Pacific Region. Mr. Aalfs is a member of the Commission on Behavioral and Social Sciences and Education (CBASSE) Panel on Human Factors in Air Traffic Control Automation.

90 AIR TRAFFIC CONTROL FACILITIES RUSSELL A. BENEL is Principal Scientist in System Engineering and Analysis at the Center for Advanced Aviation Systems Development at the MITRE Corporation. An expert in human factors, Dr. Benel received his bachelor of arts from Washington and Jefferson College, his master of art in psychology from Fairleigh Dickinson University, and his Ph.D. in experimental psy- chology from the University of Illinois at Urbana-Champaign. Before com- ing to MITRE, he worked as Manager of Human Factors Engineering for air traffic control at IBM, where he was responsible for all human engineering efforts applied to the Advanced Automation System Program for FAA; Tech- nical Director and Secretary-Treasurer for User System Technology, Inc.; Se- nior Research Scientist and Manager in the Human Engineering Department of Essex Corporation; and Postdoctoral Associate at the United States Air Force School of Aerospace Medicine. Dr. Benel is a member of the Human Factors and Ergonomics Society. GEORGE J. COULURIS is Vice President of Research Operations at Seagull Technology, Inc., where he is evaluating the potential operational, technical, and economic effects of future air traffic management systems. Dr. Couluris has more than 25 years of experience in aviatioii research, operations re- search and analysis, and program management. He has specialized in air traf- fic systems development, human operator work load analysis, and analytic modeling of air traffic operations. His previous positions include Director of Operations and Senior Operations Analyst at IAT Corporation; Program Director at ATAC; and Department Director, Program Manager, and Senior Systems Analyst at SRI International. Dr. Couluris received his bachelor and master of science degrees at Rensselaer Polytechnic Institute and his Ph.D. in civil engineering at the University of California, Berkeley. He is a member of the Air Traffic Control Association and a past member of the TRB Committee on Airfield and Airspace Capacity and Delay. PIUS J. EGBELU is Professor and Chair of the Department of Industrial and Manufacturing Systems Engineering at Iowa State University. Dr. Egbelu re- ceived his bachelor's degree at Louisiana Tech University and his master's de- gree and Ph.D. in industrial engineering and operations research at Virginia Polytechnic Institute and State University. He is an expert in material handling, robotics, production planning and control, manufacturing systems analysis, and operations research. Dr. Egbelu has taught at Pennsylvania State Univer- sity and Syracuse University. He also served as Program Director of the Oper- ations Research and Production Systems Programs at the National Science Foundation. He is a member of the Institute of Industrial Engineers and the Society of Manufacturing Engineers and is a member and vice chairman of the United States Academic Coalition for Intelligent Manufacturing Systems.

STUDY COMMITTEE BIOGRAPHICAL INFORMATION 91 JOE D. HINsON is Vice President of Global Operations Planning at Federal Express. He is responsible for development of computer models and statisti- cal and simulation applications to support long-range planning decisions about appropriate sizing of facilities and productivity standards for national hub operations. Mr. Hinson joined Federal Express as an Operations Re- search Analyst in 1974 and served in a variety of management positions be- fore becoming Managing Director of Operations Research. He received his bachelor of arts degree and M.B.A. from the University of Memphis. Mr. Hin- son is a member and publication officer of the Airline Group of the Inter- national Federation of Operational Research Societies and is a member of the Institute for Operations Research and the Management Sciences (INFORMS). He serves on the Research Advisory Board of the Massachusetts Institute of Technology Industry Cooperative Research Program. PAUL F. HOGAN is Vice President and Senior Economist at The Lewin Group. He received his bachelor of arts in economics from the University of Virginia and his master of science in applied economics and finance from the Graduate School of Management at the University of Rochester, where he un- dertook additional doctoral studies. He has more than 20 years of experience in applying microeconomics, statistics, and operations research methods to problems in labor economics, including labor supply and demand, efficient staffing methods, and performance and cost measurement. Mr. Hogan is a nationally recognized expert on policies for staffing and compensating the all-volunteer armed forces. He served as the Senior Analyst on the President's Military Manpower Task Force and as Director of Manpower Planning and Analysis in the Office of the Secretary of Defense. Mr. Hogan is a member of the American Economics Association, the Military Operations Research Society, and the Defense Manpower Roundtable. WILLIAM C. HOWELL is Executive Director of the Science Directorate of the American Psychological Association (APA). He is an expert in human infor- mation processing and decision making. Before joining APA, Dr. Howell was the Chief Scientist at the Air Force Human Resources Laboratory. He re- ceived his bachelor's and master's degrees and Ph.D. in psychology from the University of Virginia. Dr. Howell has held several teaching positions at Rice University, where he was Herbert S. Autrey Professor of Psychology and, be- fore that, Chairman of the Department of Psychology; at Ohio State Univer- sity he was Director of the Human Performance Center and Professor of Psy- chology. Dr. Howell is a fellow of the Human Factors Society and the American Psychological Association and is a member of the Psychonomic Society. He currently serves on the CBASSE Committee on Human Factors. DONALD A. KIMBALL retired from FAA after a 36-year career that included positions as manager, supervisor, and controller in five air traffic control tow-

92 AIR TRAFFIC CONTROL FACILITIES ers and TRACONs and as instructor at FAA's Academy in Oklahoma City. At the time of his retirement, he was the Manager of Air Traffic Career Systems at FAA Headquarters. Mr. Kimball was involved in numerous personnel is- sues with major responsibilities for labor-management relations and served as a negotiator for two national labor agreements. He managed the air traffic manager and supervisor assessment and selection programs, with emphasis on the employee partnership and problem-solving initiatives. He was also involved in an early version of the terminal staffing standards. THOMAS M. MCARDLE is Vice President of SABRE Decision Technologies, a sister company of American Airlines, where he has worked for the past 10 years applying operations research tools to airport and airspace design and capacity issues. Before that he worked as a Senior Operations Analyst for General Dynamics, where he applied optimization and simulation tech- niques to aircraft threat analysis. Dr. McArdle received his bachelor of busi- ness administration in finance from the University of Houston and his mas- ter of science in statistics and his Ph.D. in industrial engineering from Texas A&M University. He is a member of the Airports Council International, the American Association of Airport Executives, and INFORMS, and he serves on the Board of Governors of the Airport Consultants Council. NORMAN T. O'MEARA is a Research Fellow with Logistics Management In- stitute. He has had more than 25 years of experience in the development and application of operations research and mathematical tools for manpower pol- icy analysis and force structure planning for the U.S. Army. Dr. O'Meara has worked as Deputy Chief Scientist at the U.S. Army Strategic Defense Com- mand, Chief of Research Activity for the U.S. Army Training and Doctrine Command at the RAND Corporation, Chief of Officer Plans at the U.S. Army Military Personnel Center, and Associate Professor of Mathematics at the United States Military Academy. He received his bachelor of science degree at the U.S. Military Academy, his master of science degrees in mathematics and operations research and statistics at Rensselaer Polytechnic Institute, and his Ph.D. in operations research at George Washington University. PHILIP J. SMITH is a Professor in the Department of Industrial and Systems Engineering at The Ohio State University, where he has taught since 1980. He received his bachelor's degree in psychology, his master's degree in in- dustrial and operations engineering, and his Ph.D. in cognitive psychology and industrial and operations engineering at the University of Michigan. Dr. Smith is an expert in the design of cooperative problem-solving systems with applications in aviation. He teaches courses in cognitive systems engineer- ing, artificial intelligence, human-computer interaction, and the design of co- operative problem-solving systems. Dr. Smith is a member of the American Society for Information Science, the Association for Computing Machinery,

STUDY COMMITTEE BIOGRAPHICAL INFORMATION 93 the Human Factors and Ergonomics Society, and the Institute of Electrical and Electronics Engineers Society for Systems, Man and Cybernetics. KAY M. STANNEY is an Assistant Professor in the Department of Industrial Engineering at the University of Central Florida. An expert in human factors engineering, Dr. Stanney teaches courses in work measurement, probability and statistics for engineers, engineering economy, and human-computer interaction. She has also conducted research on productivity improvement, including the evaluation of work methods and time standards, at the NASA Kennedy Space Center. Dr. Stanney received her bachelor's degree in indus- trial engineering at the State University of New York at Buffalo and her master's degree and Ph.D. in human factors and industrial engineering at Pur- due University. She is a senior member of the Institute of Industrial Engineers and is a member of the Human Factors and Ergonomics Society and the Association for Computing Machinery.

The Transportation Research Board is a unit of the National Research Coun- cil, which serves the National Academy of Sciences and the National Academy of Engineering. The Board's purpose is to stimulate research concerning the nature and performance of transportation systems, to disseminate the infor- mation produced by the research, and to encourage the application of appro- .priate research findings. The Board's program is carried out by more than 400 committees, task forces, and panels composed of nearly 4,000 administrators, engineers, social scientists, attorneys, educators, and others concerned with transportation; they serve without compensation. The program is supported by state transportation and highway departments, the modal administrations of the U.S. Department of Transportation, and other organizations and indi- viduals interested in the development of transportation.' The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering re search, dedicated to the furtherance of science and technology, and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on 'scientific and technical matters. Dr. Bruce M. Alberts is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of out- standing engineers. It is autonomous in its administration and in the selec- tion of its members, sharing with 'the National Academy of Sciences the re- sponsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at neeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. William A. Wulf is president of the National Academy of Engineering. ' The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the'services of eminent members of appropriate profes- sions in the examination of policy matters pertaining to the health of the pub- lic. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal govern- ment and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth 1. Shine is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and tech- nology with the Academy's purpose of furthering 'knowledge and advising the federal government. Functioning in accordance with general policies de- termined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both the Academies and the Institute of Medicine. Dr. Bruce M. Alberts and Dr. William A. Wulf are chairman and vice chairman, respectively, of the National Research Council.

IENSPLORTTTON R[S[RC'HIBiO'RD] MARCH CrOUNC.a 2 OjLoNiIIuiIoN AvCNufjN1/ 4SHIN6jONiDTC. 2Oj8 F ROFI1.OR.POSIAG[PAID N1N, D.C. N018970