Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
i Evolving the High Performance Computing and Communications Initiative to Support the Nation's Information Infrastructure Committee to Study High Performance Computing and Communications: Status of a Major Initiative Computer Science and Telecommunications Board Commission on Physical Sciences, Mathematics, and Applications National Research Council NATIONAL ACADEMY PRESS Washington, D.C. 1995
ii NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose mem- bers are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee con- sisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce Alberts is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineer- ing programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Robert M. White is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initia- tive, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of sci- ence and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce Alberts and Dr. Robert M. White are chairman and vice chairman, respectively, of the National Research Council. Support for this project was provided by the U.S. Department of Defense through the Advanced Research Projects Agency under Grant No. MDA972-94-1-0008. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the Department of Defense or the Advanced Research Projects Agency. Library of Congress Catalog Card Number 95-67707 International Standard Book Number 0-309-05277-7 Copyright 1995 by the National Academy of Sciences. All rights reserved. Available from: National Academy Press 2101 Constitution Avenue, NW Washington, DC 20418 B-540 Printed in the United States of America
iii COMMITTEE TO STUDY HIGH-PERFORMANCE COMPUTING AND COMMUNICATIONS: STATUS OF A MAJOR INITIATIVE FREDERICK P. BROOKS, JR., University of North Carolina at Chapel Hill, Co-chair IVAN E. SUTHERLAND, Sun Microsystems Laboratories, Co-chair ERICH BLOCH, Council on Competitiveness DEBORAH ESTRIN, University of Southern California/Information Sciences Institute JOHN HENNESSY, Stanford University BUTLER W. LAMPSON, Digital Equipment Corporation EDWARD D. LAZOWSKA, University of Washington WILLIAM A. LESTER, JR., University of California at Berkeley JANE PRESTON, Telemedical Interactive Consultative Services Inc. W. DAVID SINCOSKIE, Bell Communications Research Inc. LARRY SMARR, National Center for Supercomputing Applications/University of Illinois at Urbana-Champaign JOSEPH F. TRAUB, Columbia University Staff MARJORY S. BLUMENTHAL, Director JAMES E. MALLORY, Staff Officer LESLIE M. WADE, Project Assistant
iv COMPUTER SCIENCE AND TELECOMMUNICATIONS BOARD WILLIAM WULF, University of Virginia, Chair FRANCES ALLEN, IBM T.J. Watson Research Center JEFF DOZIER, University of California at Santa Barbara DAVID J. FARBER, University of Pennsylvania HENRY FUCHS, University of North Carolina CHARLES M. GESCHKE, Adobe Systems Inc. JAMES GRAY, San Francisco, California BARBARA J. GROSS, Harvard University DEBORAH A. JOSEPH, University of Wisconsin RICHARD M. KARP, University of California at Berkeley BUTLER W. LAMPSON, Digital Equipment Corporation BARBARA H. LISKOV, Massachusetts Institute of Technology JOHN MAJOR, Motorola Inc. ROBERT L. MARTIN, AT&T Network Systems DAVID G. MESSERSCHMITT, University of California at Berkeley WILLIAM H. PRESS, Harvard University CHARLES L. SEITZ, Myricom Inc. EDWARD SHORTLIFFE, Stanford University School of Medicine CASMIR S. SKRZYPCZAK, NYNEX Corporation LESLIE L. VADASZ, Intel Corporation MARJORY S. BLUMENTHAL, Director LOUISE A. ARNHEIM, Senior Staff Officer HERBERT S. LIN, Senior Staff Officer JAMES E. MALLORY, Staff Officer RENEE A. HAWKINS, Staff Associate JOHN M. GODFREY, Research Associate GLORIA P. BEMAH, Administrative Assistant LESLIE M. WADE, Project Assistant
v COMMISSION ON PHYSICAL SCIENCES, MATHEMATICS, AND APPLICATIONS RICHARD N. ZARE, Stanford University, Chair RICHARD S. NICHOLSON, American Association for the Advancement of Science, Vice Chair STEPHEN L. ADLER, Institute for Advanced Study SYLVIA T. CEYER, Massachusetts Institute of Technology SUSAN L. GRAHAM, University of California at Berkeley ROBERT J. HERMANN, United Technologies Corporation RHONDA J. HUGHES, Bryn Mawr College SHIRLEY A. JACKSON, Rutgers University KENNETH I. KELLERMANN, National Radio Astronomy Observatory HANS MARK, University of Texas at Austin THOMAS A. PRINCE, California Institute of Technology JEROME SACKS, National Institute of Statistical Sciences L.E. SCRIVEN, University of Minnesota A. RICHARD SEEBASS III, University of Colorado LEON T. SILVER, California Institute of Technology CHARLES P. SLICHTER, University of Illinois at Urbana-Champaign ALVIN W. TRIVELPIECE, Oak Ridge National Laboratory SHMUEL WINOGRAD, IBM T.J. Watson Research Center CHARLES A. ZRAKET, MITRE Corporation (retired) NORMAN METZGER, Executive Director
PREFACE vii Preface In early 1994, acting through the Defense Authorization Act for FY 1994 (Public Law 103160), Congress asked the National Research Council (NRC) to examine the status of the High-Performance Computing and Communications Initiative (HPCCI). Broad-based interest in and support for the HPCCI exist. Given its scope and size, concerns had been raised about its goals, management, and progress. Congress asked that at a minimum the study address: â¢ The basic underlying rationale(s) for the program, including the appropriate balance between federal efforts and private-sector efforts; â¢ The appropriateness of its goals and directions; â¢ The balance between various elements of the program; â¢ The effectiveness of the mechanisms for obtaining the views of industry and users for the planning and implementation of the program; â¢ The likelihood that the various goals of the program will be achieved; â¢ The management and coordination of the program; and â¢ The relationship of the program to other federal support of high-performance computing and communications, including acquisition of high-performance computers by federal departments and agencies in support of the mission needs of these departments and agencies. For this study the NRC's Computer Science and Telecommunications Board (CSTB) convened a committee of 12 members, expert on pioneering applications of computers and communications and the major components of the HPCCI: High-Performance Computing Systems, Advanced Software Technology and Algorithms, the National Research and Education Network, Basic Research and Human Resources, and Information Infrastructure Technology and Applications. Congress asked the committee to accelerate the normal NRC study process in order to provide an interim report by July 1, 1994, and a final report by February 1, 1995. The committee was able to meet this rapid turnaround by drawing on the knowledge and experience of its members as expressed in committee deliberations and by obtaining input from numerous outside experts. The full committee met six times between March 10, 1994, and December 20, 1994, to hear more than 25 high-performance computing and communications users, builders, and scientists; to discuss the HPCCI in detail; and to produce this report. Additionally, smaller groups of committee members made site visits to discuss first- hand the use of high-performance technologies. These visits involved another 6 individuals from the Ford Motor Company and approximately 50 high-performance computing and communications users who had gathered at a workshop to discuss the use of high-performance systems in environmental research and simulation. In addition to examining the current status of the program, the committee considered the evolution of the HPCCI
PREFACE viii and its goals and alternate government investment strategies related to technological development. The committee took into account varying perspectives on the initiative's goals and available assessments of progress toward achieving them. The committee's interim report provided technical background and perspective on the overall development of high-performance computing and communications systems, as well as on the HPCCI, formally started in 1991.1 The interim report made two recommendations: (1) strengthen the National Coordination Office to help it meet increasing future demands for program coordination and information functions; and (2) immediately appoint the congressionally mandated HPCCI Advisory Committee to provide broad-based, active input to the initiative. As this final report goes to press, both recommendations have yet to be acted on and thus require executive and legislative attention. This final report, Evolving the High-Performance Computing and Communications Initiative to Support the Nation's Information Infrastructure, purposely adopted a broad perspective so as to examine the HPCCI within the context of the evolving information infrastructure and national economic competitiveness generally. Committee deliberations consistently pointed out the important contributions that computing and communications research have made to the nation's economy, scientific research, national defense, and social fabric. That research has nourished U.S. leadership in information technology goods, services, and applications. Traditionally, the most powerful computers and the fastest networks made many of those contributions. Recently, however, the widespread availability of significant computing and communications capabilities on the nation's desktops and factory floors has also produced many benefits. The broadening and interconnection of more and more computer- based systems call attention to research needs associated with system scale as well as performance. The Committee to Study High Performance Computing and Communications: Status of a Major Initiative is grateful for the help, encouragement, and hard work of the NRC staff working with us: Marjory Blumenthal, Jim Mallory, Susan Maurizi, and Leslie Wade. Our meetings went smoothly because of their careful preparation. This report came together because of their attention and diligence. They patiently assembled sometimes conflicting text from diverse authors and helped reconcile it with the critique of our reviewers. They searched out facts of importance to our deliberations. Their excellent staff preparation helped us focus on the substance of our task. Many others also made valuable contributions to the committee. In addition to individuals listed in Appendix F who briefed the committee, the committee appreciates inputs from Sally Howe and Don Austin (National Coordination Office); Bob Borchers, Paul Young, Dick Kaplan, and Robert Voight (National Science Foundation); Eric Cooper (FORE Systems); Stephen Squires (Advanced Research Projects Agency); Sandy MacDonald (National Oceanic and Atmospheric Administration); Robert Bonometti (Office of Science and Technology Policy); and Al Rosenheck (former congressional staffer). It is also grateful to the anonymous reviewers who helped to sharpen and focus the report with their insightful comments. Responsibility for the report, of course, remains with the committee. 1 Computer Science and Telecommunications Board (CSTB), National Research Council. 1994. Interim Report on the Status of the High-Performance Computing and Communications Initiative. Computer Science and Telecommunications Board, Washington, D.C.
PREFACE ix Finally, we want to acknowledge the contributions to our present task of research projects 30 years past. Of course this text was all word processed. Of course the charts were drafted on computers. Of course we used Internet communication nationwide to plan our meetings, share our thoughts, reconcile our differences, and assemble our report. We plugged our portable computers into a local network at each of our meetings, sending drafts to local laser printers. In short, we have partaken fully of the fruits of the HPCCI's precursors. We thank the visionaries of the past for our tools. Frederick P. Brooks Ivan E. Sutherland Co-chairs Committee to Study High Performance Computing and Communications: Status of a Major Initiative
CONTENTS xi Contents EXECUTIVE SUMMARY 1 1 U.S. LEADERSHIP IN INFORMATION TECHNOLOGY 13 Information Technology Is Central to Our Society 13 Information Technology Advances Rapidly 14 Retaining Leadership in Information Technology Is Vital to the Nation 15 The Federal Investment in Computing Research Has Paid Rich Dividends 16 Continued Federal Investment Is Necessary to Sustain Our Lead 23 Today the HPCCI Is the Umbrella for Most Government-sponsored Computing and Communi- 25 cations Research Notes 27 2 THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 28 HPCCI: Goals and Emphases 28 Basic Objectives 28 Expanded Objectives 30 HPCCI Accomplishment 31 The Issue of Measurement 31 Better Computing and Computational Infrastructure 32 Increasing Researcher-Developer-User Synergy 32 Impact on Mission Agencies 36 Five Gigabit Testbed Projects: Collaboration and Impact 36 Evolution of HPCCI Goals and Objectives 37 Improving the Information Infrastructure 37 Evolving Research Directions and Relevance for the Information Infrastructure 39 Overall Computing and Communications R&D Planning 41 Toward a Better Balance 42 Moving ForwardâBasic Issues 42 Balance of Private and Public Investment 42 Coordination Versus Management 43 Coordinating Structure 45 Budget 49 Notes 49
CONTENTS xii 3 RECOMMENDATIONS 53 General Recommendations 53 Recommendations on High-Performance Computing 54 Recommendations on Networking and Information Infrastructure 57 Recommendations on the Supercomputer Centers and Grand Challenge Program 59 Recommendations on Coordination and Program Management in the HPCCI 61 Comments Relating This Report's Recommendations for High-Performance Computing and 64 Communications Research to Administration Priorities Notes 66 BIBLIOGRAPHY 67 APPENDIXES A The High-Performance Computing and Communications Initiative: Background 77 B High-Performance Communications Technology and Infrastructure 92 C Review of the High-Performance Computing and Communications Initiative Budget 97 D Current High-Performance Computing and Communications Initiative Grand Challenge Activi- 105 ties E Accomplishments of National Science Foundation Supercomputer Centers 108 F Individuals Providing Briefings to the Committee 118
PREFACE xiii Industry, if left to itself, will naturally find its way to the most useful and profitable employment. Whence it is inferred that manufacturers, without the aid of government, will grow up as soon and as fast as the natural state of things and the interest of the community may require. Against the solidity of this hypothesis . . . very cogent reasons may be offered . . . [including] the strong influence of habit; the spirit of imitation; the fear of want of success in untried enterprises; the intrinsic difficulties incident to first essays towards [competition with established foreign players]: the bounties, premiums, and other artificial encouragements with which foreign nations second the exertions of their own citizens ... To produce the desirable changes as early as may be expedient may therefore require the incitement and patronage of government. âAlexander Hamilton, 1791, Report on Manufactures