Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 28 2 The High Performance Computing and Communications Initiative The High-Performance Computing and Communications Initiative (HPCCI) has been the focal point of federal support for U.S. computing and communications research and development since 1989. It became official in 1991 with Office of Science and Technology Policy (OSTP) support and enactment of the High-Performance Computing Act of 1991. It includes five programs: Advanced Software Technology and Algorithms, Basic Research and Human Resources (although there is basic research in the other four programs also), High- Performance Computing Systems, National Research and Education Network (NREN), and since FY 1994, Information Infrastructure Technology and Applications (IITA).1 Appendix A outlines the origins and early history of the HPCCI, including an explanation of associated technology trends and indications of evolution of the initiative's emphases. This chapter discusses the HPCCI's goals and contributions to date and identifies key substantive and practical issues to be considered as the initiative evolves.2 HPCCI: GOALS AND EMPHASES The HPCCI has several broad goals (NCO, 1993): â¢ Extend U.S. leadership in high-performance computing and networking technologies; â¢ Disseminate the technologies to accelerate innovation and serve the economy, national security, education, and the environment; and â¢ Spur gains in U.S. productivity and industrial competitiveness. Because these goals relate advances in computing and communications technologies to the achievement of benefits from their use, the HPCCI has from its inception provided for the joint advancement of technologies and applications. The HPCCI has pursued several specific strategic objectives. Basic Objectives Teraflop Capability The specific objective for computer development was to develop teraflop capability by the mid-1990s.3 This objective was comparable to two that had been achieved earlier by forerunners of today's computer science community at the request of the federal government: peak available
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 29 computing power was increased by several orders of magnitude during World War II, when federal interests in cryptanalysis and other wartime needs led to the development of the vacuum tube computer, and in the late 1950s, when federal interests in military command and control led to transistorized computers.4 As economist Kenneth Flamm has observed, "By tracking the origins and history of key pieces of technology, a simple but important point can be established: at certain, crucial moments in history, private commercial interests, in the absence of government support, would not have become as intensely involved in certain long-term basic research and radical new concepts" (Flamm, 1988, p. 3). The teraflop objective has inspired parallel multi-microprocessor computers as the means for providing the next major jump in computer power.5 The teraflop objective has generated both attention and misunderstanding. Progress required building a number of machines large and fast enough to reward software researchers and application users with major gains in performance, thereby motivating them to develop the code that could make the high-performance machines useful. (See Appendix A for more information on the development of high-performance hardware and software and their interaction.) The costliness of this undertaking, compounded by the highly publicized financial difficulties of two entrepreneurial ventures, Thinking Machines Corporation (TMC) and Kendall Square Research (KSR), aimed at commercializing massively parallel computing systems, attracted criticism of the HPCCI. However, that criticism appears largely misdirected. First, entrepreneurial ventures are always risky, and the two in question suffered from managerial weakness at least as much as questionable technology choices.6 Contemporaneously, more established firms (e.g., Cray Research, IBM, Intel Supercomputing, Convex Computer, and Silicon Graphics Inc.; Parker-Smith, 1994a) have persevered, and others (e.g., Hitachi and NEC in Japan; Kahaner, 1994b, and Parker-Smith, 1994b) have entered or expanded their presence in the parallel systems market. Second, focusing attention on the high initial costs for stimulating development and use of parallel processing systems detracts from the achievement of successful proofs of concept and dissemination of new approaches to computation. Although the teraflop objective was ambitious for the time scale set, it was intended as a driver and thus is best viewed as indicating a direction, not a destination; the need for progress in computing will continue beyond the teraflop capability.7 In that respect, its appropriateness was affirmed by the 1993 Branscomb panel.8 The teraflop objective has, in fact, served to focus attention on the task of combining and harnessing vast amounts of computer power from many smaller computers. The technology is now sufficiently developed that a teraflop machine could be realized today, although exactly when to do so should be left to the economics of users and their applications. 9 High-speed Networks Another direction-setting objective of the HPCCI was the achievement of data communications networks attaining speeds of at least 1 gigabit per second. Although by the mid 1980s major telecommunications networks already had gigabit-plus trunk circuits in their backbones, the HPCCI was intended to lead to much broader deployment of and access to gigabit-speed networks connecting general-purpose computers.10 This objective drove progress in switching, computing hardware and software, interfaces, and communication protocols.11 (See Appendix B.) Grand Challenges A third original objective related to applications of high-performance computing and communications technologies: to define and attack Grand Challenge problems. High-performance
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 30 computing and communications national centers in the various agencies already were providing access to tens of thousands of researchers in hundreds of institutions when the HPCCI began. Drawing from this national base of users, various HPCCI agencies have defined a series of Grand Challenge problems (see Appendix D for list) and chosen teams to attack them. The Grand Challenge teams are typically both interdisciplinary and multi- institutional. The scientific problems are picked for their intrinsic scientific merit, the need for high-performance computation and communications, the opportunities for synergistic interaction of computer scientists with computational scientists, and the scientific and societal benefits to be gained from their solution. For example, better weather prediction involves solving massive sets of equations, experimenting with models, and comparing the results obtained with them to increasingly large volumes of data collected by weather-monitoring instruments.12 High-performance computing provides the faster model computation essential to timely assessment of a sufficiently large volume of alternative weather patterns for a given period (e.g., a month).13 The results include not only greater scientific understanding, but also the benefits to businesses, individuals, and governments that come from faster, more accurate, and more detailed forecasts. Expanded Objectives The set of HPCCI objectives has been expanded through legislative and agency activities. The High- Performance Computing Act of 1991, Public Law 102-194, broadened the applications concerns to include the so-called National Challengesâexplorations of high-performance computing and communications technology for applications in such areas as education, libraries, manufacturing, and health care. PL 102-194 also reinforced the communications aspects of the HPCCI, elaborating the concept and objectives for the NREN program and emphasizing networking applications in education. Officials involved with the HPCCI have noted that although PL 102-194 was never complemented by specific appropriations legislation, its principles have driven HPCCI activities in relevant agencies, including early explorations relating to National Challenges and the formation of the Information Infrastructure Technology and Applications (IITA) component in FY 1994.14 The National Challenges, IITA, and the network aspects of PL 102-194 also included attention to short-term and practical concerns (e.g., expanding access to technology facilities and capabilities), complementing the long-term, basic research problems that remain at the heart of HPCCI.15 The tension between long-term and short-term, between basic research and applications, is fundamental to the differences in opinion voiced about the HPCCI and its merits, accomplishments, and desired directions. Based on its direct observations of work funded under the HPCCI and on its discussions with others in universities, industry, and the government, the committee affirms the value of the basic research associated with the HPCCI, research that is informed by needs associated with important applications of national interest. A fundamental issue shaping the evolution of the HPCCI is the balance to be struck between the support of applications that use high-performance computing and communications technologies and the support for computer science research on new high-performance computing and communications technologies.16 The committee's analysis of the FY 1995 HPCCI budget request (Appendix C) shows that out of the total request of $1.15 billion, $352 million (30 percent) would be invested in basic research in computer, software, or communication technologies; $205 million (18 percent) in applied computer science research in common applications support, artificial intelligence, and human-machine interface; $176 million (15 percent) in direct support of applications and computational science; and $297 million (26 percent) in supercomputing and communications infrastructure. It is hard to interpret these statistics, however, without an understanding of the nature and the value of the work labeled "applications." The HPCCI has been aimed at catalyzing a paradigm shift, which involves the synergistic interaction of people developing
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 31 the technology and people using the technology.17 The HPCCI includes mission-related activities that may drive computing and communications research and development (R&D) and/or applications that call for significant technology development. Within the computer science and engineering field, there has been considerable debate over the degree to which computing research should be driven by applications concerns as opposed to intrinsic computer science concerns, given that both approaches to research have historically yielded considerable spinoffs to other sciences and the economy.18 To computer scientists and engineers, HPCCI is viewed as the first major federal initiative that emphasizes the science of computing and communications, which is addressed in conjunction with exploration of problems involving other fields of science and engineering, loosely aggregated as computational science. To computational scientists, the emphasis is predictably on the problems in their domains and on the difficulty of developing appropriate domain-specific computational techniques. These differences in outlook result in differences in what each community calls an "application," as well as differences in requirements for R&D. HPCCI ACCOMPLISHMENTS Accomplishments under the HPCCI to date reveal two key trends: better computing and computational infrastructure and increasing researcher-developer-user synergy. In the committee's expert judgment, HPCCI has been generally successful. That assessment is necessarily qualitative and experiential, because it is too early yet to observe the full impact of the initiative. The Issue of Measurement Early measurement of the impact of HPCCI research is problematic. As Chapter 1 points out, the time for progress from idea to product involves a decade or more, well beyond a single fiscal year. Independent of impact, individual projects may take a few years simply to reach completion.19 Consequently, the accomplishments of the HPCCI are only just becoming apparent. Moreover, it is difficult to evaluate early on how good individual ideas are and what their worth may prove to be. Many researchers have expressed concern that the push for immediately measurable results has led to an unrealistic emphasis on short-term gains and has diverted efforts from conducting productive research to maintaining "paper trails."20 However, the pressures on agencies to maximize the return on limited research funds seems to discourage funding of more innovativeâand therefore riskierâexploration that may not necessarily succeed (Rensberger, 1994). The problem of measurement is compounded by the fact that a considerable amount of HPCCI research addresses enabling technologies whose benefits or outcomes may be evident only indirectly. How best to assess results is unclearâkey questions include the kinds of reviews already undertaken by agencies and with what effect; how evaluations based on outside expertise should be combined with in-house agency know-how; whether to focus on reviewing progress for a program as a whole or progress in individual grants; the costs in time and money of different approaches and comparison of the benefits in terms of review quality, scope, and timeliness to the costs; and so on. The committee recognizes that data and analysis are needed to support decision making about any new approaches to evaluation; it did not have the time or resources to pursue such analysis.21 Moreover, complementing the committee's observation that much of the evidence on outcomes is anecdotal is a recent National Research Council study pointing out that good, relevant data (on scientific research in general) are hard to find and even harder to draw inferences from (CPSMA, 1994).
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 32 Better Computing and Computational Infrastructure The HPCCI has contributed substantially to the development, deployment, and understanding of computing and communications facilities and capabilities as infrastructure. It has helped transform understanding of how to share resources and information, generating proofs of concept and understanding that are of value not only to the scientific research community but also to the economy and society at large. The HPCCI has directly strengthened academic computing and computational infrastructure, building on the National Science Foundation's (NSF's) significant investments in university computing infrastructure over more than a decade.22 The NSF infrastructure program has stimulated complementary investments by other federal agencies, industry, and universities themselvesâan impact that, like other HPCCI contributions to stimulating a growing foundation of activity, is difficult to assess directly. This academic base, in particular, academic research in experimental computer science and engineering,23 is fundamental to the development and application of high- performance and other computing and communications technologies (CSTB, 1994a). By providing access (often over the Internet) to state-of-the-art computer resources and to expertise to help researchers learn how to use them, the HPCCI has also enabled research in a wide range of science and engineering disciplines to be performed that would not otherwise have been possible. Appendix D lists relevant examples from the Grand Challenge activities, and Appendix E points out instances related to the NSF supercomputer center activities, which fall under the HPCCI umbrella despite having some separate roots. Within the NREN program, NSFNET and other components such as ESNet and the NASA Science Internet have helped to extend networking across the science research community (CSTB, 1994d). Through the internetworking provided by the Internet, connectivity and experimentation with network-based infrastructure have begun to spread rapidly beyond the research community into primary and secondary education, government, industry, and other elements of the private sector (CSTB, 1994d). The Internet has demonstrated the value of widespread access to a common, sophisticated, and increasingly integrated technology base, and it illustrates how a small investment by the federal government can be highly leveraged by additional investments from industry.24 The HPCCI approach to developing high-performance computing and communications infrastructure has been affirmed in similar steps taken recently by the Japanese government and industry. David Kahaner of the Office of Naval Research has chronicled Fujitsu's progress in developing parallel processing technology, noting its establishment of research facilities providing worldwide access to its systems in order to obtain the large user base needed to refine its hardware designs and, in particular, to develop the software and applications required to make systems successful (Kahaner, 1994a). Kahaner has also reported on Japanese plans and progress for upgrading high-performance capabilities in public institutions, noting, among other things, the Japanese government's increasing emphasis on basic research.25 Increasing Researcher-Developer-User Synergy The HPCCI has fostered productive interactions among the researchers and developers involved in creating high-performance computing and communications technology and those who use this technology in their own work, most notably computational scientists, but also a broad spectrum of other users. Building on the varying needs and perspectives of the three groups, complex problems are being solved in unique ways. In particular, the HPCCI has funded cross-disciplinary teams associated with the Grand Challenge projects to solve complex computational problems and produce new software for the new
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 33 parallel systems. These teams interact with hardware and operating system/compiler researchers and developers to address complex problems through use of the largest computer systems, including those housed at the NSF supercomputer centers. Their work has provided vendors with key insights into the limitations of their architectures.26 Although these users' requirements are more specialized than those typical of the commercial market for parallel systems, such collaborative work has contributed to enhancing the development and application of high-performance computing and communications technologies. For example, astrophysicists' work on problems in cosmology has stimulated improved handling of fast Fourier transforms in high-performance system compilers that has also benefited commercial applications of seismology in oil exploration.27 Like collaboration in other areas, that between computer and computational scientists has not always come easily. In particular, there has been some controversy concerning the relative emphasis on advancing disciplinary knowledge, on the one hand, and advancing the state of the art in high-performance computing and communications, on the other. Nevertheless, the HPCCI has provided a structure and a set of incentives to foster collaborations that many computational scientists believe would not be supported under programs aimed at nurturing individual disciplines. 28 Impact of Broad Collaboration Many notable HPCCI accomplishments are the result of broad collaborations. In many instances, they build on foundations that predated the HPCCI, although HPCCI funding, facilities, and focus may have provided the push needed for their realization. The Mosaic browser (Box 2.1) epitomizes both the cumulative nature and broad impact of the development of technologies associated with the HPCCI. â¢ The HPCCI has driven progress on Grand Challenge problems in disciplines such as cosmology, molecular biology, chemistry, and materials science. Parallel computation has enhanced the ability to do rapid simulations in science and engineering (Kaufmann and Smarr, 1993). Recognition of this development continues to spread across the research community. â¢ The HPCCI has furthered the development of new modes of analyzing and/or visualizing complex data and in many cases has contributed to more effective interworking between supercomputers and desktop graphics workstations. Visualizations of the numerical output of the largest computers require specialized graphics computers, whose speed would have made them supercomputers in their own right a few years ago. Examples include visualization of complex motions of large biomolecules, intricate engineering assemblies, and the large-scale structure in the universe. â¢ The HPCCI has made parallel computing widely accepted as the practical route to achieving high- performance computing, as can be seen in the recent growth in sales of parallel systems.29 Although the market for larger-scale parallel-processing systems is inherently small, it is nevertheless growing. Box 2.2 gives a few of many possible examples of the applications being developed.
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 34 BOX 2.1 MOSAIC AND THE WORLD WIDE WEB The development in 1993 of the National Center for Supercomputing Applications (NCSA) Mosaic browser shows how the HPCCI has been able to create successful new applications enabled both by new capabilities and by prior developments in information technology. The forerunner of the Internet (ARPANET) was developed in the late 1 960s to link computers and scientists performing defense-related research throughout the United States. By the time of the HPCCI's formal initiation in FY 1992, the Internet had become the most popular network linking researchers and educators at the post-secondary level throughout the world. The development of gopher at the University of Minnesota in the early 1 990s was a key step in establishing the Internet as an information resource that could be used through a consistent user interface. At about the same time, researchers at the European Laboratory for Particle Physics, CERN, had developed and implemented the World Wide Web (WWW), a network-based hypertext system that allowed the linking of one piece of information to another across the network. Users accessed WWW information through ''browsers" that allowed them to activate a hypertext link in a document to retrieve and display related information regardless of its physical location. Early browsers were text-based, presenting the user with a menu of numbered choices, whereas slightly later browsers made use of the "point-and-click" capabilities of the mouse within a graphical user interface. The WWW and its browsers sought to present users with a consistent interface with which to access all existing methods of Internet navigation and information retrieval. Meanwhile, the HPCCI had provided funding for research into advanced networking technologies and for the deployment of a high-capacity backbone, enabling the rapid transfer of large amounts of data across the network. In 1993, software developers at the NCSA, one of the centers supported by HPCCI funds from the National Science Foundation, developed an easy-to-use graphical browser for the WWW known as NCSA Mosaic, or sometimes simply Mosaic. It allowed the inclusion of images in WWW documents and even allowed the images themselves to be links to other information. Continuing development of Mosaic enabled the inclusion of audio and video "clips" within hypermedia documents. By November 1993, Mosaic browsers were available for the three most popular computer operating environments: Apple Macintosh, Microsoft Windows, and X Window on UNIX platforms. One year later, users have downloaded more than 1 million copies of Mosaic software, NCSA's scalable WWW server (the world's busiest) is handling over 4 million connections per week, and Mosaic is credited by many for the current and dramatic surge in use of and interest in the Internet. Perhaps even more significantly, Mosaic has served as the genesis of a wide range of commercial developments. The University of Illinois, which owns the intellectual property associated with Mosaic, has named Spyglass, Inc. as the master sublicenser for the software. So far over 20 companies have licensed Mosaic, creating over 12 million commercially licensed copies. In addition, other companies such as Netscape, IBM, Pipeline, Booklink, MCC, and NetManage have created alternative WWW browsers. Many other entities have become information providers (a 100-fold increase in WWW servers has occurred in the last 2 years), and new security additions to the underlying Mosaic/WWW infrastructure have enabled electronic commerce on the Internet. Spectacular growth in commercial use of this new information infrastructure is expected in 1995 and beyond because of the relative ease with which the Mosaic/WWW combination allows for highly accessible information servers to be established on the global network. Because of the decentralized nature of the Internet, it will be difficult to gauge the total business income generated by the introduction of Mosaic; however, there is already enough commercial activity to believe that there will be significant payback on the critical federal HPCCI investment in the NSF supercomputer centers that led to this unexpected software development. Even more important is the paradigm shift in the use of the Internet that Mosaic and the WWW have generated.
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 35 BOX 2.2 SOLUTIONS TO COMPLEX PROBLEMS Parallel computing has enabled creative solutions to a number of industrial and scientific problems. The following examples are but a few of many possible illustrations of such successes. â¢ Defense. Simulation of the interaction between the electromagnetic spectrum and various aircraft design features has enhanced the performance of stealth aircraft. Parallel computing enabled rapid calculations for many different wave lengths and aircraft parts. â¢ Petroleum. Oil exploration and production have been made more productive by three-dimensional analysis and visualization of huge amounts of seismic data. Parallel computing enabled the move from two- to three-dimensional processes. â¢ Finance. Forecasting and simulation of various trading strategies for mortgage-backed security instruments has created a new market and contributed to a reduction in rate spreads. Parallel computing enables simultaneous calculations for numerous instruments within the very short time frames of a fast-moving market. Growth of the finance market arena will provide market pull that should help lower the costs of high-performance processing systems for all types of users. â¢ The HPCCI has provided large numbers of academic scientists with peer-reviewed access to their choice of high-performance computing architecture to enable their computational science projects. The NSF supercomputing centers, whose core budgets are entirely within the HPCCI, provided access to 23 high- performance computing machines in FY 1995 to 7,500 academic researchers in over 200 universities. The capacity of the centers' supercomputers was 75 times as great as in FY 1986, their first full year of operation, and users represented a broad range of scientific and engineering disciplines (Appendix E lists representative projects). â¢ The Internet, the flagship of the NREN component of the HPCCI, has become the basis for computer- mediated communication and collaboration among researchers and others. The geographical dispersion of Grand Challenge team members has resulted in pioneering use of electronic collaboration methods, beginning with conventional electronic mail and expanding to multimedia electronic mail, audio and video conferencing, and shared tools for accessing and using remotely stored data and for controlling remote instruments. Development of these methods and tools has been fostered and funded by the HPCCI, demonstrating the potential for electronic collaborators and other approaches to using information technology to support distributed work (CSTB, 1993, 1994e). â¢ The Internet plus a collection of advances and applications in data storage, analysis, retrieval, and representationâsome involving high-performance technologyâhas catalyzed exploration of digital library concepts. Early Internet-based collaborations among computer scientists, information scientists, cognitive and social scientists, and domain-specific groups provided the basis for a multidisciplinary research effort in digital libraries under the IITA program that was launched in mid-1993 (NSF, 1993). â¢ The gigabit network testbeds, an element of the NREN component of the HPCCI, pioneered in advancing the frontiers of communication bandwidth essential to achieving enhancements envisioned for the nation's information infrastructure. In the process, they helped to bridge the gap in perspective and emphasis between the computing and communications research communities.
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 36 Transfer of Expertise and Technology In addition to enabling explicit collaborations, the HPCCI has indirectly affected industry and other sectors outside of academia by stimulating the spread of human experts and thus the transfer of technology, building on a tradition of interaction typical of the computing field. As Kenneth Flamm observed, "People are clearly the medium in which computer technology has been stored and transmitted. The history of the computer industry is the history of trained individuals acquiring knowledgeâformal and informalâ[and] then applying that knowledge, often in a new organizational or institutional setting" (Flamm, 1988, p. 3).30 The importance of educated and trained talent is reflected in the HPCCI's Basic Research and Human Resources component, which produces the intellectual and human capital, the most general benefit and therefore perhaps the least easy to identify. Impact on Mission Agencies In addition to its broad national accomplishments, the HPCCI's contributions to the federal mission agencies, the initial customers for high-performance computing and communications technology, must also be considered. Based on its discussions with agency officials and its own insights into the fit between these technologies and agency activities, the committee believes that the existing and potential contribution of high-performance computing and communications technology to federal mission agencies does justify the investment. The policy decision to eliminate nuclear weapons testing has greatly increased the need for high-performance computer simulations at the Departments of Energy and Defense, for example, and the need to control costs for defense materiel makes simulation in the manufacture of defense-related products an attractive prospect.31 Thus much of the HPCCI effort at the Advanced Research Projects Agency (ARPA) relates to design and simulation. Although defense-specific applications are sometimes unique, applications-related investment can have impacts beyond meeting agency needs. As in the case of the gigabit testbeds, for example, such work can provide proofs of concept that encourage private investment by lowering risks. Five Gigabit Testbed Projects: Collaboration and Impact The gigabit testbeds provide a case study of how to achieve progress through cross-sectoral, developer-user collaboration to advance high-performance computing and communications technologies. Since 1989, the testbeds have provided the means to test applications and thus extend the state of the art in gigabit networking to link high-performance computers to each other and to applications sites. The five original gigabit testbed projects, funded by NSF and ARPA and administered by the Corporation for National Research Initiatives, were started as a 5-year program.32 They received considerable support from the telecommunications industry, mainly in the form of donated transmission facilities. All three major long-distance carriers and all of the major local-exchange carriers participated. In addition, at least three similar independent testbeds were created through public-private partnerships in the United States, and imitators sprang up in Europe. The testbeds were intended to address two key questions: (1) How does one build a gigabit network?, and (2) What is the utility of such a network? All five became operational by 1993, showing that gigabit networks could be built. They were largely but not completely successful in illuminating how best to build a gigabit network.33 The difficulties in many cases were not with the networks but rather the computers connected by the networks, which could not handle the very high bandwidths. Although no research on computer
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 37 architecture was included in the testbed projects, the projects demonstrated the need for better computer systems, both hardware and software, to achieve better communicationsâthey demonstrated the intimate linkage between computing and communications systems. As to the utility of the gigabit testbeds, opinions in the community differ sharply. Demonstrations of the potential utility of gigabit networks in some Grand Challenge applications were achieved, including global climate modeling and radiation treatment planning. What is debated was whether the very high bandwidths actually added much value to the applications. No large-scale use of the testbeds for applications research was really possible because of the rather experimental nature of the networking and limited reach of the networks. Also, this work demonstrated that there were essentially no computers that could take advantage of the high-bandwidth gigabit lines because their internal buses were too slow. Although the gigabit testbeds emphasized speed, discussions within the research community, industry, and the HPCCI agencies have suggested that future testbeds should address architecture, scale, and heterogeneity of systems as well as communications speed. EVOLUTION OF HPCCI GOALS AND OBJECTIVES Since early 1994, the policy context for the HPCCI has shifted at least twice, and the change in the Congress heralded by the fall 1994 elections suggests the potential for further change. Improving the Information Infrastructure The first shift in policy affecting the HPCCI reflected growing interest in the information infrastructure and thus in the universal, integrated, and automated application of computing and communications technologies to meet a wide variety of user needs. The increasing linkage between the HPCCI and information infrastructure can be seen in the "Blue Books," the principal public documentation of the purpose, scope, participation, budget, achievements, and prospects of the HPCCI.34 Box A.2 in Appendix A outlines the evolution of HPCCI goals as articulated in the Blue Books. Box A.3 indicates the broadening of focus from science to other kinds of applications and drivers of high-performance computing and communications technologies. As discussed above, PL 102-194 marked the first explicit congressional step toward greater attention to information infrastructure; the Blue Books track concurrent thinking of HPCCI agency officials. The President's FY 1995 budget request bundled the HPCCI together with other items, notably nonresearch programs intended to broaden access to communications and information infrastructure, into the National Information Infrastructure (NII) initiative.35 The initiative built on the level of technology generally available in the early 1990s, proofs of concept provided by the NREN program, and industry trends, including growing use of computer-based networking within and between a variety of organizations and the rise of a variety of network- based businesses. Once the policy focus-in the government, the press, and most of the agenciesâcentered on information infrastructure, high-performance computing seemed to be greatly downplayed. In many 1993 and 1994 government documents and administration speeches, the first "C" of HPCCI effectively disappeared, notwithstanding the fact that achieving many of the goals for improving the information infrastructure would depend on rapid continuation of progress in high-performance computing. The NII, previously absent, was featured in subtitles of the 1994 and 1995 Blue Books even though there is no formal, specific NII research program. The formulation of the NII initiative raised questions about the nature and extent of political support for the original HPCCI objectives, and it may have led to expectations that were not embodied in the HPCCI as originally proposed. It perhaps inadvertently underscored the
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 38 misperception that because the HPCCI emphasizes the high end of the technology spectrum, it is less relevant or useful than the NII initiative. Cast in more populist terms, the NII initiative included a variety of efforts to explore broadening access to increasingly sophisticated computing and communications services and attention to associated practical concerns. These perceptionsâand misperceptionsâthreaten to slow the momentum of the HPCCI at just the time when its potential to support improvement of the information infrastructure is most needed. The missing link appears to be the failure of some HPCCI critics to appreciate the dynamism of computing and communications technology: almost by definition, relatively few really need and/or can afford leading-edge computing and communications. But as demonstrated in Chapter 1, the rapid pace of technical development quickly brings these technologies into the mainstream, and they become accessible to a broad populace. Attention to performance is justified by the expectation for rapid transitions from leading-edge technologies to cost- effective, ubiquitous technologies-as well as the kinds of applications expected to grow. For example, multimedia communications will require high-bandwidth, low-delay delivery based on high peak network capacity and on protocol support for negotiating and enforcing service guarantees. The Internet and efforts associated with the development of digital libraries already illustrate the importance of high-performance computing and communications to a broad set of information infrastructure capabilities. Greater attention to information infrastructure does not imply that performance should be abandoned. But rather than drive toward a narrow goal, such as a teraflop machine or gigabit network, per se, the goal should be systems that scale over several orders of magnitude. This goal should include not only processing rates and communication rates, but also storage capacity, data transfer rates, and retrieval times, as well as the problems inherent in serving millions of users. FIGURE 2.1 Scale and speed-important dimensions of the information technology "tent." One can view information technology as a tent: the height of the center pole corresponds to speed and the breadth of the base corresponds to scale (Figure 2.1). Both speed and scale are important research issues. The HPCCI's focus has been mainly, though not exclusively, on speed. We can move toward an enhanced national information infrastructure by adding more cloth to the tent so as to further emphasize scale without deemphasizing speed, or by shifting the focus somewhat from height to breadth, from the research issues of speed to those of scale. Both changes are appropriate; both dimensions are important for the tent to work. Additional opportunities and needs are also suggested by the tent metaphor, recognizing that there is more to advancing information technology and the information infrastructure than speed and scale. Other important goals include: â¢ Reliability (Will the tent stay up?); â¢ Software productivity (How long to move the tent to a new site?); â¢ Malleability (Can the tent's shape be changed?); â¢ Human-computer interface (Can people use the tent?); and â¢ Intelligent search and retrieval (Can people find what they want in the tent?).
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 39 Advancing the information infrastructure presents many practical and some urgent needs, but it has been and can continue to be a driver for the long-term research challenges addressed by the HPCCI. High-performance computing and communications will help provide the technologies needed to provide flexible, high-rate, affordable data communications. In Office of Management and Budget guidance for developing FY 1996 R&D budgets, the HPCCI is acknowledged as "helping develop the technological foundation upon which the NII will be built," as a prelude to the articulation of several priorities under the broad goal of "harnessing information technology," one of six broad goals.36 See Box 2.3 for an illustrative discussion of how telemedicine needs, for example, can help to drive high-performance computing and communications technology development and deployment, and how the HPCCI can foster paradigm shifts in application domains. Evolving Research Directions and Relevance for the Information Infrastructure The public debate over information infrastructure is at heart a debate over how to make computing and communications systems easier to use, more effective, and more productive. The challenge for research policy is to translate usability needs into research topics and programs. The HPCCI itself was built on the recognition that the fundamental challenge to greater acceptance and use of high-performance technologies is to make them more usable. Since the 1970s it has been recognized that more usable parallel processing machines imply the development of algorithms, programming support software, and native parallel applications, but the problem persists despite considerable progress. (See Appendix A.) For information infrastructure in the fullest senseâ reaching to ordinary citizensâthese efforts must be extended to address intuitive models of use and supporting user interface technologies to enable a class of information appliances that will become a part of everyday life. The acceptance and popularity of Mosaic demonstrate the importance of user models, human factors, and other areas where research is critically needed. More generally, intelligent information retrieval systems, systems for understanding speech and pictures, and systems for enabling intelligent dialogues between people and computer systems are capabilities that will build on HPCCI research and enhance the usefulness and level of use of information infrastructure. In addition, research and development of core software technologies are needed to achieve progress in security, privacy, network measurement and management, transaction processing, application integration, and other capabilities that may be less directly visible to individuals but that make computing and communications facilities more usable. For example, HPCCI and other computing and communications research can enhance capabilities for distributed, remote measurements of quantities that relate to the environment, traffic flows and management, or health conditions. Yet other research should build on the movement to digital transmission of more and more information. As this list of possibilities suggests, information infrastructure is bigger than an initiative, although one or more initiatives, including the HPCCI, can help to organize and accelerate progress in developing and using it. Complicating decision making regarding information infrastructure research is the recognition that an advanced information infrastructure is not something that will spring full-grown from any one development. Rather, it is something that will grow from new capabilities in many different sectors of the economy and society. Having to provide for migration, evolution, integration, and interoperability compounds the technical challenges.
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 40 BOX 2.3 TELEMEDICINE: AN EXAMPLE OF HPCCI-ENABLED TELE-EXPERTISE The provision of expert professional services, such as medicine, law, and education, is a current consumer of HPCC technologies as well as a driver of future developments. Generically, this provision of services is often referred to as tele- expertise and can be thought of as the live, interactive provision of services and education between individuals who are geographically separated but electronically connected. Tele-expertise holds the promise of reducing costs and lessening geographical disparity in the availability of services. In particular, telemedicine will be an important part of the National Challenge in health care as evidenced by funding from the National Institutes of Health and other organizations for several projects, including a 3-year contract to use advanced network technologies to deliver health services in West Virginia. Functionally, telemedicine supplies an audio, visual, and data link designed to maximize understanding between provider and patient. In telemedicine, visual contact and scrutiny are particularly important to accurate communication: studies have suggested that body language and facial expression can convey up to 80 percent of meaning. Clinically, although touch is currently denied, video zoom capability often augments visual examinations beyond what is the norm in face-to-face services. In addition, various endoscopes and physiometers may be utilized across a network to further enhance a health care worker's observations. Limited telemedicine field trials began in 1958 and expanded with federally funded research demonstrations between 1971 and 1980. Considerable research was done on reliability, diagnostic replicability, user satisfaction, and multiple- specialty services. Currently, a few projects address tele-expertise more broadly by combining telemedicine and distance learning, and trials are being conducted in Montana and Texas that encourage the integrated use of remote services in medicine, industry, law, and education, the "MILES" concept. More specifically, telemedicine has made some advances in the years since the early trials: â¢ Elaboration and extension of transmission media from early microwave and satellite channels to 384-Kbps service and direct fiber-optic links; â¢ Reduction of costs due to digital signal compression and decreased long-distance rates in constant dollars; and â¢ Expansion of the number of pieces of medical equipment that may be connected to the remote terminal, chiefly a variety of endoscopes and physiometers. Nonetheless because of health care cost issues and large disparities in the medical services available in different geographical areas, telemedicine has great potential impact as a National Challenge application for HPCCI technologies. Telemedicine urgently needs several HPCCI-related technologies that can be deployed rapidly and inexpensively and that scale well. Among others, these include: â¢ Rapid, high-capacity, multipoint switching. The telephone became increasingly useful as improved switching and networks enabled rapid expansion across the nation. So it is with interactive video-improved switching and networks will activate the distance-spanning benefits of the interactive video market. â¢ Rapid, high-capacity, multipoint switching. The telephone became increasingly useful as improved switching and networks enabled rapid expansion across the nation. So it is with interactive video-improved switching and networks will activate the distance-spanning benefits of the interactive video market. â¢ Translators to interconnect divergent computing and communications technologies. New technologies are being developed and deployed so rapidly and in so many different places on the globe that it may be more feasible to develop facile, high-performance translators than to struggle for standards. â¢ Compact video storage and good retrieval techniques. Transparent technologies must be developed to enable a physician to efficiently store and easily retrieve salient clinical moments without distracting from the clinical challenge at hand. SOURCE: Committee on Information and Communication (1994), p. 28.
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 41 Although the U.S. telecommunications industry is a world leader in developing and deploying networks on a large-scale, the concepts inherent in an advanced national information infrastructure go beyond connecting a large number of relatively homogeneous end systems supporting a relatively small number of applications, such as today's telephones or televisions. Learning how to build large-scale systems, like learning how to build high- performance ones, requires research; it is not simply a matter of deploying lots of boxes and wires. Envisioned for an advanced national information infrastructure is the interconnection of a much larger number and variety of networks than are interconnected today, with more nodes and switches per network and new mixes of wireline and wireless transmission. The end systems of such networks will run a much wider set of applications and call for a broader set of software-based support capabilities often referred to as "middleware." There will be great complexity, increasing the emphasis on scale and architecture and on areas such as accommodating heterogeneity of systems, decentralization of control and management, routing, security, information navigation and filtering, and so on, all of which will depend on software. The evolutionary nature of information infrastructure also underscores the importance of engaging industry in the planning, support, and conduct of research. Advisory committees and collaborative projects are but two examples of how this engagement can be achieved. See Appendix B, Box B.1, for a discussion of the development of asynchronous transfer mode as an illustration of fruitful industry-university-government interaction. There have been many government, academic, and industry efforts, some still under way, to identify and clarify research issues associated with improving information infrastructure. The recent CSTB report Realizing the Information Future (1994d) provides a unifying conceptual framework from which it derives strategic emphases for research; a multiparty effort generated several lists of research ideas (Vernon et al., 1994); a more focused workshop generated ideas for funding under the NSF networking and communications research program (NSF, 1994); and ARPA's NETS program and other programs have continued to develop and enrich a technology base for bitways and midlevel services to support defense-relevant applications.37 Common to these various efforts is the need for research to enhance such critical information infrastructure middleware capabilities as security and reliability; the basic research underlying many of these concepts had been done by high-performance computing and communications researchers funded mainly by ARPA. In addition, it is important to advance true communications research, including such fundamental areas as transmission, switching, coding and channel access protocols realized in electronic, optical, and wireless technologies, as well as basic computer networking research in such areas as internetworking protocols, transport protocols, flow and congestion control, and so on. These complement and enable efforts relating to distributed computing, which tends to be concerned with the upper or applications level of a total system. See Box 2.4 in the section "Coordination Versus Management" below and Appendix B for an examination of HPCCI communications research efforts and Appendix C for the larger budget allocation picture. Now is the time to explore a wide variety of technical problems, enlisting as many approaches and perspectives as possible. Overall Computing and Communications R&D Planning The second major influence on the policy context for HPCCI is a broad rethinking of computing and communications R&D, building on the reorganization of the federal coordinating structures for R&D and factoring in a broad range of technology and policy objectives. The broadest coordination of computing and communications research and development activities across federal agencies is the responsibility of the Committee on Information and Communications (CIC) under the National Science and Technology Council. The CIC was formed in 1994 and is led by
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 42 the director of defense research and engineering, the associate director for technology of OSTP, and the assistant director of Computer and Information Science and Engineering, NSF. In late-1994, the CIC launched a strategic planning activity to provide input into the FY 1996 budget-setting process, expected to conclude in early 1995, and inform efforts for the next 5 years. Indications from briefings based on preliminary versions of that strategic plan show a broader and richer set of concerns than previously evident. Strategic focus areas identified in preliminary materials include global-scale information infrastructure technologies, scalable systems technologies, high-confidence systems, virtual environments, user-centered interfaces and tools, and human resources and education. The HPCCI relates at least somewhat to all of these topics, and the planning process is examining where other, mission-related agency activities can build on HPCCI as well as other activities. Key research activities are classified as components, communications, computing systems, software toolkits, intelligent systems, information management, and applications.38 Software and usability are crosscutting themes. Toward a Better Balance There is a natural evolution of the HPCCI, many aspects of which are associated with improvement of the information infrastructure. The newest component of the HPCCI, the Information Infrastructure Technology and Applications (IITA) program, is one of the most visible signs of this evolution, but also important are the trends within the programs at both ARPA and NSF, which show increasing emphasis on software solutions and tools. ARPA, for example, is devoting attention to software and tools to support design and simulation for development of defense systems; its emphases on security and scalable systems both involve substantial effort relating to software.39 This evolution should continue and indeed accelerate. Practical experience with the HPCCI and the volatile policy context both suggest that the ideal research agenda for high-performance computing and communications should be driven by strategic priorities, but focused more broadly than on just those priorities. A stable yet flexible approach would combine substantial focus on goals of current national importance, including directly targeted research, with a flexible program that sustains a healthy base of computing, computation, and communications science and technology. The comprehensiveness of the emerging CIC strategic plan appears to provide a broader platform than previously available for supporting the nation's public computing and communications R&D, including that relating to high-performance technology. Also, the commendable inclusion of a technology "maturity model" in CIC's preliminary strategic planning material illustrates recognition of the technology "trickle-down" phenomenon. MOVING FORWARD-BASIC ISSUES Balance of Private and Public Investment The possibility of reduction or even premature termination of the HPCCI, suggested by congressional requests for inquiries by the General Accounting Office (GAO) and the Congressional Budget Office (CBO) and for this committee's report, is troubling. (See Appendix A for a brief discussion of issues raised by GAO and CBO.)40 Some HPCCI critics expect industry to pick up the task. They seem to assume possible a larger program of basic research from industry than is
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 43 reasonable based on history, the growth in competition, which reduces the profit margins needed to sustain R&D, and the economics of innovation generally. Leading-edge high-performance computing and communications technology is aimed at the most demanding customers, a niche or subset on the order of 10 percent of the larger computing and communications market. Truly high-end systems tend to be nonstandard and to require considerable customer support, for example, which limits their market potential. It may be more appropriate, therefore, to assume that truly high-end systems are aimed at particular classes of problem for which the systems and associated software have particular value, rather than to assume that these systems will become universal. For example, better weather prediction would save an enormous amount of money and should be carried out on high-performance computers even if millions of people do not have them. The lower end of the market will grow as parallel processing vendors reposition their products, addressing broader industrial and commercial needs for information storage and analysis, multimedia servers, data mining, and intelligent transactions systems.41 Observers within the computing and communications research communities, including members of this committee, are concerned about the impact of computer and communications industrial restructuring. Changes in the organization of these industries, plus the inherent difficulties incumbent companies face in using research results, prevent companies from undertaking the kind of large-scale, long-range research needed to tackle the challenges inherent in advancing the HPCCI objectives or the broader objectives associated with information infrastructure. This concern is almost impossible to substantiate, because it is inherently intuitive, albeit shaped by expert judgment and the experience of committee members working in or with a variety of computing and communications companies, and because the results of current trends will not be evident for several years.42 Coordination Versus Management The HPCCI became an integrated, multiagency, cross-cutting initiative because agency and congressional officials recognized that there would be economies of scale and scope from connecting complementary efforts across research disciplines and funding agencies.43 By cooperating, agency officials have successfully leveraged the dollars available in the initiative budget, facilitating collaborative efforts with industry. The NREN infrastructure investments, including the NSFNET backbone and gigabit testbeds, provide examples. Network connections, research tools, and delivery of educational products appear to motivate the broadest interagency activity within the HPCCI context, helping to extend collaborations beyond the conduct of research per se and into a wider circle of agencies. Through its accomplishments and esprit de corps, the HPCCI has become a model for multiagency collaboration.44 Each agency retains responsibility for its own part of the program, focusing its participation to meet agency needs and resources. The voluntary compliance of HPCCI agencies with the spirit of PL 102-194 reflects the special cooperation that has characterized the HPCCI. These conditions have enabled the initiative to grow and adapt relatively quickly to changing national needs, technology prospects, and the fit between the two. Perhaps because they see themselves as principal architects of the program, officials from the four initial HPCCI agencies (Department of Defense (DOD), Department of Energy (DOE), National Aeronautics and Space Administration (NASA), and National Science Foundation), in particular, have carried high levels of enthusiasm, dedication, inventiveness, and energy into undertaking the HPCCI. These intangible qualities are widely recognized within the computing and communications research community. The level of interagency coordination observed today took time to grow. As one might expect when organizations with different missions, budgets, and cultures are faced with a joint task, the HPCCI agencies have disagreed on issues of emphasis and approach over the years. For
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 44 example, DOE and NSF have had different views on evolving the NREN program with respect to scope and speed. What is important for the future of the HPCCI, however, is not that differences have arisen but rather that legitimate differences owing to varying missions have been respected, and cooperation and coordination have improved over time. For example, NSF and ARPAâwhich respectively emphasize small science and larger projectsâhave worked well in the management of their joint network and computing research activities, as described in Box 2.4. BOX 2.4 COORDINATION IN PRACTICE: THE CASE OF COMMUNICATIONS R&D The HPCCI currently includes a relatively modest but vigorous communications research program. Three large programs account for $77 million of the HPCCI communications research budget. Research is concentrated mainly in four areas (see Appendix B for more details and context): 1. Optical networks (the longest-term research), 2. Gigabit networking (medium-term research), 3. Multimedia communications (fairly near-term research), and 4. Internetwork scaling (near- and medium-term applied research). The ARPA networking program, at $43.1 million, is the largest communications research program activity. The milestones include: â¢ Demonstration of diverse Internet capabilities such as cable and wireless bitways, â¢ Demonstration of rate-adaptive quality of service negotiation in asymmetric networks, â¢ Demonstration of bandwidth and service reservation guarantees for networks in support of real-time services, â¢ Demonstration of secure routing systems, and â¢ Interconnection of gigabit testbeds. The ARPA Global Grid program, at $23 million, intends to accomplish (in 1995): â¢ Demonstration of multi-wavelength reconfigurable optical network architecture, and â¢ Demonstration of integrated DOD and commercial networks in support of crisis management applications. NSF's Very High Speed Networks and Optical Systems program, at $11 million, supports research in a wide variety of high-performance networking technologies, including: â¢ Gigabit testbed research (switching, protocols, and management); â¢ Resource discovery; â¢ Information theory; â¢ Network security; â¢ Modulation, detection, and coding for information storage; and â¢ Optical networking. ARPA and NSF have coordinated well to avoid duplication of efforts. ARPA funds most of the research on internetworking, and NSF concentrates on the deployment of internetworking infrastructure via its NSFNET activities. NSF and ARPA have jointly funded the gigabit testbed research program, which involves demonstration of cross-country gigabit networking technologies. With the reinforcement provided by PL 102-194, the set of agencies involved in the HPCCI grew. This broader participation better positions the HPCCI to support development and application of computing and communications technologies essential to improving the nation's information infrastructure. However, as reflected in both executive and congressional efforts to promote such
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 45 improvements, the information infrastructure raises issues such as deployment, regulation, and other practical aspects that require engaging a broader and somewhat different set of agencies, such as the Federal Communications Commission, the National Telecommunications and Information Administration, and so on, to address a wider range of issues than those relating to R&D. The diversity of the HPCCI approach allows many views to compete, first for funding, later in the evolution of thinking among researchers, and finally in the marketplace. It also fosters pursuit of the intellectual questions posed by the HPCCI via a range of complementary modes including classical single principal-investigator (PI) research, multiple-PI experimental research, multiple-PI/multiple-field collaborations, intramural research in institutes and national laboratories, and joint industry-government-academia experiments or proofs of concept. A variety of mechanisms are used to foster interagency cooperation and coordination: â¢ Joint funding of projects, from relatively specific or narrow activities to the federal portions of the Internet; â¢ Consortia, such as the consortium on Advanced Modeling of Regional Air Quality involving six federal agencies plus several state and local governments; â¢ The MetaCenter concept pursued by NSF supercomputer centers (see Appendix E) and extending to other entities and users via the MetaCenter Regional Affiliates program; and â¢ Cross-agency reviews of grants and contracts, such as the NSF, ARPA, and NASA digital library initiative, and joint testbeds. The diversity in approach and tactics makes it less likely that the nation will miss some important approach. It also facilitates participation by a variety of agencies, which tend to have different styles as well as emphases for supporting research or procuring technology, consistent with their different missions, histories, and cultures. As to diversity of mechanisms, the multiple-PI/multiple field category is epitomized by the Grand Challenge teams, which involve multiple institutions attacking frontier research problems with multiple-year horizons, often drawing on access to the leading-edge machines in the NSF supercomputer centers and benefiting from interactions between computer scientists and computational scientists.45 The joint industry-government-academia experiment category is currently epitomized by the gigabit network testbeds. More specifically, NASA's FY 1995 HPCCI effort includes integrated multidisciplinary computational aerospace vehicle design and multidisciplinary modeling and analysis of earth and space science phenomena (Holcomb, 1994). Coordinating Structure The coordinating structure of the HPCCI has evolved steadily, largely in response to external pressures for improved visibility of decision making, requirements for accountability for expenditures, and the flow of information into and out of the initiative. Some HPCCI observers have continued to argue for a more uniform approach to related activities with thorough planning, precise milestones, and presumably no wasted effort, in a more centralized program. This is the essence of early criticism lodged by the Computer Systems Policy Project (1991 and 1993). Drawbacks of Centralization The central question about coordination is whether the special vitality of HPCCI would survive and whether centralized control would convey sufficient benefits, or merely disrupt current arrangements. A more centralized approach would have several drawbacks that could vitiate the
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 46 HPCCI: potential loss of variety in perspectives on applications now arising from agencies with different missions; greater risk of concentrating on the wrong thing; and increased bureaucratic overhead and costs associated with efforts to overlay separate programs. Moreover, because much of the existing effort involves previously existing programs, there is a risk that agencies would not participate in a program that involved a loss of their control to a more centralized authority. This concern, probably paramount to the agencies, arises from the recognition that much of the funding associated with the HPCCI is not new, just classified as relevant to the initiative. A virtue of the current arrangement is that the central coordination is provided by a relatively small entity that lacks the resources for micromanagement. That approach maximizes the benefits provided by a diverse group of agencies. National Coordination Office BOX 2.5 NATIONAL COORDINATION OFFICE: STAFFING AND STRUCTURE THROUGH 1994 Chaired by the director of the National Coordination Office (NCO), the High-Performance Computing, Communications, and Information Technology (HPCCIT) Subcommittee and its Executive Committee coordinate planning, budgeting, implementation, and program review for the overall initiative. The HPCCIT Subcommittee has also been the major vehicle for communication with other federal agencies, the U.S. Congress, and numerous representatives from the private sector. The director of the NCO reports to the director of the Office of Science and Technology Policy (OSTP). The director of OSTP has specified that overall budget oversight for the HPCCI be provided by the National Science and Technology Council through the Committee on Information and Communication (CIC). Actual appropriations for the initiative are made in the regular appropriation bills for each participating agency. The HPCCIT coordinates program and budget planning for the initiative and reports to the CIC. Under the umbrella of the HPCCIT, several working groups have been formed to help guide and coordinate activities within the five components of the initiative. For example, the Information Infrastructure Technology and Applications Task Group, established in 1993, has encouraged and coordinated participating agencies' plans for research and development aimed at providing needed technologies and capabilities for an enhanced nationwide information infrastructure and the National Challenge applications. The Science and Engineering Computing Working Group coordinates activities and software development relating to Grand Challenge applications. The direct operating expenses of the NCO are jointly borne by the participating agencies in proportion to their HPCCI budgets, and further support is provided by the interagency detailing of staff to the NCO for varying periods of time. Currently the NCO has eight permanent staff and two staff ''on loan" from the Department of Energy. In addition to providing general administrative functions such as payroll and personnel administration, the National Library of Medicine also contributes specialized assistance such as public information functions, budget preparation, legislative analysis and tracking, graphic arts services, procurement, and computing and communications support. SOURCES: See Lindberg (1994); NCO (1994); and CIC (1 994). Additional information from letter dated August 8, 1994, to Marjory Blumenthal (CSTB) from Donald A.B. Lindberg (NCO/NLM) in response to committee's interim report (CSTB, 1994c). The HPCCI coordination focus lies in the National Coordination Office for High-Performance Computing and Communications, which was established in September 1992. It operates under the aegis of the Office of Science and Technology Policy and the National Science and Technology Council (NSTC; see Figure 2.2). Box 2.5 provides information on the structure and staffing of the NCO.
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 47 FIGURE 2.2 Organizational context for HPCCI coordination. The NCO was established to aid interagency cooperation and to serve as liaison for the initiative to the Congress, other levels of government, universities, industry, and the public. It assists the mission agencies in coordinating their separate programs, offering a forum through which the separate agencies can learn of each other's needs, plans, and actions. As part of its coordinating function, the NCO gathers information about the HPCCI activities of different agencies and helps to make this information available to Congress, industry, and the public. Since its formation, the NCO has produced the impressive FY 1994 and FY 1995 Blue Books as visible manifestations of its coordination efforts. The FY 1995 Blue Book is the best documentation available of HPCCI activities and where the money is invested. Strengthening the NCO. The public debate over the HPCCI attests to the need for improved communication regarding the initiative's purpose and accomplishments. Because lack of external understanding is damagingânot least because it leads to criticisms and investigations that divert energy and resources from pursuing HPCCI goals âthe committee believes that the HPCCI
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 48 could benefit from a stronger NCO that can do a better job of telling the program's many constituencies about its goals and successes; see the committee's Interim Report (CSTB, 1994c) and Chapter 3. As authorized in the 1991 High-Performance Computing Act, the NCO was to have been assisted by an advisory committee that could provide regular infusions of ideas from industry and academia. To date, the HPCCI has been led mostly by computing visionaries and by people active in science and science applications. That is the right kind of leadership to drive the creation of enabling technology and to create computer architectures that are appropriate for the pursuit of science objectives. But the initiative now also needs the perspective on applications and on making computing and communications technologies more usable that would be provided by an advisory committee of recognized experts with membership balanced between academia and industry, and balanced with respect to application areas and the core technologies underlying the HPCCI.46 The growing dependence of more and more people on infrastructure, the rise in potential liabilities of varying kinds, and growth in competitive challenges from abroad increase industry's stakes in the quality of information technology available. Industry input into such issues as standards, security, reliability, and accounting, for example, becomes more important as advancing the information infrastructure and "high-confidence systems" come to drive more of the research agenda. In lieu of having an advisory committee, the NCO has taken the initiative to convene some industry and other groups to obtain focused input on HPCCI-related issues and directions. In conjunction with its regular meetings with federal HPCCI agency representatives, the NCO has engaged in dialogues with representatives of the computer systems, software, and telecommunications industries; managers of academic computing centers; and others; and it has held a similar discussion with representatives of the mass information storage industry. It has also participated in workshops, conferences, and public meetings sponsored by participating agencies and the subcommittee on High-Performance Computing, Communications, and Information Technology (HPCCIT). However, NCO leadership notes that given the restrictions on external interactions imposed by the Federal Advisory Committee Act, the absence of an official advisory committee prevents it from obtaining needed input on an ongoing basis, limiting it instead to one-time interactions and thus foregoing the insights that can arise where parties benefit from repeated interactions. The committee echoed the NCO's concern by recommending in its interim report that the long-awaited HPCCI advisory committee be established immediately. In view of the delays and difficulties in establishing an HPCCI advisory committee and the apparent tendency of federal science policy leaders to enfold HPCCI in a larger NII initiative, there is some expectation that one advisory committee may be empaneled to provide input into the broader CIC agenda, and to the HPCCI. This National Research Council committee thinks that solution might work, but it urges some action now. Understanding the Changing Management Context. Actions taken to reinforce the NCO must account for the larger, evolving administrative and management context in which the NCO fits. A key component of that context is the CIC. It receives limited staff support from OSTP and, presumably, from agency-based staff members. Its members include directors of computer- and communications-related research units from across the federal government. Their participation in the CIC provides information exchange and coordination, but the CIC is not an implementation entity. The NCO director participates in both CIC and HPPCIT. Planning, coordination, and management for the HPCCI have been further confounded by the rise of additional bodies to address technology policy and other policy relating to the NII initiative. The Information Infrastructure Task Force (IITF), formed in 1993, has a Technology Policy Working Group with overlapping representation with the HPCCIT. Its focus is supposed to
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 49 the original; line lengths, w print version of this publica be technology policy, versus the HPCCIT's focus on research and development. The IITF receives input from the NII Advisory Committee, also formed in 1993. And on networking issues there are yet more special coordination and advisory entities, such as the Federal Networking Council (FNC) that has associated with it the FNC Advisory Committee. This proliferation of cross-agency entities itself presents many possibilities for confusion. Moreover, by all accounts-from virtually every HPCCI official the committee has heard from and from several private-sector parties-the processes of communication and decision making have been slowed by a calendar-filling profusion of meetings. This situation raises basic practical questions of what work can get done, when, how, and by whom, when committee meetings appear to be the order of the day. Budget A detailed overview of the HPCCI budget is presented in Appendix C. The overall level has been subject to misunderstanding. According to the Blue Books, the HPCCI budget has grown from $489.4 million in FY 1992 to the $1.1 billion requested for FY 1995. When the HPCCI was proposed in the executive budget for FY 1992, the agencies involved identified from their existing FY 1991 activities a base that contributed to the goals of the program. The HPCCI's multiagency budget is more complex than it would be had the program been started "from scratch" within a single agency. Although complexity is inherent in multiagency programs and budgets, it has added to the confusion about spending priorities and accomplishments for the initiative. The agencies that had activities included in the FY 1992 base were the (Defense) ARPA, DOE, NASA, NSF, National Institute of Standards and Technology, National Oceanic and Atmospheric Administration (NOAA), Environmental Protection Agency, and National Institutes of Health/National Library of Medicine. In each subsequent year, agencies have added to this base in two ways: (1) by identifying additional existing programs that contribute to HPCCI goals and (2) by reprogramming and relabeling agency funds to support relevant aspects of the HPCCI. To this base of "identified" activities, Congress has added some funding each year for new activities or the expansion of existing efforts. The result is that the $1.1 billion requested for FY 1995 is composed of three elements: (1) funds for the continuation of agency activities that were in existence when the HPCCI started and were designated in the FY 1992 base budget, (2) funds for existing or redirected programs that have since been designated as being a part of the HPCCI, and (3) additional funds for new activities or expansion of existing efforts. It is difficult to determine exactly how large each element is and to make interagency comparisons, because each agency has used slightly different approaches for identifying existing efforts and somewhat different formats for supporting program and budgetary detail. Also, this situation has allowed some agencies (e.g., NOAA) to be considered participants in the initiative without receiving any new money.47 NOTES 1. The substance of these components is outlined in the HPCCI's annual Blue Books; see FCCSET (1991, 1993, and 1994). 2. Note that many of the concerns raised here were expressed or articulated in a CSTB report released at the dawn of the HPCCI, The National Challenge in Computer Science and Technology (CSTB, 1988). 3. A teraflop refers to 1012 (or 1 trillion) floating point operations per second (FLOPS), 1,000 times the performance of the best machines available when the HPCCI began. 4. See Flamm (1988); this book discusses the major computer development projects of the 1940s, 1950s, and 1960s and their dependence on government stimulus and combined government, university, and industry development of technology.
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 50 5. In turn, the use of multiple microprocessors in large-scale parallel machines also exposed problems that would have to be resolved for microprocessors as the dominant computing element. 6. See Zatner (1994), pp. 21-25, for a discussion of events leading to Chapter 11 status for TMC. KSR suffered from an accounting scandal, then had to contend with 12 class-action shareholder lawsuits (Snell, 1994). The impact of the lack of software has also been implicated as an indicator of management ineffectiveness in the fates of TMC and KSR (Lewis, 1994). 7. Indeed, talk of the next hurdle, the petaflop system, has already begun. NSF, NASA, DOE, and DOD hosted a 1994 workshop on enabling technologies for petaflop computing. The report is said to argue that that goal can be met at reasonable cost in 20 years using today's paradigms. See Anthes (1994), p. 121. 8. "Recommendation A-2: At the apex of the HPC pyramid is a need for a national capability at the highest level of computing power the industry can support with both efficient software and hardware. "A reasonable goal for the next 2-3 years would be the design, development, and realization of a national teraflop-class capability, subject to the effective implementation of Recommendation B-1 and the development of effective software and computational tools for such a large machine. Such a capability would provide a significant stimulus to commercial development of a prototype high-end commercial HPC system of the future." (NSF. 1993, p. 11) 9. The fundamental computer unit is the microprocessor, which today has a peak speed of around 300 megaflops. It seems premature to build a 3,000 processor teraflop machine in 1995, but as the microprocessors increase in speed to I to 2 gigaflops by the late 1990s, it seems reasonable that 512- to 1024-processor teraflop machines may be built if the economics of users and their applications require it. For example, Kenneth Kliewer, director of the Center for Computational Sciences at Oak Ridge National Laboratory, was quoted in December 1994 as saying: "The scale here is clearly a function of time, but we could have nearly a teraflop computer today by coupling the Oak Ridge and Los Alamos computers with the ones from Cornell and Maui" (Rowell, 1994). In November 1994, a new product announcement by Japan's NEC indicated that the maximum configuration, with a total of 512 processors, could be rated at a theoretical peak of 1 teraflop (Parker-Smith, 1994b). The committee notes that if trends at the NSF supercomputer centers continue, the MetaCenter (which pools some of the centers' resources) could achieve an aggregate teraflop in mid-FY 1998 and each center would reach a peak teraflop machine by the end of FY 1999. By contrast, even the aggregate performance would not reach a teraflop until after the year 2000 if acquisition of higher- performance architectures were to revert to pre-HPCCI levels. 10. The HPCCI also triggered considerable debate about what broad availability means-what capabilities, in what locations, accessible by whom and at what cost-anticipating the more recent debates about how universal service in telecommunications should evolve. 11. The gigabit goal, as defined in the NSF-ARPA-CNRI testbeds, was to achieve an end-to-end speed of at least I Gbps between two computers on a network. The telephone achievement was to multiplex about 25,000 64-Kbps voice conversations onto a transmission line operating at 1.7 Gbps (late 1980s technology, now more than doubled). The gigabit testbeds have demonstrated end-to-end speeds between two computers of about 500 Mbps, limited by the internal bus speeds of the computers, not the network. 12. Comments by Sandy MacDonald, NOAA, at "Environmental Simulation: The Teraflop and Information Superhighway Imperative Workshop," August 18-20, 1994, Monterey, Calif. He noted an increase from 3,200 numerical observations per day for Kansas in 1985 to 86,000 daily observations, many from automated instruments. 13. Comments by Steve Hammond, National Center for Atmospheric Research, at "Environmental Simulation: The Teraflop and Information Superhighway Imperative Workshop," August 18-20, 1994, Monterey, Calif. He observed that teraflop computing helped reduce processing times to 90 seconds per modeled month, yielding 1,000 modeled months in 30 hours of processing time. 14. Briefings to the committee. Legislative codification and appropriation for a broader vision for the HPCCI have been attempted but have been unsuccessful, most recently in connection with S4/H1757, in 1994. 15. For example, in addressing networking, PL 102-194 also anticipated many of the practical concerns associated with enhancements and expansion of the nation's information infrastructure, such as user charging and protection of intellectual property rights. 16. Policy documents emerging from the administration in mid-1994 and congressional actions in 1993-1994 emphasize a commitment to linking R&D spending to strategic, national concerns (Panetta, 1994; NSTC, 1994b). 17. Of course, there will also be people using the technology who are not in close contact with developers and vice versaâhence the value of a solid base of funding for both computing and communications research and for the sciences that increasingly depend on computation. 18. See CSTB (1988) for a discussion of national challenges within computing. See also CSTB (1992) for a discussion of methods of combining intrinsic problems with problems inspired by needs in other areas. 19. The lengthy time scales associated with developing complex computer-based systems are outlined in CSTB (1994a). 20. Rigorous cost-accounting and auditing can be elaborate, costly, and inflexible: "As a result, R&D done under federal contract is inherently more expensive and less effective than R&D done by an organization using its own funds" (Cohen and Noll, 1994, p. 74).
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 51 21. The National Coordination Office (NCO) has taken a first step in the evaluation area through its development of the HPCCI implementation plan. The format of the project and program records in that volume provides a basis for subsequent efforts to assess progress. The NCO management has expressed some interest in tracking progress relative to plan elements. 22. The NSF infrastructure program was motivated by the recognition in the late 1970s of major deficiencies in the academic environment for experimental computer science and engineering. The initial stimulus was the report by Feldman and Sutherland (1979). When that report was written, the discipline of computer science and engineering was perceived to be in crisis: faculty members were underpaid relative to research positions in industry and were leaving universities at an increasing rate; the number of new Ph.D.s fell far short of meeting the national demand; most departments lacked experimental computing research facilities; and there was a significant gap in research capability between the top three or four departments, which had benefited from a decade of ARPA investment, and the rest. 23. The Feldman and Sutherland (1979) report resulted in the establishment in 1981 of the Coordinated Experimental Research (CER) program at NSF. The CER program made awards of approximately $1 million per year (including an institutional match of typically 25 percent) for durations of 5 years to support significant experimental research efforts by building the necessary infrastructure. There have been very substantial increases in the number of departments producing strong experimental research, the number of departments producing strong students in experimental areas, the number of departments conducting leading-edge research in a significant number of areas, the overall rate of Ph.D. production in the field. and other similar measures. The success of the CER program was important in shaping several subsequent NSF programs that also contributed to the infrastructure of the field, such as the Engineering Research Centers (ERCs) program. A number of ERCs are in computing-related areas, which in turn influenced the Science and Technology Centers (STCs) program; three STCs are in computing-related areas. The CER program itself ultimately became the Research Infrastructure program and was complemented by an Educational Infrastructure program. A number of other agencies instituted research and/or educational infrastructure programs. 24. The NSFNET backbone has involved NSF spending authorized on the order of $30 million but complemented by in-kind and other investments by IBM and MCI through Advanced Networks and Services, which has deployed and operated NSFNET under a cooperative agreement with NSF. The Internet overall has been growing through proliferation of a variety of commercial Internet access providers. See CSTB (1994d). 25. "Although the infrastructure, including networking, software applications and tools, visualization capabilities, etc., is still not strong enough, raw computing power is becoming comparable, and in some cases greater than what is available at NSF Centers in the U.S. This increase in resources comes at a time when the Japanese government is also increasing its emphasis on basic research for its own needs and to insure that Japan is viewed as a [sic] equitable contributor to the global science community. Readers might want to reflect on the impact the NSF centers have had on U.S. science output and the potential for this to occur in Japan." (Kahaner, 1994b) 26. For example, as a result of detailed interactions between a high-performance computing and communications vendor and a staff member of an NSF supercomputer center, a Grand Challenge computer code uncovered previously undiscovered hardware bugs in newly released microprocessors installed in a scalable supercomputer at the center. This led to the vendor using a version of the Grand Challenge code inside the company as a standard test to uncover both hardware and compiler bugs. 27. Jeremiah P. Ostriker, Princeton University Observatory, personal communication, December 23, 1994. 28. Jeremiah P. Ostriker, Princeton University Observatory, personal communication, December 23, 1994. 29. Examples include the Silicon Graphics Everest/Challenge systems (some 3500 Challenge, Power Challenge, Onyx, and Power Onyx systems were sold in the 15 months following their September 1993 introduction) and the IBM SP2 and Power Parallel systems; see Parker-Smith, 1994a. See also Appendix A. 30. Even the rise and fall of individual ventures shows this generally positive pattern: TMC was launched in part by the expertise of Danny Hillis, previously at MIT, and his associates. With TMC's contraction in 1994, Hillis' team of over 20 engineers from TMC's Future Systems Group went to Sun Microsystems, where they are working on a scalable massively parallel processing system, while other TMC talent continued with a TMC parallel software descendent (Riggs, 1994). 31. Briefings to committee by Victor Reis (Department of Defense) and Howard Frank (Advanced Research Projects Agency). 32. No follow-on program to the gigabit testbed projects has yet been announced. In July 1994, an NSF and ARPA workshop proposed a research agenda for gigabit networking and called for an experimental gigabit research network facility. NSF and ARPA are extending the existing program by a few months, into early 1995. 33. Note that there is continuing popular confusion over the term "gigabit networks" and the fact that the speed most often quoted for them is 640 megabits per second. Each gigabit connection consists of two one-way circuits, each operating at 640 Mbps. Thus the overall speed of the two-way connection is 1.28 gigabits per second when properly compared to the quoted two-way capacity of application networks. Also, the 640-Mbps circuits in at least one case (Aurora) were derived by splitting 2.4-Gbps trunk circuits.
THE HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS INITIATIVE 52 34. Each year beginning in 1991 the director of the Office of Science and Technology Policy submits a report on the HPCCI to accompany the president's budget. The FY 1992, FY 1993, and FY 1994 books were produced by the now-defunct FCCSET: the FY 1995 report was produced by the NCO (acting for the CIC). The report describes prior accomplishments and the future funding and activities for the coming fiscal year. These reports have collectively become known as "Blue Books" after the color of their cover. 35. The NII initiative was framed in 1993 and included in the FY 1995 budget request. 36. Other goals, such as "a healthy, educated citizenry," also include applications of computing and communications among their priorities. 37. CSTB (1994b); Vernon et al. (1994); and NSF (1994). The ARPA NETS program is covered by the Blue Books for FY 1994 and FY 1995. 38. Briefing to committee by Edward Lazowska, based on a Computing Research Association briefing by John Toole, and augmented by briefings by Anita K. Jones and Howard Frank, December 20, 1994. 39. Briefing to committee by Howard Frank, December 20, 1994. 40. GAO (1993); CBO (1993). 41. Committee briefings by Forest Baskett, Silicon Graphics Inc., April 13, 1994; Justin Ratner, Intel Supercomputer Systems Division, June 27, 1994; Steve Nelson, Cray Research, Inc., June 28, 1994; and Steven Wallach, Convex Computer Corporation, June 28, 1994. Also. see Lewis (1994) re "gigaflops on a budget." See also Furht (1994) for a description of how Encore, Hewlett-Packard. IBM, Pyramid, Tandem, Stratus, and AT&T have changed their focus to transaction processing and fault-tolerant computing. 42. For example, in FY 1994, the NSF centers had an income derived by recovering cycle costs from noncomputer industrial partners of around $1 million to $2 million. In comparison to their NSF Cooperative Agreement level of $16 million per year, this has a small impact. Indeed, the situation is even worse, since a typical NSF supercomputer center receives only half its annual budget from the NSF Cooperation Agreement, the other half coming from state and university matching funds, other grants, and equipment donations by computer vendors. The center experience also shows that over the last few years, industry spending to attain center know-howâtraining, software development, information infrastructure application development, virtual reality and visualization projects, and so onâand to use centers as vehicles for collaborative research has increased and exceeds spending on computer processing cycles at some centers. Because the centers have an existing staff for these projects, the industrial income generally covers only the marginal cost of providing that service and therefore does not increase net "new dollars." 43. Other cross-cutting initiatives contemporaneous with HPCCI include advanced manufacturing technology; global change; advanced materials and processing; biotechnology; and science, mathematics, engineering, and technology education (FCCSET, 1993). 44. The HPCCI is understood by a variety of federal officials to have been a model for the "virtual agency" concept advanced through the National Performance Review efforts to improve the organization and effectiveness of the federal government (Gore, 1993). 45. There are 16 grants, 7 awarded in FY 1992 and 9 in FY 1993. Their source of funds can be broken into three parts (NSF/CISE, NSF/ non-CISE, and ARPA). The FY 1994 and FY 1995 numbers are shown below. As the chart indicates, CISE's percentage is less than one- third of the funding. This shows great leverage, even greater than that of the centers, roughly one-half of whose budget comes CISE. NSF/CISE NSF/non-CISE ARPA TOTAL FY 1994 $M 2.77 5.00 1.91 9.68 FY 1994 % 29 52 20 100 FY 1995 $M 2.79 4.93 1.44 9.16 FY 1995 % 30 54 16 100 46. For example, NASA feels pressure from the HPCCI objectives to orient its program in certain directions, but is encouraged by the aeronautics industry to orient its activities in other directions. NASA is caught in the middle. Aeronautics industry representation at the HPCCI leadership level could help guide the HPCCI in directions that better support the goal of enhancing U.S. industrial competitiveness. 47. Letter dated August 25, 1994 to Marjory Blumenthal from Jerry D. Mahlman (NOAA) in response to committee's interim report (CSTB, 1994c).