Information technology (IT) is on the verge of another revolution. Driven by the increasing capabilities and ever declining costs of computing and communications devices, IT is being embedded into a growing range of physical devices linked together through networks and will become ever more pervasive as the component technologies become smaller, faster, and cheaper. These changes are sometimes obvious—in pagers and Internet-enabled cell phones, for example—but often IT is buried inside larger (or smaller) systems in ways that are not easily visible to end users. These networked systems of embedded computers, referred to as EmNets throughout this report, have the potential to change radically the way people interact with their environment by linking together a range of devices and sensors that will allow information to be collected, shared, and processed in unprecedented ways. The range of applications continues to expand with continued research and development. Examples of ways in which EmNets will be applied include the following: EmNets will be implemented as a kind of digital nervous system to enable instrumentation of all sorts of spaces, ranging from in situ environmental monitoring to surveillance of battlespace conditions; EmNets will be employed in personal monitoring strategies (both defense related and civilian), combining information from sensors on and within a person with information from laboratory tests and other sources; and EmNets will dramatically affect scientific data collection capabilities, ranging from new techniques for precision agriculture and biotechnological research to detailed environmental and pollution monitoring.
The use of EmNets throughout society could well dwarf previous milestones in the information revolution. The effects of Moore’s law1 and related trends in computing and communications are making all of this possible. Ongoing work in microelectromechanical systems (MEMS) will enable sensing and actuation on the scale of a nanometer. The possibilities for miniaturization extend into all aspects of life, and the potential for embedding computing and communications technology quite literally everywhere is becoming a reality. IT will eventually become an invisible component of almost everything in everyone’s surroundings.
WHAT IS DIFFERENT ABOUT EMNETS?
EmNets are more than simply the next step in the evolution of the personal computer or the Internet. Building on developments in both areas, EmNets will also be operating under a set of constraints that will demand more than merely incremental improvements to more traditional networking and information technology. EmNets will tend to be tightly coupled to the physical world. Unlike a desktop computer, which is itself a piece of office furniture, EmNets will be integrated into furniture and other objects in the environment. Individuals will interact with the objects and devices of which EmNets are a part, but it is unlikely that they will think of it as interacting with a computer system. A complex, networked, computational system will often be invisible when things are working properly.
EmNet components will also be highly resource constrained. In contrast to the Internet, which still consists primarily of tethered devices, EmNet components are likely to be small, untethered devices operating under physical constraints such as limited energy and the need for adequate heat dissipation. EmNets will also be constrained by bandwidth and memory limitations.
In addition to the physically coupled, resource-constrained nature of these systems, another constraint on EmNets is the fact that often they will be integrated into objects or systems that are likely to last for long periods of time. EmNets in buildings, bridges, vehicles, and so on will be expected to last as long as the objects in which they are embedded. This expectation of longevity will need to be taken into account when designing, deploying, and managing these systems. A further constraint is the
likely heterogeneity and large number of interacting elements that will make up an EmNet; this makes interoperability a key concern. Finally, EmNets will often be used and interacted with by people who are not experts in EmNet-related technology. Managing all of these constraints and creating a system that functions properly for the application domain while remaining understandable and manageable by human operators, users, and—in many cases—casual passersby, is a large challenge for EmNet designers.
As an example, consider a transportation information system based on EmNet technology. Such a system will certainly be large in size and scale, possibly encompassing the entire highway system of the United States. Components of it would probably be embedded in long-lived physical structures (such as bridges, traffic lights, individual cars, and perhaps even the paint on the roads). Some components will be tethered, but many would be resource constrained while computing data and communicating it wirelessly when necessary. The many pieces of such a system will of necessity be heterogeneous, not only in form but also in function. There may be subsystems that communicate to consumers in private vehicles, others that relay information from emergency vehicles to synchronize traffic lights, still others that provide traffic data and analysis to highway engineers, and perhaps some that communicate to law enforcement. Issues of how information will be communicated to those interacting with the system are of great importance in such an environment. Safety is a critical concern; issues of privacy and security arise as well, along with concerns about reliability.
The rest of this report identifies areas in which research is needed to enable such EmNets and to make them a successful reality. Below are highlights of some of these areas as well as particular recommendations to federal funding agencies.
KEY AREAS OF INQUIRY
Realizing the great promise of EmNets requires more than the mere advance of individual technologies—it will rely on numerous subsystems working together in an efficient, unattended, comprehensible, and trustworthy manner. Many aspects of the needed research are highly interdisciplinary because of the intricate ways in which EmNet systems interact with the physical world. In the absence of programs aimed at solving some of the basic research problems, it is likely that many of the benefits of EmNets will simply not be realized.
As with any technology there are risks. In the case of EmNets, the potential benefits come with associated risks that may be exacerbated by the EmNets’ very pervasiveness. Pervasive information creates security,
safety, and privacy protection issues. As EmNets become increasingly critical to our communication, transportation, power distribution, and health-care infrastructures, the consequences of failures and security breaches will become increasingly severe. By the time EmNets are broadly deployed, it may not be feasible to give them technological fixes because their components are so widely dispersed.
This report by the Committee on Networked Systems of Embedded Computing, convened by the Computer Science and Telecommunications Board of the National Research Council, identifies and explores the many research questions that must be answered before there can be implementation and use of widespread networked embedded computing devices. It examines the enabling technologies that will facilitate the development and broad deployment of EmNets, and it explores three key areas in which a great deal of new research will be required for EmNets to achieve their full potential: (1) self-configuration and adaptive coordination, (2) building trustworthy EmNets (including issues of privacy, security, reliability, safety, and usability), and (3) models of computation. Enabling technologies and these key areas of research, explored in depth in Chapters 2, 3, 4, and 5, are briefly described below.
Self-configuration and Adaptive Coordination
Given the expected pervasive and ubiquitous nature of EmNets, it will be necessary for these systems to be able to configure themselves and adapt to their environments automatically. Self-configuration and adaptive coordination comprise a spectrum of changes that a system makes to itself in response to occurrences both internal to it and external. EmNets will be relatively long lived, which greatly increases their chances of being upgraded, extended, and otherwise modified. Moreover, EmNets will be exposed to both continual environmental and component dynamics. In effect, the original EmNet must be designed with automatic reconfiguration and adaptation in mind, especially when the specifics of that reconfiguration cannot be known at design time. Current work in distributed systems has not solved the problem of systems operating under the constraints that networked systems of embedded computers will experience, particularly with respect to computational resources, communication limitations, and energy restrictions.
Self-configuration is the process of interconnecting available elements into an ensemble that will perform the required functions at the desired performance level. Self-configuration in existing systems is evidenced by the notions of service discovery, interfaces, and interoperability. In this report, the research challenges related to self-configuration focus on mobile code and discovery. EmNets present a number of constraints: They
will appear in hybrid environments of mobile and static networks; their nodes will be diverse in capability, energy availability, and quality of connectivity; the wireless layer is both diverse and limited by energy constraints, making low power discovery a challenge. Some of the issues that will need to be investigated and resolved for configuration and adaptation to succeed in EmNets include stable localized control, abstraction, and memory use. Research issues related to service discovery include the scaling of discovery protocols, security, and the development of adequate failure models for automatically configured networks.
Adaptive coordination involves changes in the behavior of a system as it responds to changes in the environment or system resources. Coordination will not be mediated by humans because EmNets are so large and the time scale over which the adaptation will need to take place is too short for a human to be able to intervene. Achieving adaptive coordination in EmNets will not only require drawing on the lessons learned from adaptive coordination in existing distributed systems, but it will also require meeting the radical new challenges of EmNets that are due to the physically embedded nature of the collaborative control tasks and the massive numbers of elements, all combined with the relatively constrained capabilities of individual elements. Adaptive coordination is a fairly new area of investigation, particularly as it applies to EmNets. To obtain necessary adaptability in EmNets, research is needed in three areas: exploiting massive redundancy to achieve system robustness and longevity, decentralized control, and collaborative processing.
Building Trustworthy EmNets
EmNets will be deployed in large numbers and will become an essential part of the fabric of everyday life. In the same way that people often assume that electric power and telephone service will be available (recent events in California notwithstanding), they will assume the availability and proper functioning of EmNets. But in contrast to those utility services, EmNets will be deployed in situ, often without the dedicated expert service and maintenance associated with utilities, making the trustworthiness of EmNets triply difficult: EmNets are real-world systems, often directly affected by wind, weather, and interference; they must embody the redundancy needed for dependability without compromising the basic economics, and they must adequately and safely convey to a nonexpert user how much of that redundancy is available (thereby determining the system’s safety margins) so that users can make reasonable decisions concerning their use. This report discusses five features that must be addressed in the design of EmNets from the outset: reliability, safety, security, privacy, and usability.
Reliability is the quality of a system that is satisfying its behavioral specifications under a given set of conditions and within defined time periods. Current verification techniques are not readily applicable to EmNets because of the large number of elements, highly distributed nature, and environmental dynamics. Simply testing individual components is insufficient. Moreover, it is not clear that the community has the vocabulary to fully characterize what will be required of EmNets. Research is needed on fault models and recovery techniques for EmNets, monitoring and performance-checking facilities, and verification tools and techniques.
Safety refers to the ability of a system to operate without causing an accident or unacceptable loss. It is distinct from reliability and poses another set of research problems for EmNets. EmNets increase the number of possible behaviors and the complexity of the possible interactions within the system. Further, they operate in real time and with limited human intervention and are likely to exhibit emergent or unintended behaviors. Analyzing and designing such systems with regard for safety considerations is a challenge. Several safety topics deserve further research effort, including hazard analysis for EmNets, validating requirements, designing for and verifying safety, and ensuring safety in upgraded hardware.
Security is difficult to achieve in virtually all information systems, but EmNets again present particular challenges. The networking of embedded devices will greatly increase the number of possible points of failure, making security analysis even more difficult. Defining and then protecting system boundaries where physical boundaries are likely to be nonexistent and where nodes can automatically move in and out of the system will be a serious challenge. Further, managing the scale and complexity of EmNets while at the same time handling the security challenges of mobile code and the vulnerability to denial-of-service attacks will require significant attention from the research community.
Related to but separate from the issue of security is the issue of personal privacy. EmNets of the future will be able to gather more information than current systems and will do so in a much more passive manner. Achieving consensus on privacy and confidentiality policies will be exacerbated by the pervasiveness and interconnectedness of EmNet systems. Notifying users that they are being monitored, especially in the case of wide-ranging sensor networks, is a challenge, and acquiring consent in a meaningful fashion is an even greater challenge. Determining how to handle the vast amounts of personal information that will be collected and implementing privacy policies once they are decided on is a large area ripe for research.
Finally, and related to all of the above, EmNets will need to be usable by persons with little or no formal training. Unfortunately, usability and safety often conflict, and decisions on trade-offs will need to be made. Understanding the way people create mental models of the systems they use and interact with is a good way for designers to begin to address the issues of usability and manageability. In particular, more research is needed in designing for a range of persons—including system administrators, users who are explicitly operating the EmNet, and persons who are interacting with objects in their environment without explicit knowledge of the system behind them—and in enhancing mental models and user training.
Models of Computation
While there is always some divide, the gulf between theory and practice in EmNets seems to be extremely wide and continuing to grow. In addition to the systems research proposed, more theoretical work is also required. In particular, new models of computation are needed to describe, understand, construct, and reason about EmNets effectively. A critical question is, How should large aggregates of nodes be programmed to carry out their tasks in a distributed and adaptive manner?
Current distributed computing models such as distributed objects and distributed shared memory do not fully address all of the new requirements of EmNets. EmNets’ tight coupling to the physical world, the heterogeneity of their systems, the multitude of elements, and timing and resource constraints, among other things, demonstrate the need for a much richer computing model. Computational models for EmNets will need to incorporate resource constraints, failures (individual components may fail by shutting down to conserve energy, for example), new data models, trust, concurrency, and location.
Developing these computational models for EmNets will require a new approach. As experience is gained with applications and implementations of the technology, designers and implementers will discover which of the new abstractions are useful. Research in this arena will thus require a balance between system implementation and experimentation and the development of the model itself. Run-time environments will also be required that support the models being developed, allowing for faster construction of the experimental systems. This cycle of concurrent development—whereby the computational model feeds into the implementation, experimental results from which feed back into the computational model—will facilitate more accurate and effective models for EmNets.
The evolution leading to EmNets derives from the revolutionary advances in information technology during the last several decades, with silicon scaling as the driving force. Exponentially increasing processor performance has contributed to a world in which sophisticated chips can be manufactured and embedded easily and cheaply. Continued improvements (in line with Moore’s law) in the price and performance of chip technology are expected throughout the decade. Even though the creation of EmNets will be supported in general by advances in the enabling information technologies, research is needed on specific aspects of communications, geolocation, software and operating systems, and MEMS.
As silicon scaling has drastically reduced the cost of computation, it has also driven down the cost of communication for both wireline and wireless systems. As wireless technology continues to become less expensive and more sophisticated, the vision of connecting embedded processors everywhere becomes increasingly feasible. However, most of the progress to date in wireless technology has focused on medium- to long-range communications (as in cellular phones and pagers) and is not sufficient for the widespread deployment of EmNets. Work is needed to understand how to create network architectures and designs for low-power, short-range wireless systems.
Related to wireless are the issues surrounding geolocation technology. Unlike conventional computer networks, which are more dependent on the relative positioning of elements in a network topology, EmNets are often inextricably tied to the physical world (a primary purpose often being to measure and control physical-world attributes or objects), so location in physical space is more important. Many EmNets will therefore require ready access to absolute or relative geographic information.
Work should continue in MEMS technology in order to achieve real-world physical sensing and actuation. Experimental progress in EmNets will be enabled by the availability of a wider range of MEMS-based sensor components. While this technology has advanced tremendously in the past decade, attention must be given to the effective integration of MEMS devices into EmNets.
Continuing research into operating systems for networks of embedded computers and into the development of software that has the required characteristics will also be necessary. EmNets software will need to be tailorable to physical constraints and application requirements in deployment, be upgradable, have high availability, and be able to work with new hardware. EmNets will be embedded in long-lived structures but will also have to evolve, depending on changing external conditions
and advances in technology as time passes. Software (operating systems and applications) that can cope with this type of evolution will be critical. Further, EmNets will often impose real-time and performance-critical constraints on software. New methods of software development may be needed in order to ensure that complex EmNet software is up to coping with the constraints placed on it.
RECOMMENDATIONS AND RESEARCH THEMES DISTILLED
Networked systems of embedded computers will be implemented and deployed even if there is no additional research. Some of them may succeed, and others may appear to have succeeded at least for a time. But any such attempts will somehow have to overcome the fundamental gaps in knowledge that are described throughout this report. To realize functionally powerful, flexible, scalable, long-lived, and trustable systems, a spectrum of research is essential. Moreover, the committee (composed of people from both academia and industry) believes that while some of the questions raised in this report may be answered without a concerted, publicly funded research agenda, leaving this work solely to the private sector raises a number of troubling possibilities. Of great concern is that individual commercial incentives will fail to bring about work on problems that have a larger scope and that are subject to externalities: interoperability, safety, upgradability, and so on. Moreover, a lack of government funding will slow down the sharing of the research, since the commercial concerns doing the research tend to keep the research private to retain their competitive advantage. The creation of an open research community within which results and progress are shared is vital to making significant progress in this arena.
The committee generated eight overarching themes that intersect the three key areas for research described above (self-configuration and adaptive coordination, trustworthiness, and computational models). Research into all of the themes is required before EmNets can fulfill their potential. Research in broadly relevant areas such as networking and usability that pervade many of the themes described below is also essential:
Predictability and manageability. Methodologies and mechanisms for designing predictable, safe, reliable, manageable EmNets;
Adaptive self-configuration. Techniques to allow adaptive self-configuration of EmNets to respond to volatile environmental conditions and system resources in an ongoing dynamic balance;
Monitoring and system health. A complete conceptual framework to
help achieve robust operation through self-monitoring, continuous self-testing, and reporting of system health in the face of extreme constraints on nodes and elements of the system;
Computational models. New abstractions and computational models for designing, analyzing, and describing the collective behavior and information organization of massive EmNets;
Network geometry. Ways to support and incorporate network geometry (as opposed to just network topology) into EmNets;
Interoperability. Techniques and design methods for constructing long-lived, heterogeneous systems that evolve over time and space while remaining interoperable;
Integration of technical, social, ethical, and public policy issues. Fundamental research into the nontechnical issues of EmNets, especially those having to do with the ethical and public policy issues surrounding privacy, security, reliability, usability, and safety; and
Enabling technologies. Ongoing research into the various component and enabling technologies of EmNets.
The committee also recognizes that to ensure that the right kinds of research are conducted to advance EmNets, the structure and conduct of the research enterprise need to be adapted. Achieving these adaptations may not be easy, but the committee identifies them as goals: Effective collaboration between industry and academia, with support from federal funding agencies, is a necessity. Further, inter- and multidisciplinary endeavors will be crucial to the success of this field. Balancing the roles of industry and university, balancing applications with fundamental research, and incorporating multidisciplinary perspectives are all requirements for the EmNet research endeavor that will require a fresh perspective from the community.
Recommendations to Federal Agencies
The Defense Advanced Research Projects Agency (DARPA), the National Institute of Standards and Technology (NIST), the National Science Foundation (NSF), and other federal agencies all have significant roles to play in the development of robust EmNets and EmNet-related research.
Defense Advanced Research Project Agency
DARPA has an ongoing investment in EmNet technologies. Indeed, EmNets will be incredibly important and have tremendous implications for almost all aspects of defense activities, from battlespace monitoring and coordination to asset monitoring to logistics. EmNets will support
defense activities from the seafloor to space. It is now time for DARPA to build on past programs in this area; to expand research in information technology, networking, and the particular areas described above; and to move forward to meet the challenges posed by networked systems of embedded computers. Without DARPA-guided investment in this area, systems issues will not get the critical attention that they need, resulting in more expensive and much less robust systems. The effort requires immediate and sustained attention. A single program will not meet the challenges presented by EmNets. Several programs could be set up, including the following:
Designing for predictability, reliability, and safety;
Collaborative signal processing;
Multiscale location-aware systems; and
Interoperability over time and space.
While the committee considers that work in these programs is necessary, this list is by no means comprehensive. Instead, it is intended to serve as a starting point for ideas for future programs.
The research agenda for EmNets (outlined in depth in this report) is broad and deep, requiring long-term attention. Follow-on programs even beyond the ones described above will be critical. DARPA should aggressively pursue programs that build upon and interact with one another’s intellectual contributions and with some of the seed programs that have already begun explorations in related areas. To better meet the needs of EmNet-related research, the committee also makes two specific recommendations to DARPA:
Recommendation 1. The Information Technology Office of the Defense Advanced Research Projects Agency should revise both the substance and process of its EmNet-related programs to better address the research needs identified in this report. DARPA has several ongoing programs that could be revised or expanded to better meet the needs outlined here.
Recommendation 2. The Defense Advanced Research Projects Agency should encourage greater collaboration between its Information Technology Office and its Microelectronics Technology Office to enable greater experimentation. Greater collaboration between these offices would facilitate rich and significant experimentation in EmNet-related areas.
National Institute of Standards and Technology
NIST has worked in a variety of areas to help make information technology more secure, more reliable, more usable, and more interoperable. All of these characteristics are crucial to current and future EmNet-related technologies. Specifically, the committee recommends as follows:
Recommendation 3. The National Institute of Standards and Technology should develop and provide reference implementations in order to promote open standards for interconnectivity architectures. It will be important to promote open standards in the area and promote system development using commercial components by making public domain device drivers available.
Recommendation 4. The National Institute of Standards and Technology should develop methodologies for testing and simulating EmNets in light of the diverse and dynamic conditions of deployment. Comprehensive simulation models and testing methodologies for EmNets will be necessary to ensure interoperable, reliable, and predictable systems. In particular, the development of methodologies for testing specification and interoperability conformance will be useful.
National Science Foundation
NSF’s multidisciplinary efforts, its work to integrate research and education, and its coordinated systems efforts will be of great importance in the support of EmNet-related research projects. NSF should continue these efforts and include cross-divisional efforts where appropriate. Specifically, the committee recommends as follows:
Recommendation 5. The National Science Foundation should continue to expand mechanisms for encouraging systems-oriented multi-investigator, collaborative, multidisciplinary research on EmNets. NSF can facilitate collaborative multidisciplinary research both through the programs it supports and through the use of a flexible process that encourages the incorporation of perspectives from a broad range of disciplines.
Recommendation 6. The National Science Foundation should develop programs that support graduate and undergraduate multidisciplinary educational programs. It could take the lead in tackling institutional barriers to interdisciplinary and broad systems-based work. NSF has a history of encouraging interdisciplinary programs
and could provide venues for such work to be explored as well as foster and fund joint graduate programs or joint curriculum endeavors.
Other agencies such as the Department of Energy (DOE) and the National Aeronautics and Space Administration (NASA) can play an important role by sharing their specialized knowledge in this area with others working in less specialized areas in the broader community. These and other federal agencies should coordinate their EmNet-related development efforts with the programs at DARPA, NSF, and NIST to ensure that open-platform systems of various scales, low-power components and their software drivers, debugging techniques and software, and traffic generators can all be shared among research programs when applicable, avoiding redundancy in those parts of the system where there is more certainty. It is expected that this sharing and associated coordination needs can be supported by the various organizations and groups associated with federal information technology research and development.
EmNets will radically transform the way in which people interact with and control their physical environment. They have tremendous implications for all aspects of society, from national defense and government applications to wide-ranging commercial concerns to systems that private individuals will use in everyday life. As it moves forward in the research areas described above, the research community, including academia, industry, and funding agencies, must remain cognizant of one basic message: New approaches to the study of systems (not just individual components) must be developed in order to harness the emergent properties of the many networked, physically embedded computing elements that will make up EmNets. Attention must be paid to designing systems in a way that incorporates strategies from a range of disciplines and to designing systems that can address a range of problem domains. Without concerted effort on the part of the research community to address the questions outlined in this report, the potential inherent in networked systems of embedded computers will not be realized. With significant inter- and multidisciplinary research efforts that focus on the systems issues that EmNets bring to the fore, the promise of this technology can be realized.