Virtual Reality Scientific and Technological Challenges (1995) / Chapter Skim
Currently Skimming:
PART I OVERVIEW
PART I OVERVIEW
Pages 11-88
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 13... ...
In virtual reality (VR) or virtual environment (NIE)
|
From page 14... ...
Another source of confusion results from political and public relations considerations. Virtual reality and virtual environment (`two terms that we regard as equivalent)
|
From page 15... ...
computer generation of virtual environments, (2) design of telerobots, (3)
|
From page 16... ...
Concern with haptic interfaces and force-feedback sensor display systems is also intense. As a consequence, Japan has established 10 national consortia for research and development in the SE area that, taken together, provide more funds per year than all SE investment in the United States (Larinaji, 1994~.
|
From page 17... ...
Here, we outline some of the principal defining ideas and indicate how the terms virtual environment, teleoperator, and augmented reality are related to each other and to other closely related terms such as simulator, telerobot, and robot. Our purpose is to provide background on the meaning of the terms we use in order to permit readers to understand later sections of the report.
|
From page 18... ...
Each virtual object has a location and orientation in the surrounding space that is independent of the operator's viewpoint, and the operator can interact with these objects in real time using a variety of motor output channels to manipulate them. The extent to which a virtual environment is designed to simulate a real environment depends on the specific application in mind.
|
From page 19... ...
Thus, for example, VE systems are now being introduced as subsystems of teleoperator systems in order to assist the human operator in controlling the telerobot. In particular, when the telerobot is sufficiently far removed from the human operator to cause significant time delays in the transmission of information between the telerobot and the human operator, virtual environments can be used to present computer-generated information derived from predictive models in the computer.
|
From page 20... ...
. In many such cases, information from the real environment is sensed directly by means of a see-through display, and the supplementary information from the virtual environment is overlaid on this display.
|
From page 21... ...
If the components to the right of the link are deleted and the computer on the left is used to generate a virtual environment, the system reduces to a pure virtual environment system. If all the components are included, but the computer on the left is not used to generate a virtual environment, then Me system reduces to a pure teleoperator system (which would have supervisory control if local, low-level actions were controlled by one of the computers)
|
From page 22... ...
that, whereas a simulator is most intimately tied to the given physical system with which the user is expected to interact (i.e., it is designed to simulate this physical system) , a VE system is most intimately tied to the human operator (i.e., it is designed to include a general-purpose interface to match the human organism, as well as the capability for generating a large range of virtual worlds)
|
From page 23... ...
Other devices, such as force-reflecting joysticks, act as tool handles and serve not only a control function but also a display function because they are capable of providing force feedback. Haptic interfaces in which hand position and posture are tracked and object properties such as texture and temperature are displayed to the hand (as well as simple force information)
|
From page 24... ...
With suitable sensors and actuators, the object can be made to feel stiff or spongy by systematically manipulating the characteristics of the force cues as a function of the position and motion of the fingers relative to the position of the object. In this manner, it is possible to create haptic images of virtual objects (a further defining characteristic of VEs)
|
From page 25... ...
This system includes a special virtual-heart computer program obtainedfrom the National Medical Library of Physical/Computational Models of Human Body Systems and a special haptic interface that enables her to interact manually with the virtual heart. Special scientific visualization subroutines enable her to see, hear, andieel the heart (and its various component subsystems)
|
From page 26... ...
26 ~ Rig sack ~ ~y If file rel~ff~e p~sfU~ ~fAe she/ And fDa If cab be prec~se/y co~fro/Zed exam away fag Geoff ~ De~ff~ he A~ ~e~for ~ fAese sac/ fe/eoper~for sysfe~s ~e~ar~/ly Has Carcass Oaf only fo ra~l-f/~ ~is~1 /~es ~ fag turf aft fag feleroDofic cameras amp/oyed {~ fag sysfe~/ Oaf /so fo ~~a~faJ-re~/Oy f-~ffo~ Jerf~ediom offs ~se~sf~ ~d o~erl~f~ on file real /~es. Some ~ fresh offer bates, lfRe file ~lfr~so~d flute me~No~ed anode/ are derf~e~ fig gag/ b~~ offers size f~rm~ffo~ off of pre~fo~s awes ~i co~frf8~fe fo file s~eo~'s Less file p~ffe~f's he~rf Aisfo~.
|
From page 27... ...
OVERVIEW Gael AND S66 Eat fha per~if~ D~ scat assay/ f~~/s Rosa ma~s~ra~a~fs era r~pfi/y cA~yf~. ^~ scat Ads par ~ad If fag Offal copay ~sso~f~fad Offs fag SE sf~ffo~/ ~ sf~J~rd Do~-sc~ pry pro~fdad 8y fag sA~pf~ ~af~ort fo iffy fag I/ 0~d s~8scrfDad, ~d ~ staffs/ Dog sfoctf~ Offs Miff Baas fo cf~f~fa fag ~fo~~ff~ ~a~s~rama~f procass.
|
From page 28... ...
High School Education Peter Roberts, the 16-year-old son, is doing what previously was called school homework. The distinction between doing school work at school and doing homeworkfor school at home has become very muddy; in both cases, much of the time is spent interacting with teachers, other students, and virtual worlds via networks of SE stations.
|
From page 29... ...
Mathematics In the mathematics course being taken by Peter, the SEfaciZities are used to provide course participants with an intuitive understanding of various non-Euclidean geometries. Participants in the course enter into virtual worlds in which the properties of the space are determined by the axioms of the particular geometry being studied.
|
From page 30... ...
30 ~ Rig so file ~f~ospheric co~diffo~ ifspl~ys ~e~er~fed Dy file sysfe~. Such as each ~e~s~re~e~f Off records of Olin femper~f~re/ press~re/ ~i A~~f~ffy/ Off also cerf~f~ mo/ec~/~r c~sf8~e~fs ~ file s~/ed off/ fore are ~~y prefers If bane fo be r~rese~feJ fig file ifspl~.
|
From page 31... ...
Each student visiting a virtual world is assigned a virtual actor in this world and must learn to control this actor in a manner that satisfies the constraints designed into the specific scenario considered. In general, these constraints are used to give the participants experience in living in different physical environments (e.g., in the desert, in the arctic, on the moonJ; in different social or anthropological settings (e.g., as a member of an ancient culture, a highly discriminated-against minority, a person with severe physical disabilities; or even as a member of a different animal species (e.g., as a member of an insect society or as a sea-dwelling creature low down on the food chainJ.
|
From page 32... ...
The third set of issuesfocuses on the creation of appropriate laws for governing behavior within virtual environments. The number of cases in which VE crimes and misdemeanors are occurring is increasing, and no significant body of law is available to handle these cases.
|
From page 33... ...
With the firs t kind, each individual in thefamily wears a head-mounted display and works through his or her own SE station; in the second, rather than using the traditional head-mounted displays, the family sits together in a special room outfitted with a wall-sized visual display, a set of acoustical loudspeakers, and a set of video cameras. Each individual is assigned one of the video cameras, which then tracks that individual as he or she moves around the room.
|
From page 34... ...
Thefact that Henry's desire to get back to nature is rather common is evidenced by the enormous growth taking place in the camping equipment industry. Henry has also increased the amount of time he spent exercising in the real world.
|
From page 35... ...
Application Domains of SE Systems The range of potential applications for SE systems is extremely large. Application domains currently receiving considerable attention include: (1)
|
From page 36... ...
In the following subsections, we discuss briefly the other application domains listed above. As indicated in these subsections, significant re
|
From page 37... ...
It is not yet clear, however, how to choose the tasks that will eventually prove most appropriate for the application of SE technology. Not only are the results obtained in the various application domains still too meager to allow one to specify the nature of such tasks from empirical data, but also there is no evidence that much effort has been given to answering the question "What is SE technology good for?
|
From page 38... ...
Augmentedreality systems are being studied to present visual displays in which information previously obtained from special imaging techniques is overlaid on the normal direct view of the patient; integrated NfE and teleoperator systems are being developed for use in telediagnosis and telesurgery and for the training of surgeons. In general, the potential benefits of telemedicine that are being considered include not only the ability to obtain medical information and perform medical actions at a distance, but also the ability, as in any other application of teleoperation, to effectively transform the sensorimotor system of the operator to better match the task at hand.
|
From page 39... ...
Aside from the general lack of excitement engendered by visions of teleoperation compared with visions of virtual reality, perhaps interest in the use of teleoperator systems for hazardous operations is limited by the lack of personal experience most people have with hazardous operations (e.g., defusing a bomb or locating and carrying to safety a child from a burning building)
|
From page 40... ...
Of particular concern is the issue of training transfer. Much remains to be known about which of the possible differences between the real task and the task as realized in the envisioned VE training systems are likely to be important, either positively or negatively, and which are insignificant.
|
From page 41... ...
(The term visualization is used here in its most general sense and only for historical purposes; we do not mean to imply that the information is necessarily presented only through the visual channel.) This application domain, like the training domain, cuts across the other application domains considered: effective visualization of information is important in essentially all domains.
|
From page 42... ...
Specialists in various fields of science are beginning to make use of advanced computer graphics techniques for improved visualization, and preliminary research and development are being conducted on the use of the auditory and haptic channels for this purpose. Telecommunication and Teletravel The domain of telecommunication and teletravel, like training and information visualization, cuts across essentially all of the other domains considered.
|
From page 43... ...
One set of such topics focuses on human performance characteristics and includes, for example, sensorimotor resolution, perceptual illusions, information transfer rates, and manual tracking. Knowledge about all of these topics is essential to cost-effective design of SE systems.
|
From page 44... ...
Although alterations involving time delays or noise generally have negative effects and are therefore to be avoided in designing and constructing SE systems, alterations involving fixed transformations or distortions may be introduced intentionally to enhance performance. For example, such conditions arise automatically in any teleoperator system that employs a nonanthropomorphic telerobot (e.g., a telerobot that has four eyes, six arms, and moves about on wheels)
|
From page 45... ...
Cognitive scientists have been working for many years to describe the processes used by humans to acquire and build knowledge structures. This work has led to a variety of hypotheses and some research results suggesting that knowledge acquisition strategies and features of effective information presentation depend on a person's level of knowledge, the characteristics of the content area, and the type of task performance required.
|
From page 46... ...
This asymmetry arises because the overwhelming majority of previous work on computer generation of NIEs has been restricted to the visual channel. As the field matures and the computer science community becomes more involved in the generation of auditory and haptic images as well as visual images and the community concerned with the auditory and haptic channels becomes more involved with computer synthesis of environmental signals and objects, this imbalance will become less severe.
|
From page 47... ...
In addition, we consider position tracking and mapping, motion interfaces, speech communication, physiological responses, and a display system that presents information by means of odors and radiant heat. Visual Channel Of all the devices associated with the human-machine interface component of VE systems, visual displays have received the greatest attention.
|
From page 48... ...
, from unintentional optical distortions in an HMD, or from time delays and noise generated somewhere within the visual channel. Also, we still know relatively little about how various characteristics of the visual display influence performance on various types of tasks.
|
From page 49... ...
to attenuate unwanted signals from the immediate real environment are effective, inexpensive, and ergonomically reasonable. (The main ergonomic problem occurs when earphones and visual displays have to be used together and the design of the helmet that includes the visual display does not take proper account of the need to also stimulate the auditory channel.)
|
From page 50... ...
Such substitution is being considered both when the visual channel is overloaded and when appropriate haptic feedback is unavailable. Another set of issues being studied, which is highly relevant to synthetic auditory displays (independent of whether they are being used for sensory substitution)
|
From page 51... ...
Hybrid systems, in which bodybased and ground-based devices are combined, are also likely to be required for some applications (e.g., both to track finger motion and to provide force feedback to the hand without causing forces to be applied to other portions of the body)
|
From page 52... ...
At present, one of the main experimental thrusts is directed toward the development of tool-handle systems, in which the human user manipulates a real tool handle, the actions of the tool handle are used to control some feature of a telerobot or a VE, and force feedback is displayed through the tool handle according to interactions of the telerobot with the real environment or of the virtual tool with virtual objects in the VE. Relatively little work, even at the research and development level, has been directed toward haptic interface devices that provide feedback (related to perception of texture or temperature)
|
From page 53... ...
Illustrative issues of particular concern in this research include determination of the mechanical properties of the soft tissues in contact with haptic interfaces; quantification of limits on human sensing and control of contact forces and hand displacements; identification of stimulus cues in the perception of contact conditions and object properties; and characterization of human sensorimotor performance in the presence of time delays, distortions, and noise. Basic research on technology includes development of novel technologies for sensor and actuator hardware; design of computer architectures for fast computation of physical models; and development of algorithms for real-time control of devices that render factual images.
|
From page 54... ...
Currently, motion interfaces for passive transport are being used primarily in flight simulation for flight training, in the entertainment industry for "thrill rides," and in research projects directed toward improved understanding of human perception and performance (including motion sickness) in a wide variety of contexts involving real or simulated passive transport.
|
From page 55... ...
for each foot with position and force sensors and force feedback. Although a number of systems that are considered more advanced than the common treadmills and exercise machines are beginning to be developed and evaluated by research groups, no such systems are yet available commercially.
|
From page 56... ...
Computer Generation of VEs To many people, and certainly to most computer scientists, computer generation of VEs is the core of the SE field; to them, human-machine interfaces, telerobots, and even the human operators, are of secondary importance. Furthermore, most past and current work in this area has focused on the generation of visual images.
|
From page 57... ...
Such a system is beyond the current technology, and it is anticipated that for a long time to come, trade-offs between realism and interactive capacity will be required. Furthermore, due to these limitations, effective VE implementation will depend on targeted application domains.
|
From page 58... ...
Software In order to provide a fully interactive, real-time, natural-appearing environment, software development is required in a wide variety of areas. The real-time generation of VEs requires consideration of interaction, navigation, modeling, the creation of augmented reality, hypermedia integration, and operating system software.
|
From page 59... ...
When a VE application requires a replica of a real environment, it is generally considered preferable to map
|
From page 60... ...
Automatic generation of effective augmented reality is still at the research stage. A major issue is the ability to create and maintain accurate registration between the real and synthetic environments, particularly when they are both rapidly varying.
|
From page 61... ...
Such applications frequently arise in the domains of hazardous operations and medicine. In all such applications, telerobotics plays an essential role; human-machine interfaces and computer-generated NIEs are not sufficient.
|
From page 62... ...
increase, and as the importance of haptic interfaces with force feedback decreases. It should also be noted, however, that transport delays will increase in importance in VE systems not only as such systems increase their use of haptic interfaces with force feedback, but also as VE networks with tightly coupled players at distant locations come into being.
|
From page 63... ...
Currently, the two application domains in which the most networking activity is occurring are entertainment and national defense. In the entertainment industry, VR companies are in the process of forming cooperative arrangements with cable television companies to develop multiuser games and interactive shopping.
|
From page 64... ...
High Performance Computing and Communication program, which is supporting the installation of OC-12 networks at five regional test beds for research purposes. Workstations with three-dimensional graphics will be connected to the WANs discussed above through local-area networks (LANs)
|
From page 65... ...
Other questions focus on how well the design accounts for human perceptual and cognitive features or for human responses to alterations in sensorimotor loops. In the cognitive area, research indicates that it is difficult to make generalizations about the
|
From page 66... ...
Furthermore, there are the important concerns about induced motion sickness in SEs and the potential after-effects of adaptation to SEs when an individual moves back into the real environment. There are many types and levels of evaluation that can be used to provide direction, understanding, and a general picture of performance effectiveness.
|
From page 67... ...
The research agenda we propose covers four main categories: (1) Application domains, (2)
|
From page 68... ...
The committee has not assigned priority to the application domains of education; information visualization; and telecommunication and teletravel. Although committee members agreed that the education domain is exceedingly important perhaps the most important of all the domains considered it was not assigned priority because of our judgment that the development of improved education technology will have only a minor effect on the quality of education actually received.
|
From page 69... ...
Similarly, work on hazardous operations and training will necessarily involve work on telecommunication and teletravel. An additional reason for denying the information visualization domain priority is the fact that work in the area of scientific visualization using VE is already quite active.
|
From page 70... ...
Not only should it be possible to develop models in which the visual components are superior to those now achieved with conventional techniques, but the addition of the auditory and haptic channels in such efforts should also greatly facilitate the education process. Studies of training transfer, the second special project area cited, are essential to the successful development of VE training procedures for a wide variety of training tasks in a wide range of application domains.
|
From page 71... ...
Development of a comprehensive, coherently organized review of theory and data on human performance characteristics (including consideration of basic sensorimotor resolution, perceptual illusions, information transfer rate, and manual tracking) from the viewpoint of SE systems.
|
From page 72... ...
RECOMMENDATION: The committee recommends support of research on visual displays, haptic interfaces, and locomotion interfaces, with emphases as outlined below.
|
From page 73... ...
. Independent of whether the images that are combined with computer-generated VE images are derived directly from the real environment by means of see-through displays or indirectly through the artificial eyes of a telerobot, the psychophysics of such merged displays must be carefully studied.
|
From page 74... ...
Particular attention should be given to the development of tool-handle interfaces in which the possible haptic interactions are constrained by the nature of the tool. Special support also should be given to basic haptic science relevant to the development of improved haptic interfaces.
|
From page 75... ...
Computer Generation of VEs Hardware Advances in computational and communication hardware are essential to the full realization of VEs. The hardware capabilities available today have given researchers, entrepreneurs, and consumers just a taste of virtual worlds and a promise of possible applications.
|
From page 76... ...
Even within We confines of the visual channel, it appears that relatively little attention is being given to time-deterministic generation of images (i.e., systems that guarantee compliance with appropriate bounds on graphics update rate and lag, possibly at the expense of resolution)
|
From page 77... ...
computer generation of auditory and haptic images. There is a need to develop methods and software to interpret and respond to multimodal inputs from a wide variety of devices, including those associated with position tracking, haptic manipulation, and speech commands, often occurring in concert.
|
From page 78... ...
Simulating the mechanics of the everyday world will be of central importance in giving virtual environments a sense of solidity and allowing users to effectively manipulate virtual objects through haptic interfaces. The problems that arise in generalizing standard batch-simulation methods to handle interactive VEs are analogous to those that arise in the extension of static rendering techniques.
|
From page 79... ...
. In addition to the general areas discussed above, the committee recommends that research and development on the following topics be supported: navigational cues in virtual space, the behavior of autonomous agents, and the computer generation of both auditory and haptic images for VE.
|
From page 80... ...
In order to combat the effects of such delays, continued efforts should be directed toward the development of improved control algorithms that ensure stability and yet, to the extent possible, provide reasonable gains. At the same time, continued effort should be directed toward the development of improved methods for constructing and using predictive displays and for realizing effective supervisory control strategies.
|
From page 81... ...
However, for applications with demanding input/output operations, the committee does not recommend new real-time development systems or operating systems. The most demanding VE system will require powerful real-time input/output capabilities to handle haptic interfaces, trackers, visual dis
|
From page 82... ...
As a result, we do not recommend additional investment in network hardware development at this time. Nevertheless, it is important to acknowledge the existence of significant infrastructure problems that could impede the use of networks for VE applications.
|
From page 83... ...
It also recommends that the federal government help coordinate the development of standardized testing procedures for use across studies, systems, and laboratories, particularly in those areas in which the private sector has not acted. i' SE technology is in the early stages of development, is growing rapdly, and is the subject of highly optimistic projections about its usefulness.
|
From page 84... ...
These procedures should include methods for identifying key system dimensions that affect task performance, developing special metrics uniquely suited to evaluating SEs, and comparing SE system performance to performance of other systems intended to meet the same or similar goals. Suggestions for Government Policy and Infrastructure The magnitude, quality, and effect of the SE-oriented research and development that is accomplished will clearly depend on the role played by the federal government.
|
From page 85... ...
In order to make such computational models available to society, the federal government might consider establishing a national system for standardizing, collecting, storing, and disseminating such models. In view of society's current concerns with health care, initial efforts in this area might be focused on computational models related to the structure and function of the human body and modifications of the human body associated with injury, trauma, disease, aging, and medical and surgical treatments.
|
From page 86... ...
, special efforts are required to ensure adequate communication and cooperation across disciplines. Similarly, because of the high value placed on competition within our society, special efforts are needed to ensure adequate communication and cooperation across government agencies, military branches, industrial firms, and academic institutions.
|
From page 87... ...
Although military applications trail behind those associated with the entertainment industry as an economic driving force, they nevertheless constitute a force that is significant. This significance is derived not only from the overall magnitude of the associated economic activity, but also from the special role played by defense agencies in stimulating the development of relatively high-quality systems for military applications.
|
From page 88... ...
Analyze and Evaluate Market Forces and Societal Impact The extent to which government funds will be directed toward specific SE research depends, at least in part, on the likelihood that such projects will be funded independently, i.e., by industry. Estimating this likelihood requires not only an analysis of current market forces, but also predictions of how market forces will evolve in the future.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.