Virtual Reality Scientific and Technological Challenges (1995) / Chapter Skim
Currently Skimming:
8 Computer Hardware and Software for the Generation of Virtual Environments
8 Computer Hardware and Software for the Generation of Virtual Environments
Pages 247-303
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 247... ...
Limiting VE technology to primarily visual interactions, however, simply defines the technology as a more personal and affordable variant of classical military and commercial graphical simulation technology. A much more interesting, and potentially useful, way to view VEs is as a significant subset of multimodal user interfaces.
|
From page 248... ...
rendering hardware and software for driving modality-specific display devices; (2) hardware and software for modalityspecific aspects of models and the generation of corresponding display representations; (3)
|
From page 249... ...
With regard to computer hardware, there are several senses of frame rate: they are roughly classified as graphical, computational, and data access. Graphical frame rates are critical in order to sustain the illusion of pres
|
From page 250... ...
Experience has shown that, whereas the graphical frame rate should be as high as possible, frame rates of lower than 10 frames per second severely degrade the illusion of presence. If the graphics being displayed relies on computation or data access, then computation and data access frame rates of 8 to 10 frames per second are necessary to sustain the visual illusion that the user is watching the time evolution of the VE.
|
From page 251... ...
There are some commonalities of hardware and software requirements, and it is those commonalities on which we focus in our examination of the state of the art of computer hardware and software for the construction of real-time, three-dimensional virtual environments. HARDWARE FOR COMPUTER GRAPHICS The ubiquity of computer graphics workstations capable of real-time, three-dimensional display at high frame rates is probably the key development behind the current push for NIEs today.
|
From page 252... ...
There is no current graphics hardware that provides this, so we must make approximations at the moment. This means living with less detailed virtual worlds, perhaps via judicious use of hierarchical data structures (see the software section below)
|
From page 253... ...
The frame buffer is scanned repeatedly by the display hardware to generate visual imagery. Each of the fragments includes x and y window coordinates, a color, and a depth for use with the z-buffer for hidden surface elimination.
|
From page 254... ...
There are four major graphics systems that represent different architectures based on crossbar location. Silicon Graphics RealityEngine is a flow-through architecture with a primitive crossbar; the Freedom series from Evans & Sutherland is a flow-through architecture with a fragment crossbar; Pixel Planes 5 uses a tiled primitive crossbar; and PixelFlow is a tiled, pixel merge machine.
|
From page 255... ...
An Example: The Virtual Wind Tunnel In this section, we consider the implications of the VE performance constraints on the computation and data management requirements of a VE system. An example of an application that is both computationally intensive and works with large numbers of data is the virtual wind tunnel (Bryson and Gerald-Yamasaki, 1992~.
|
From page 256... ...
This is two orders of magnitude beyond the seek time capabilities of current disk systems. Another way to address the data size problem is to develop data compression algorithms.
|
From page 257... ...
The development of appropriate data compression techniques for many application domains is an open area of research. Another strategy is to put as much of the dataset as possible in physical memory.
|
From page 258... ...
There are two run-time models that are currently common in computer graphics: the simulation loop model, in which all operations in the visualization environment (including interaction, computation, data management, and graphics) are performed in a repeated single loop; and the event-driven model, in which operations occur in response to various events (usually generated by the user)
|
From page 259... ...
An alternative run-time model that is gaining popularity is the concurrent model, in which different operations in the environment are running simultaneously with one another, preferably on several processors. The example of the simulation loop broken into the two asynchronously running graphics and computation processes discussed above is a simple example of concurrency.
|
From page 260... ...
Dataset size will be the dominant problem for an important class of applications in NIE. In the near term, an effective VE platform would include the following: multiple fast processors in an integrated unit; several graphics pipelines integrated with the processors; very large shared physical memory; very fast access to mass storage; operating systems that support shared-memory, multiprocessor architectures; and very high-speed, low-latency networks.
|
From page 261... ...
SOFTWARE FOR THE GENERATION OF THREE-DIMENSIONAL VIRTUAL ENVIRONMENTS There are many components to the software required for the real-time generation of VEs. These include interaction software, navigation software, polygon flow minimization to the graphics pipeline software, world modeling software (geometric, physical, and behavioral)
|
From page 262... ...
This inference may be complicated by inaccuracies in the hardware providing the signal. Existing Technologies Although there are several paradigms for interaction in VEs, including direct manipulation, indirect manipulation, logical commands, and data input, the problem of realistic, real-time interaction is still comparatively unexplored.
|
From page 263... ...
Data or text input can be provided by conventional keyboard methods external to the VE. Within the environment, speech recognition may be used for both text and numerical input, and indirect manipulation of widgets provides limited numerical input.
|
From page 264... ...
How does dealing with this special case change the general form of any user interface techniques we might devise? · Unfortunate Coincidences The archetypical example of this phenomenon is the famous optical illusion in which a person stands on a distant hill while a friend stands near the camera, aligning his hand so that it appears as if the distant friend is a small person standing in the palm of his hand.
|
From page 265... ...
Some solutions to the selection problem involve ray casting or voice input, but this leaves open the question of specifying actions and parameters by means of direct manipulation. Some solutions emphasize a body-centric approach, which relies solely on the user's proprioceptive abilities to specify actions in space.
|
From page 266... ...
An important aspect of the selection process is the provision of feedback to the user confirming the action that has been taken. This is a more difficult problem in three dimensions, where we are faced with the graphic arts question of how to depict a selected object so that it appears unambiguously selected from an arbitrary viewing angle, under any lighting circumstances, regardless of the rendering of the object.
|
From page 267... ...
There are many component parts to this software, including control device gesture interpretation (gesture message from the input subsystem to movement processing) , virtual camera viewpoint and view volume control, and hierarchical data structures for polygon flow minimization to the graphics pipeline.
|
From page 268... ...
For more abstract applications, there have been several experimental approaches to controlling the vehicle. The most common is the point and fly technique, wherein the vehicle is controlled via a direct manipulation interface.
|
From page 269... ...
One key method to get the visual scene updated in real time at interactive update rates is to minimize the total number of polygons sent to the graphics pipeline. Hierarchical data structures for polygon flow minimization are probably the least well understood aspect of graphics development.
|
From page 270... ...
: View volume definition View point, reference point Bounding volume (BV) Floo~ Floor 2: ~ Floor n: ~ ~ ·~- ~ Rm1,1 Rm1,2 Rm1,3 Rm2,1 Rm2,2 Rm2,3 R 1 Rm n,2 Rm n,3 BV BV\ BV BV BV BV Bvm n, BV BV \Minimal set of polygons \Medium set of polygons \Complete set of polygons FIGURE 8-5 Hierarchical data structure for polygon flow minimization.
|
From page 271... ...
, and Turk (1992~. Application-Specific Solutions Polygon flow minimization to the graphics pipeline is best understood by looking at specific solutions.
|
From page 272... ...
In principle, methods such as finite element modeling could be used as the basis for representing these properties and for physical simulation and rendering purposes. At the other extreme, separate, spatially and temporally coupled representations could be maintained that represent only those properties of an object relevant for simulating and rendering interactions in a single modality (e.g., auditory events)
|
From page 273... ...
require models that may not be physically based. In this section, we concentrate on the visual domain and examine the problems of constructing geometric models, the prospects for vision-based acquisition of real-world models, dynamic model matching for augmented reality, the simulation of basic physical behavior, and simulation of autonomous agents.
|
From page 274... ...
Meanwhile, automatic or nearly automatic acquisition of geometric models is practical now in some cases, and partially automated interactive acquisition should be feasible in the near term (Ohya et al., 1993; Fuchs et al., 1994~. The most promising short-term approaches involve active sensing techniques.
|
From page 275... ...
Dynamic Model Matching and Augmented Reality The term augmented reality has come to refer to the use of transparent head-mounted displays that superimpose synthetic elements on a view of the real surroundings. Unlike conventional heads-up displays in which the added elements bear no direct relation to the background, the synthetic objects in augmented reality are supposed to appear as part of the real environment.
|
From page 276... ...
Although NIE displays provide direct motion measurements of observer movement, these are unlikely to be accurate enough to support high-quality augmented reality, at least when real and synthetic objects are in close proximity, because even very small errors could induce perceptible relative motions, disrupting the illusion. Perhaps the most promising course would use direct motion measurements for gross positioning, using local image-based matching methods to lock real and synthetic elements together.
|
From page 277... ...
In the last few years, researchers in computer graphics have begun to address the unique challenges posed by this kind of simulation, under the heading of physically based modeling. Below we summarize the main existing technology and outstanding issues in this area.
|
From page 278... ...
In principle, the ability to simulate simple objects such as rigid bodies, together with the ability to prevent interpenetration, could suffice to model most such compound objects. For instance, a working desk drawer could be constructed by modeling the geometry of a tongue sliding in a groove, or a door by modeling in detail the rigid parts of the hinge.
|
From page 279... ...
Another approach to simulating constrained systems of objects builds on the classic method of Lagrangian multipliers, in which a linear system is solved at each time step to yield a set of constraint forces. This approach offers several advantages: first, it is general, allowing essentially arbitrary holonomic constraints to be applied to essentially arbitrary (not necessarily rigid)
|
From page 280... ...
Motivated by the obvious need in both computer graphics and engineering for realism and physically based environments that support various levels of object detail and interaction (depending on the application) , Metaxas (1992, 1993; Metaxas and Terzopoulos, 1992a, 1992b, 1993; Terzopoulos and Metaxas, 1991)
|
From page 281... ...
Maintaining a constant frame rate will require the development of new methods that degrade gracefully in such situations. Autonomous Agents The need for simulated autonomous agents arises in many VE application areas, such as training, education, and entertainment, in which such agents could play the role of adversaries, trainers, or partners or simply supernumeraries to add richness and believability.
|
From page 282... ...
Beyond the added realism that the presence of virtual actors can provide in those situations in which the participants would normally expect to see other human figures, autonomous actors can perform two important functions in NIE applications. First, autonomous actors can augment or replace human participants.
|
From page 283... ...
In the computer graphics community, there is a long history of human figure modeling, but this work has considered, for the most part, kinematic modeling of uncoupled motion exclusively. With today's graphics workstations, kinematic models of reasonably complex figures (say, 30 to 40 degrees of freedom)
|
From page 284... ...
, and Rijpkema and Girard (1991) have reported some work with automated grasping, but their systems seem to be focused on key frame-like animation systems for making animated movies, rather than for real-time interaction with virtual actors.
|
From page 285... ...
In the absence of real sensors, the observation module uses probabilistic models and inference rules to generate the belief system of the autonomous force. It accounts for battlefield conditions, as well as the capabilities and knowledge of individual autonomous forces, to determine whether and with how much accuracy various events on the simulated battlefield can be observed.
|
From page 286... ...
Authoring is especially useful in training and analysis because of this ability to play back the engagement from a specific point of view. Some examples of the uses of hypermedia in virtual worlds are presented in the following paragraphs.
|
From page 287... ...
and may point the way for future VE representation, simulation, and rendering approaches. In addition, the concept of frame rate, both in terms of update rate
|
From page 288... ...
This number can potentially place burdens on the computation system one order of magnitude greater than currently acceptable update rates of visual-only VE. How these issues and the numerous others that are likely to be encountered with further exploration are to be handled by computational systems is still an open and important area for further research.
|
From page 289... ...
(Disk bandwidth and size thus arise as limiting factors in video on demand and hypermedia integration in virtual worlds.) Current projections suggest that workstations capable of supporting 15 Gbytes of physical memory might be available sometime in 1994 but that the cost will be prohibitive for all but the most well-funded research groups.
|
From page 290... ...
In the near term, an effective VE platform would include multiple fast processors in an integrated unit; several graphics pipelines integrated with the processors; very large shared physical memory; very fast access to mass storage; operating systems that support shared memory within multiprocessed architectures; and very high-speed, low-latency networks. To ensure continued development, it is necessary to encourage both private- and public-sector participation.
|
From page 291... ...
are important to SE developers requiring some level of real-time control. Substantial subsets of the capabilities specified in these standards are now available on some graphics workstations (e.g., SGI workstations running the IRIX 5 version of the operating system)
|
From page 292... ...
Because language, which provides much of the intentional information in the real world, is not currently available for use in virtual worlds, other options must be thoroughly explored and developed. Unfortunately, the size of current government research efforts for work on providing natural interfaces is small.
|
From page 293... ...
The difficulty of polygon flow minimization depends on the composition of the virtual world. This problem has historically been approached on an applicationspecific basis, and there is as yet no general solution.
|
From page 294... ...
These redevelopment efforts also slow the progress of new development. There are additional technical issues in polygon flow minimization that are important.
|
From page 295... ...
Although VE displays provide direct motion measurements of observer movement, these are unlikely to be accurate enough to support high-quality augmented reality, in situations in which real and synthetic objects are in close proximity. Even very small errors could induce perceptible relative motions that could disrupt an illusion.
|
From page 296... ...
Thus the fragment crossbar is itself more easily parallelized than the primitive crossbar, allowing for the necessarily greater bandwidth of rasterized fragments over window-coordinate primitives. The primary disadvantage of the fragment crossbar com - ~>I 1 ~ 1 Per Primitive FIGURE 8-6 Primitive broadcast.
|
From page 297... ...
Per Fragment pared with the primitive crossbar is that fragment crossbar systems have difficulty rendering primitives in the order that they were presented to the graphics system, whereas primitive broadcast systems easily render primitives in the order presented. Whereas the frame buffers in the primitive broadcast and fragment crossbar systems were disjoint, collectively forming a single, screen-size buffer, the frame buffers of a pixel crossbar system (Figure 8-8)
|
From page 298... ...
A secondary disadvantage exists only with respect to primitive broadcast systems: the pixel crossbar, like the fragment crossbar, has difficulty rendering primitives in the order presented. (Each path renders the primitives presented to it in the order that they are presented, but the postrendering pixel merge cannot be done in order.)
|
From page 299... ...
Finally PixelFlow, the proposed successor to Pixel Planes 5, is a tiled, pixel merge machine. Thus it is unable to efficiently render primitives in the order in which they are received, and the rendering latency of PixelFlow is perhaps twice that of Freedom and RealityEngine, though less than that of Pixel Planes 5.
|
From page 300... ...
In support of this role, Silicon Graphics has engaged in the development of multiple processing, graphics workstations at the leading edge of technology since late 1983. A brief look at the graphics performance numbers of their high-end systems since that time is warranted (Table 8-1~.
|
From page 302... ...
capable of speeds well beyond 3 million polygons/ s, high-speed pixel routing interconnection, high-speed access to frame buffer for image processing (up to 100 million pixels/s) , and a pixel fill rate of 95 million pixels/s.
|
From page 303... ...
PixelFlow performs anti-aliasing by supersampling. It supports deferred shading with separate hardware shaders that operate on composite images containing intermediate pixel data.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.