Tactical Display for Soldiers: Human Factors Considerations (1997)

Situation awareness is a critical element of successful performance in the combat environment. The battlefield poses a variety of challenges to situation awareness: information overload, nonintegrated data, rapidly changing information, and a high degree of uncertainty brought on by lack of needed information. Overcoming these problems is a major goal of the Land Warrior System. Evaluating the degree to which proposed system designs actually provide benefits to situation awareness and help the soldier to think and act quickly, however, is a critical issue that needs to be addressed through careful design testing. Assessment of situation awareness in a systematic fashion will allow potentially critical problems to be detected, many of which are outlined in this chapter. The final designs selected must provide soldiers with the situation awareness they need to be successful in performing their many functions. Although it is beyond our current understanding to specify how much situation awareness is enough, researchers in the area believe that good performance is linked to good situation awareness (Endsley, 1995).

The chapter begins with a discussion of the individual characteristics that affect situation awareness, including perception, attention allocation, working memory, long-term memory stores, goal-directed behavior, and individual differences in relevant abilities. It then covers factors related to the task and the display system, including workload, complexity, automation, and environmental stressors. Next we consider the situation awareness of the combat unit as a team and how information is distributed, as well as how the display design can be expected to provide the specific type of support that soldiers need to perform their tasks. We then outline various approaches to measurement of situation awareness, in-

cluding process indices, direct measures, behavioral measures, and performance measures. The chapter ends by summarizing the key points and key design recommendations in support of situation awareness.

A person's situation awareness can be described as his or her state of knowledge or mental model of the surrounding situation or the environment. It is not just spatial orientation but includes an understanding of the dynamics of the situation and the actions that are expected to take place in the future. Many definitions have been developed, some very closely tied to the aviation domain and some more general (see Dominguez, 1994, and Fracker, 1988, for reviews). A general, applicable definition describes situation awareness as ''the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning and the projection of their status in the near future" (Endsley, 1988a). It includes not only perceiving or attending to information, but also the integration of multiple pieces of information and determination of their relevances to one's goals, as well as the ability to forecast future situation dynamics providing for timely and effective decision making.

Situation awareness requirements for soldiers consist of the dynamic information needed to support each of their tasks and objectives. Although the specific objectives and tasks of the soldier vary from mission to mission, some common critical tasks can be hypothesized to include: detection and identification of targets, identification of terrain features, navigation and localization of self and others, engagement to include fire and maneuvering, communications between and within units, mission rehearsal planning and replanning, and development of tactics.

The elements of the environment that form the situation awareness requirements for the infantry soldier are as many as required to support each of these goals. They can include factors relevant to both the local and the global situation, as shown in Figure 3-1. Global situation awareness needs can be construed to include one's location within a broad geographical area, navigation information such as the relative location of important features, the current location and direction of movement of other units (friendly and enemy) and current commands and directions from headquarters. All these factors are relevant to the soldiers' ability to navigate and plan strategically to meet their goals. In order to know where to move to, and where not to, this type of information is critical.

Local information needs can include the location of a desired target in the immediate environment, the identity (friend, foe or neutral) of an entity under current targeting, terrain and object location (as needed for basic mobility and maneuvering), and cueing of the presence and movement of enemies in the immediate environment. This information is critical to the soldier's basic aware-

FIGURE 3-1 Situation awareness needs of the infantry soldier.

ness of hostile presences around him and the ability to act and react quickly in accordance with mission goals.

Both global and local needs for situation awareness are critical for effective functioning in a given environment. The local information is required for effectively acting to meet immediate needs. The global information is required for employing oneself effectively in concert with other units to meet strategic goals.

It should be noted that the advent of helmet-mounted displays in this domain may affect each of the two types of situation awareness needs quite differently. For example, although the displays being considered may produce better situation awareness at the global level, it may also reduce local situation awareness by removing the soldier's attention from the immediate surroundings. (This issue is discussed in more detail later). This examples illustrates two major issues to be noted: (1) there is a definite need for the soldier to maintain an adequate level of awareness across all elements of the environment, and (2) due to limits on attention, gains in situation awareness on some elements can occur at the expense of losses on other elements. The potential for these trade-offs to occur needs to be very closely examined during the design process in order to ensure that adequate situation awareness is provided across all requirements.

Endsley (1988a, 1994, 1995) has proposed a framework model of situation awareness based on information-processing theory (Wickens, 1992), Figure 3-2. The model includes a number of factors that influence situation awareness and that can be seen to be potentially affected by the proposed helmet-mounted display.

It is believed that situation awareness is primarily restricted by the limits of attention and the capacity of working memory, which constrain a person's ability

to take in and process multiple channels of information. Long-term memory stores, when they have been developed through experience, most likely in the form of schemata and mental models, are proposed to circumvent these limits by providing for the integration and comprehension of information and the projection of future events (higher levels of situation awareness), thus significantly offloading working memory and efficiently directing perceptual processes. The use of mental models in achieving situation awareness is considered to be dependent on the individual's ability to match critical cues in the environment and elements in the mental model. Schemata of prototypical situations are incorporated in this process and in many instances may also be associated with scripts to produce single-step retrieval of actions from memory, thus providing for very rapid decision making, as has been noted by Klein (1989).

In addition, information held in working memory (current goals, expectations, other situational information) is believed to affect attention deployment and the perceptual process. Situation awareness is largely influenced by a person's goals and expectations, which influence how attention is directed, how information is perceived, and how it is interpreted. The model shows this goal-directed, top-down processing operating in tandem with bottom-up processing, in which highly salient cues activate appropriate goals and mental models. Thus, situation awareness is the result of an ongoing process of alternating between goal-directed processing and data-driven processing based on a theorized link between goals and mental models. In the sections that follow, we discuss several factors that affect the accuracy and completeness of that situation awareness that soldiers derive from their environment.

First, basic perceptual processes may be affected. Trade-offs may occur between the degree to which the helmet-mounted display enhances perception (with night vision, for example) and the degree to which it interferes with normal perceptual processes, including hearing and vision. Situation awareness can be negatively impacted if the device prevents perception of important environmental information or creates misperceptions.

The way in which attention is employed in a complex environment with multiple competing cues is essential to determining which aspects of the situation will be processed. A major factor of situation awareness that may be negatively influenced by the helmet-mounted displays is the soldier's attention allocation.

Attentional narrowing has been cited as a major factor in errors related to situation awareness (Endsley, 1995). The display creates a new source of information that may compete with the outside world for the soldier's attention, rendering him

susceptible to attentional narrowing (focusing narrowly) on the display, thereby missing important information in the external environment. The tendency to focus attention on electronic information displays is encouraged by their high degree of perceptual salience. With a display located directly in front of the soldier's eye, this hazard may become even more pronounced. Alternative display technologies that do not directly interfere with the intake of environmental information may need to be considered in order to minimize this problem; however, it is doubtful that the danger can be completely eliminated. It may also be necessary to consider uses (e.g., distributing the capability among team members) that compensate for potential losses of local situation awareness when using the system.

Once taken in, information must be integrated with other information, compared with goal states, and projected into the future--all activities that make heavy demands on working memory. The limits of a soldier's working memory should be considered, particularly since working memory can be further reduced under stress, such as that of combat (Hockey, 1986; Mandler, 1979). The design of any electronic device that is introduced into this type of environment must take the limits of working memory into account with an interface that minimizes requirements to memorize commands, syntax, or other information. Its presentation in an easy to process format will also be critical for minimizing demands on the soldier's limited processing resources.

In this context, it is critical that the information provided truly support the attainment of the soldier's goals and not impose extra demands or provide extraneous information. New information cannot be provided to the soldier without some costs; there is a danger of going from too little information to too much information. The presence of a new source of information that must be integrated with information in the environment can degrade situation awareness by imposing extra processing requirements. Due to the limited ability to perceive and process information, significant difficulties may be encountered unless stringent measures are taken to integrate multiple sources of information, reduce extraneous information, simplify the format of information presentation, and integrate the presentation of information with the soldier's tasks.

Long-term memory stores in the form of mental models or schema are hypothesized to play a major role in dealing with the limitations of attention and working memory. With experience, people develop internal models of the system they operate and the environments they operate in. These mental models serve to help direct limited attention in efficient ways, provide a means of inte-

grating information without loading working memory, and provide a mechanism for generating projections of future system states. The new technologies being considered for the soldier may promote good situation awareness by allowing information to be presented in a format that is more compatible with these mental models. The ability to provide concrete, up-to-date maps of the environment with explicit representations of tactical units can reduce mental processing requirements. For instance, egocentric displays have been found to be superior to exocentric displays for self-locomotion by improving the compatibility of the direction of motion in the display and the motion in the environment (Wickens et al., 1989). It may also be more advantageous to receive certain information visually instead of audibly, or vice versa.

Goals are also important for situation awareness. Human information processing is seen essentially as alternating between data-driven (bottom-up) and goal-driven (top-down) processing. In goal-driven processing, attention is directed across the environment in accordance with active goals. A person actively seeks information needed for goal attainment, and the goals simultaneously act as a filter in interpreting the information that is perceived. In data-driven processing, perceived environmental cues may indicate new goals that need to be active. Dynamic switching between these two processing modes is important for successful performance.

A significant problem can occur if people are not receptive to the perception of relevant data (e.g., slight sounds indicating an enemy sneaking up) while engaged in essentially goal-directed behavior (e.g., searching for information or providing information across a network). A display that significantly loads or interferes with the broad-based perception of information from the environment will therefore be of particular concern. Good display design will enhance the salience of critical environmental cues and will make the attainment of information relevant to a particular goal as easy as possible.

Because anecdotal evidence suggests that some people are better at maintaining a high level of situation awareness, it may be important to consider factors that relate to individual differences in this area during the training and selection process. On the basis of the model presented here, several factors have been hypothesized to be important determinants of variability among individuals in terms of situation awareness capability: (1) spatial abilities, the degree to which one can mentally visualize and manipulate objects and also visualize one's own orientation relative to those objects, (2) attention abilities, specifically attention sharing as needed to achieve situation awareness in a complex environment,

(3) memory, including working memory capacity and the quality and quantity of long-term memory stores, (4) perception, the ability to rapidly perceive and assimilate new information, and (5) logical and analytical skills that may be useful in searching out information and piecing it together (Endsley and Bolstad, 1994).

Testing these hypothesized factors against a measure of situation awareness, Endsley and Bolstad (1994) found that, among pilots, those with higher situation awareness had better spatial abilities and perceptual speed. They also found partial support for a link between higher situation awareness and better pattern matching abilities and attention-sharing abilities. The degree to which these findings can be generalized to an infantry population is not known. This list may provide a starting point for generating research on factors that are relevant to situation awareness for the soldier, however.

Endsley and Bolstad (1994) found a 1 to 10 ratio in situation awareness capability separating the pilots in their study. Scores were furthermore found to be fairly stable for an individual tested at different times, indicating that some people may indeed be better at maintaining situation awareness (either due to innate ability, different strategies employed, or differences in training and experience). This issue needs further exploration in the infantry population. It may be possible to select individuals with better situation awareness capabilities for tasks that involve the new proposed technologies, or to better train them to avoid problems and use effective situation awareness skills, if such skills can be identified.

In addition to individual factors, many features of the environment may affect the soldier's awareness. The task and system factors discussed in this section need to be considered when designing an information support system.

The link between situation awareness and workload is depicted in Figure 3-3 (Endsley, 1993). With low to moderate workload, the level of situation awareness a person has can be independent of workload level. One may have low situation awareness and may not be working very hard to achieve a higher level. Or one may have high situation awareness without having to work very hard (through the benefits of a well-designed system). One may be working fairly hard and may be rewarded with a high level of situation awareness, or one may still have low situation awareness, if one's efforts are ineffective or one misinterprets the information acquired.

At very high levels of workload, situation awareness may suffer, however. If the volume of information and number of tasks are too great, only a subset of

FIGURE 3-3 Relationship between situation awareness and workload. Source: Endsley (1993).

information can be attended to. Or one may be actively working to achieve situation awareness, yet suffer from erroneous or incomplete perception and integration of information. A display that overloads the soldier can lead to low levels of situation awareness. It is important that the technologies implemented in the battlefield do not further increase workload, particularly during high workload tasks.

Poor situation awareness can also occur under low workload, however, in which case the operator may have little idea of what is going on and not be actively working to find out due to inattentiveness, vigilance problems, or low motivation. Although electronic information systems represent a way of productively increasing workload (and information) during periods of low workload, it is critical that this not occur at the expense of maintaining vigilance regarding the immediate environment. Further workload considerations are addressed in Chapter 6.

A major challenge to maintaining good situation awareness is the complexity of many of the systems that must be operated. The more complex the systems are to operate, the greater the increase in the mental workload required to achieve a given level of situation awareness. When that demand exceeds human capabilities, situation awareness will suffer. System complexity may be somewhat moderated by the degree to which the person has a well-developed internal representation of the system to aid in directing attention, integrating data, and developing the higher levels of situation awareness-mechanisms that may be effective for coping with complexity. Developing those internal models, however, requires a

considerable amount of training and may be beyond the capabilities of many soldiers, as indicated by their ASVAB scores, as we discussed in Chapter 2.

Situation awareness may be negatively affected by the automation of tasks. System operators working with automation have been found to have a diminished ability to detect system errors and subsequently perform tasks manually in the face of automation failures, compared with entirely manual performance on the same tasks (Billings, 1991; Moray, 1986; Wickens, 1992; Wiener and Curry, 1980). Although some of this problem may be due to a loss of manual skills with automation, situation awareness is also a critical component (Endsley and Kiris, 1995).

Operators who have lost their situation awareness may be slower to detect problems and also may require extra time to reorient themselves to relevant system parameters in order to proceed with problem diagnosis and the resumption of manual performance when automation fails. This has been hypothesized to occur for a number of reasons, including (a) a loss of vigilance and increase in complacency associated with the assumption of a monitoring role under automation, (b) the difference between being an active processor of information in manual processing and a passive recipient of information under automation, and (c) a loss of or change in the type of feedback provided to operators concerning the state of the system under automation (Endsley and Kiris, 1995). The degree to which the automation of tasks is incorporated in the helmet-mounted display needs to be carefully examined for this potential impact.

Several types of stressors in the combat environment may affect situation awareness, including physical stressors-noise, vibration, heat or cold, lighting, atmospheric conditions, boredom or fatigue, cyclical changes-and social/psychological stressors-fear or anxiety, uncertainty, the importance or consequences of events, self-esteem, career advancement, mental load, and time pressure (Hockey, 1986; Sharit and Salvendy, 1982). A certain amount of stress may actually improve performance by increasing attention to important aspects of the situation. A higher amount of stress can have extremely negative consequences, however, as accompanying increases in autonomic functioning and aspects of the stressors can act to demand a portion of a person's limited attentional capacity (Hockey, 1986).

Stressors can affect situation awareness in a number of different ways, including narrowing attention. With perceived danger, a decrease in attention has been observed for peripheral information-those aspects that attract less attentional focus (Bacon, 1974; Weltman et al., 1971). Broadbent (1971) found an increased

tendency to sample dominant or probable sources of information under stress. This is a critical problem for situation awareness, leading to the neglect of certain elements in favor of others. In many cases, such as in emergency conditions, factors outside the person's perceived central task are the ones that prove to be lethal.

Premature closure, arriving at a decision without exploring all the information available, has also been found to be more likely under stress (Janis, 1982; Keinan, 1987; Keinan and Friedland, 1987). This includes both considering less information and attending more closely to negative information (Janis, 1982; Wright, 1974). Several authors have found that the scanning of stimuli under stress is scattered and poorly organized (Keinan, 1987; Keinan and Friedland, 1987; Wachtel, 1967).

Another way in which stress may impact situation awareness is through decrements in working memory capacity and retrieval (Hockey, 1986; Mandler, 1979). The degree to which working memory decrements affect situation awareness depends on the resources available to the individual operator. In tasks for which achieving situation awareness involves a high load for working memory, a significant impact on situation awareness levels 2 and 3 would also be expected. If long-term memory stores are available to support situation awareness, less effect is expected.

Although anxiety is a common stressor in the battlefield, other stressors, such as fatigue and environmental conditions (cold, heat, humidity), can also take a significant toll on performance and situation awareness through these same mechanisms. To a certain degree, the impact of stressors on situation awareness is a given part of the combat environment. Many new proposed technologies can exacerbate these effects, however, if they interfere with scanning of relevant information in the environment, load working memory, or encourage dependence on highly perceptually salient technological information sources. They can also be designed to mitigate these potential problems by providing an easy to access overview of critical information that might otherwise be neglected or lost from working memory under stress. The effects of stressors on soldier performance are discussed further in Chapter 6.

Within the combat unit, individuals must work together as a team to carry out actions effectively, so overall team situation awareness becomes important. In this context, each team member has a specific set of elements of situation awareness about which he is concerned, as determined by his responsibilities within the team. There is some overlap of each team member's situation awareness requirements, and this subset of information is the basis for much of team coordination. Coordination may occur as a verbal exchange or may be provided through the system display or by some other means (e.g., nonverbal communication). The

quality of the team members' awareness of shared elements (as a state of knowledge) may serve as an index of team coordination or human-machine interface effectiveness. Thus, an important issue is the degree to which the helmet-mounted display will affect situation awareness across the team. It is possible that it may be improved for one individual, but not for others, if the display does not support needed information transfer across the team, or if it physically interferes with other means of information transfer (such as direct verbal and nonverbal exchanges).

The design of information displays should be informed by a careful consideration of the type of support the soldier really needs in achieving better situation awareness and what factors currently act to limit it in his environment. In this light, the real utility of the proposed technologies may be in providing: (1) sensory enhancement, improving the soldier's ability to localize targets and self and to navigate in the environment, (2) more dynamic information, keeping the soldier and commander up to date on changes and situational factors in the field, (3) information sharing between team members, supporting planning and dynamic decision making, (4) distributed decision making providing information across teams and between headquarters and teams, and (5) strategic decision making, allowing soldiers to look at information in different ways, thus supporting different integration, comprehension, and projection possibilities.

An analysis of the impact of the different proposed information technologies shows that they may be expected to impact situation awareness in different ways across local and global needs (see Table 3-1). On one hand, for example, GPS may be expected to dramatically improve global situation awareness. The improved information flow from headquarters to the soldier (and back) can be expected to improve the situation awareness of all parties in their knowledge of the global picture (e.g., the latest location, movements of friendly and hostile units). In addition, the graphics capabilities afforded by the helmet-mounted display should provide an improvement in situation awareness over current piecemeal audio technologies. On the other hand, the helmet-mounted displays and night vision system may reduce local situation awareness by drawing the soldier's attention away from the immediate environment and into the virtual one or by inducing certain misperceptions regarding the actual location (distance) of objects.

As many factors surrounding the helmet-mounted displays and other technologies may act to both enhance and degrade situation awareness in the environment of the infantry soldier, significant care should be taken in evaluating the

impact of proposed concepts on situation awareness. Only by testing new design concepts in carefully controlled studies can their actual impact be identified. This testing should include an examination not only of how the information technologies affect basic human processes such as the accuracy of perception, but also of how they affect the soldiers' situation awareness (across all elements) when used in dynamic and complex field settings in which multiple sources of information compete for attention and must be selected, processed, and integrated in light of dynamic goal changes. Real-time simulations employing the helmet-mounted display can be used to assess the impact of the system by carefully measuring soldier performance, workload, and situation awareness. Direct measurement of situation awareness during design testing is recommended to provide sufficient insight into potential costs and benefits of design concepts for soldiers' situation awareness, allowing a determination of the degree to which the design successfully addresses the issues we have discussed.

Situation awareness is a relatively new focus in system design. Like other constructs, such as workload, a variety of measures have been used in its assessment. At this time it is not possible to identify a particular measure as the ''gold standard." The following review provides the pros and cons of several measures. Each of these may be useful at different times in the design process.

High level performance measures of combat (e.g., kills and losses), as collected under the constrained conditions of simulation testing, are often not suffi-

ciently granular or diagnostic of differences in system designs. Whereas one design concept may be superior to another in providing the soldier with needed information in a format that is easier to assimilate with his needs, the benefits may go unnoticed under the constrained conditions of simulation testing or due to extra effort on the part of soldiers to compensate for a design's deficiencies. If situation awareness is measured directly, it will be possible to select concepts that promote it increasing the probability that soldiers will make effective decisions and avoid poor ones. Problems with situation awareness, frequently brought on by data overload, nonintegrated data, automation, complex systems that are poorly understood, excess attention demands, and many other factors can be detected early in the design process and corrective changes made to improve the design.

Multiple types of testing are desirable. At the most simple level, the system needs to be tested in part-task studies under well-controlled laboratory conditions. This type of testing examines the degree to which certain design features affect human performance in conducting very explicit tasks-for example, time and error rates for finding required information or entering information with different display formats. This testing needs to be very carefully controlled in order to detect potential problems with perceptual tasks (finding information, accurately perceiving the information, detecting information), motor tasks (entering information, range of motion tests, physical interference with environment), and cognitive tasks (finding needed displays, making decisions). This type of testing facilitates the selection of design features that enhance performance.

For situation awareness, an even more critical type of testing involves simulations of the task environment at medium to high levels of fidelity. Scenarios developed for this type of test should incorporate more realistic environmental task loads (e.g., conducting multiple tasks within a realistic mission scenario).

The tests should include both expected and unexpected events and factors-for example, enemies that are not where they were projected to be, sneak attacks, loss of friendly forces, replanning from headquarters. These tests should include multiple types of information coming from multiple sources. The objective is to provide a testing environment that accurately depicts these features of the operating environment, so that the utility of the devices can really be explored. Because situation awareness in particular is very affected by attentional deployment and competing demands, an accurate picture of the impact of any new technology can be examined only by incorporating these issues, showing how the technology may affect situation awareness (across all of its elements) when used in demanding conditions. This type of design testing is essential for the task of designing an integrated human-centered system.

Finally, testing under field scenarios should be considered. This provides the highest level of realism, but also the lowest level of control and measurability of the issues of interest. Although situation awareness may be difficult to assess directly under these conditions, it may be inferred from operational performance.

Scenarios for field testing usually include realistic mission scenarios and environmental conditions (e.g., sweat occluding vision, equipment that shifts in use).

The process model in Figure 3-4 represents the issues involved in selecting measures of situation awareness. This model shows the stages involved in the sequence from perception to action. Although they are shown as separate stages for simplicity in narration, it should be noted that these stages may be very closely coupled. Moderating factors that may influence each stage are shown on the left. On the right, classes of measures appropriate to each stage are shown. Measures at each stage are discussed in the next sections including the advantages and disadvantages of each (Endsley, 1996).

An examination of the assessment processes people use to acquire situation awareness may provide information about how soldiers allocate their attention in using a particular system design. It may indicate the relative priority of different types of information or the relative utility of information sources. In general, however, process measures provide only an indirect indication of operator situation awareness. Eye-trackers may indicate how attention is used the process of acquiring situation awareness, typical scan patterns, and relations between elements. Studying the verbal communications between soldiers may also suggest the types of information that are missing from displays, verbal techniques used for acquiring situation awareness, and differences in situation awareness strategies among individuals.

Verbal protocols may provide some useful information on not only what is attended to, but also how that information is integrated and used. Significant difficulties in processing and using the data provided by verbal protocols must be dealt with by the experimenter, however, if this technique is to be used successfully.

Each of these techniques can be viewed as providing useful partial information on processes of acquiring situation awareness, from which some inferences may be possible. Because verbal communications and verbal protocols take place in a very limited time frame, however, they cannot be regarded as complete representations of what people attend to or process. Eye-trackers and information acquisition methods do not provide any insight on how the information is used or combined to form higher-level situation awareness.

Two types of measures have been developed for assessing situation awareness directly: subjective techniques and questionnaires.

Subjective estimation of situation awareness can be made by individual soldiers or by experienced observers. Subjective assessment is very attractive in that it is fairly inexpensive and easy to administer. In addition to allowing evaluation of design concepts in simulation studies, subjective techniques can be easily applied in less controlled, real-world settings. Certain limitations, however, constrain the interpretation of subjective evaluations of situation awareness.

Self-ratings usually involve a subjective estimation of how much situation awareness a particular person feels he or she has when using a given system design. Self-ratings may not necessarily provide an accurate quantification of situation awareness, however, because people may not know about their own inaccuracies or what information they are unaware of and have a limited basis for making such judgments. In addition, self-ratings may be highly influenced by self-assessments of performance, thus becoming biased by issues that are beyond the construct of situation awareness. These self-ratings may be useful, however, as an assessment of the soldier's degree of confidence in his situation awareness (which can also affect decision making).

One of the best-known subjective scales is the situational awareness rating technique (SART) developed by Taylor (1990). SART has individuals rate a system design based on the demand on attentional resources, the supply of attentional resources, and understanding of the situation provided. As such, it considers individuals' perceived workload (supply and demand on attentional resources) in addition to their perceived understanding of the situation. SART has been developed on the basis of items that air crew report to be important to situation awareness and has been shown to be sensitive to workload variations.

In another approach to developing a standardized subjective measure of situation awareness, Vidulich and Hughes (1995) used a modified version of the subjective workload dominance (SWORD) technique to obtain subjective evaluations of the situation awareness provided by displays. SA-SWORD has subjects provide a comparative preference for displays on a nine-point scale, on the basis of their beliefs about the amount of situation awareness provided by each. The technique has not been validated for measurement of situation awareness, however.

Situation awareness may be assessed by subjective ratings of outside observers. An advantage is that trained observers may have more information than the subject about what is really happening in a given simulation, so their knowledge of reality may be more complete. A shortcoming is that observers have only limited knowledge about the subject's concept of the situation. Operator actions and verbalizations may provide useful diagnostic information on explicit problems (misperceptions or lack of knowledge) and an indication that certain information is known, supporting observer judgments. Actions and verbalizations cannot be taken to provide a complete representation of an operator's situation

awareness, however. They may know many things that they do not mention or make an immediate response to as they are performing other tasks, for example. Observer ratings therefore provide only a partial indicant of a subject's situation awareness. Efforts to elicit more information (by asking questions or providing artificial tasks) may augment natural verbalizations, but this may alter the subject's distribution of attention, thus altering situation awareness.

Questionnaires allow for the collection of detailed information about the subject's perceptions that can be evaluated against reality, thus providing an objective assessment of situation awareness on a detailed level. This type of assessment provides a direct measure and does not require subjects or observers to make judgments about situational knowledge on the basis of incomplete information, as subjective assessments do. This type of information can be gathered in one of three ways: post-test, during simulations, or during interruptions in the simulation.

A detailed questionnaire can be administered after the completion of each simulated trial, allowing ample time for subjects to respond to a lengthy and detailed list of questions. Memories of dynamic situation awareness will be less reliable with time, however; people have been shown to overrationalize and overgeneralize about past mental events (Nisbett and Wilson, 1977). Early misperceptions may be quickly forgotten as the situation unfolds over time. Posttest questionnaires will reliably capture situation awareness only at the very end of a trial. Kibbe (1988) used this technique to evaluate situation awareness as affected by automation of a threat recognition task. She found a retrospective recall measure to be insensitive to the automation and problematic.

One way of overcoming this deficiency is to ask subjects about their situation awareness while they are carrying out their simulated tasks. This strategy may alter situation awareness and task performance, however, as it can be regarded as providing an additional secondary task and intrusive.

To overcome the limitations of reporting on situation awareness after the fact, several researchers have used a technique wherein the simulation is frozen at randomly selected times, the system displays are blanked, and the simulation is suspended while subjects quickly answer questions about their current perceptions of the situation. Subject perceptions are then compared with the real situation based on simulation computer databases to provide an objective measure of situation awareness. The collection of data in this manner provides an objective, unbiased assessment of situation awareness that overcomes the problems incurred when collecting data after the fact, yet minimizes biasing due to secondary task loading or artificially cueing the subject's attention. The primary disadvantage of this technique involves the temporary halt in the simulation.

The situation awareness global assessment technique (SAGAT) is a global tool developed to assess situation awareness across all of its elements on the basis of a comprehensive assessment of operator requirements (Endsley, 1987, 1988b, 1990b). As a global measure, SAGAT includes queries about all operator situation awareness requirements, including Level 1 (perception of data), Level 2 (comprehension of meaning), and Level 3 (projection of the near future) components. It includes a consideration of system functioning and status as well as relevant features of the external environment. The approach minimizes possible biasing of attention, because subjects cannot prepare for the queries in advance (since they could be queried over almost every aspect of the situation to which they would normally attend).

SAGAT has been shown to have predictive validity, with SAGAT scores indicative of pilot performance in a combat simulation (Endsley, 1990a). Content validity was also established, showing the queries used to be relevant to situation awareness in a fighter aircraft domain (Endsley, 1990b). Empirical validity has been demonstrated through several studies that have shown that a temporary freeze in the simulation to collect SAGAT data did not affect performance and that such data could be collected for up to 5 or 6 minutes during a freeze without running into memory decay problems (Endsley, 1990b, 1995). A certain degree of measurement reliability has been demonstrated in a study that found high reliability of SAGAT scores for four individuals who participated in two sets of simulation trials (Endsley and Bolstad, 1994).

To apply this technique to the infantry combat task, a simulation environment in which soldiers perform realistic tasks with and without the aid of the proposed technologies is needed. (Applying the technique in real-world settings may be difficult or prohibitive.) A SAGAT battery of questions would also need to be constructed based on an analysis of the soldier's situation awareness requirements. This allows for domain-appropriate assessments of situation awareness and provides information on how the soldier's situation awareness is affected by each new technology.

Operators can be expected to act in certain ways on the basis of their situation awareness. Some information about situation awareness may, therefore, be determined from examining behavior on specific subtasks that are of interest. Behavioral indices could include time to make a response (verbal or nonverbal) and an estimation of correct or incorrect situation awareness as identified from verbalizations and appropriateness of a given behavior for a particular situation. Assessments of situation awareness based on these types of behavioral measures need to be viewed with caution, since they assume what the appropriate behavior will be, given situation awareness or lack of it. The assumptions may not necessarily be warranted. For example, a subject may choose not to immediately

verbalize or respond to a given event or may employ different response strategies, thus confounding this type of measure.

In general, performance measures provide the advantage of being objective and are usually nonintrusive. Computers for conducting simulation testing can be programmed to record specified performance data automatically, making them relatively easy to collect. Several limitations exist in using performance data to infer situation awareness, however. Global measures of performance (e.g., success in meeting a goal, kills and losses in a battle) are important as measures of situation awareness, however, they are limited. Because many moderating factors can influence the link between situation awareness and performance, global performance measures provide only an indirect indication of situation awareness.

Some definite task measures may readily present themselves for evaluating certain kinds of systems, but for others determining appropriate measures may be more difficult. An expert system, for example, may influence many factors in a global, not readily predictable manner. The major limitation of this approach stems from the interactive nature of situation awareness subcomponents. A new system to provide situation awareness on one factor may simultaneously reduce it on another, unmeasured, factor. In addition, it is quite easy for subjects to bias their attention to a single issue that is under evaluation in a particular study if they figure out the purpose of the study. Overall, relying exclusively on measures of performance on specific parameters can yield misleading results and should be viewed within the context of other types of measures.

A major purpose of the Land Warrior System is to improve the infantry soldier's situation awareness, which in turn is anticipated to improve his performance. There is some evidence that mission performance correlates with situation awareness in aircraft simulators. The link between performance of the infantry soldier and his level of situation awareness has yet to be studied systematically. Based on our review of the literature in this area, we draw the following conclusions regarding the helmet-mounted display and its potential role in situation awareness:

1. The helmet-mounted display system has the potential to enhance situation awareness by providing timely, more up-to-date information, and better sharing of information across team members, units, and geographic areas.
2. The helmet-mounted display may improve the soldier's situation awareness about global information (location of self and others in environment, communications with headquarters, navigation).

3. The helmet-mounted display may compromise the soldier's local situation awareness (location, presence of enemies, terrain and object perception) by competing for limited attention resources, affecting perceptual processes, or both.
4. Hand-held or wrist-mounted displays should be seriously considered as an alternative to the helmet-mounted displays in order to reduce the likelihood of negatively affecting the soldier's local situation awareness.
5. The system can reduce situation awareness if it poses a significant demand on mental resources or shifts the task load away from regular duties to system operation. These problems will be worse under stress and with higher levels of system complexity.
6. Significant increases in requirements for skills and abilities may be created by the helmet-mounted display, indicating the potential for changes in selection and training requirements to allow for acceptable levels of situation awareness and performance.

The following design recommendations are based on our discussion of the cognitive mechanisms involved in achieving situation awareness.

1. The design of the display system should minimize the degree to which it is a physical barrier to acquiring environmental information (e.g., occludes or alters normal hearing and vision). It should enhance sensory input only when needed (e.g., targeting support, night vision).
2. The display design should minimize the degree to which it distracts attention (e.g., make the system removable, place it out of the normal line of sight).
3. The display design should minimize the cognitive load it places on the user by:

• providing integrated information (e.g., fusing information from different sources),
• providing easy user input of information (e.g., menus),
• minimizing memory requirements,
• reducing extraneous information,
• simplifying the format of information presentation,
• minimizing tasks,
• presenting information in a task-oriented sequence and grouping, and
• proving information in the needed format (e.g., egocentric maps).

4. The display should be designed to enhance situation awareness by providing salient cueing, directing attention to the most important information.
5. The display design should minimize complexity and avoid high levels of automation.

6. The system should provide new capabilities needed by the soldier, such as integrating information (as needed for decision making), comparing information to pertinent goal states, allowing a projection of future states, and providing support for human memory.
7. The display design should allow for easy sharing of information between team members and between the field and headquarters.