National Academies Press: OpenBook
« Previous: 2 Overview
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

3

New Constructs for Assessing Individuals

Recent psychological research points toward a variety of constructs that can be used in assessments to provide greater accuracy or additional valuable information about the individuals being assessed. The workshop presenters described a number of such cutting-edge constructs, primarily during the first panel: Emerging Constructs and Theory. In this chapter, invited presentations during the first panel from Christopher Patrick, Michael Kane, and Todd Little are described, as well as the related presentation by James Rounds on interests, during the workshop’s second day.

NEUROBEHAVIORAL CONSTRUCTS

Christopher Patrick, a professor of clinical psychology at Florida State University, discussed one approach to revising existing constructs and developing new constructs: the psychoneurometric approach. He defined psychoneurometrics as “the systematic development of neurobiologically based measures of individual difference constructs, using psychometric operationalizations as referents.” In essence, it is a way of developing measures of individual differences by combining information and insights from neurobiology (the study of the biological aspects of the brain and nervous system) with what has been learned from psychological studies of individual differences. An associated aim of the psychoneurometric approach, Patrick continued, is to refine the individual difference constructs themselves through the incorporation of physiological data—such

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

things as brain activity, the levels of hormones and other biological molecules, and measurements of various reflexes. “I want to work back and forth between the physiological data and the starting constructs and come to things [new individual difference conceptions] that would make more sense from a neurobiological standpoint,” he said.

There are several reasons for incorporating physiology into individual differences assessments, Patrick said. First, many of the major contemporary trait-dispositional models refer to neurobiology, but the neuro-biological referents tend to be added after the fact. That is, the traits used in these models were initially developed on the basis of self-report data, and it was only afterward that researchers sought to identify their neuro-biological counterparts. By contrast, the psychoneurometric approach seeks to incorporate neurobiological indicators from the beginning so that the trait conceptions themselves are shaped by neurobiological data.

Patrick gave two other reasons for incorporating physiology into the assessment of individual differences: (1) to help address the issue of response bias (which refers to the tendency of people answering questions to be influenced by what they believe the questioner expects), and (2) to gain insight into the processes involved in how people confront and cope with a given situation. The old model of understanding behavior in a situation, the stimulus-response model, was superseded by the stimulus-organism-response model, in which processes occurring within the organism are considered crucial for understanding the connection between the stimulus and the response. Biology is an important part of understanding such relationships. Finally, Patrick noted that understanding the physiological basis of capabilities is important to the design of optimal training performance methods.

Before introducing the two main constructs that he studies, Patrick offered two key points in thinking about the psychoneurometric approach. First, neurobiological indicators of any type are complex and multi-determined. In particular, the reliable person-variance in any physiological indicator will reflect sources other than just the performance capability of interest. And second, linking the domains of physiology and adaptive performance requires a bridge of some sort. The bridging approach he employs is the use of neurobehavioral constructs, by which he means “constructs that have clear referents in both neurobiology and behavior.” Constructs of this type can serve as referents for combining physiological indicators with indicators from other domains (e.g., self-report or overt behavioral responses) to form composite measures that have meaning both psychologically and physiologically.

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

Defensive Reactivity

Patrick’s work in psychoneurometrics to date has focused on two neurobehavioral constructs. The first is defensive reactivity, which he defined as “proneness to negative emotional reactivity in the face of threat.” It is a cue-specific negative response, he emphasized, in contrast to, for instance, a free-floating anxiety or neuroticism—that is, the sort of worry or negative emotion that has no obvious immediate cause or trigger.

The presumed neural basis of defensive reactivity is “individual differences in the sensitivity or responsiveness of the brain’s defensive system, including the amygdala and affiliated structures.” The amygdala is a part of the brain that plays a key role in the processing of cues signaling uncertainty or danger and in the formation of memories of events that have emotional content, such as the memory of a frightening event. Defensive reactivity may also involve interactions with frontal cortical systems of the brain, he said, in terms of what we think of as emotion regulation or inhibition. Frontal cortical brain regions are involved with, among other things, the representation of complex emotional cues or contexts, the formation of long-term memories associated with emotions, and control (regulation) of affective responses.

Patrick’s operational model of defensive reactivity—that is, the specific way in which he measures it in human subjects—is based on measures that have been developed to index variations in fear versus fearlessness (or boldness). The model was based on an analysis of data from 2,500 twin participants who completed self-report questionnaires whose scores have been shown in experimental studies to be related to fear-potentiated startle. This fear-potentiated startle is a standard physiological indicator of fear that is often, in practice, the observation of an eye blink in response to some unexpected stimulus, such as a loud noise (Kramer et al., 2012). Figure 3-1 shows how the model represents dispositional fear versus boldness as the common individual difference dimension indexed by differing scale measures of fear/fearlessness.

“We found evidence for a general factor [or dimension] on which all of these measures either loaded positively or negatively,” Patrick said. (Two measures are positively correlated when increases in one are associated with increases in the other; they are negatively correlated when an increase in one is associated with a decrease in the other.) Some of the measures Patrick discussed reflect the expression of fear versus boldness in the social domain; others reflect such expression in the activity preference or the sensation-seeking domain, and still others reflect expression in the perceived experience (feeling) domain. “From the standpoint of this model,” he said, “we think of neurobiological fear as the core of expres-

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

image

FIGURE 3-1 Operational model: trait fear versus boldness.
NOTE: EAS-F = emotionality-activity-sociability tearfulness scale; FSS = fear survey schedule-Ill; PPI = psychopathic personality inventory (F = fearlessness subscale, SI = stress immunity subscale, SP = social potency subscale); SSS-TAS = sensation seeking scale, thrill and adventure seeking subscale; TPQ-HA = tridimensional personality questionnaire, harm avoidance scale (HA1 = anticipatory worry and pessimism subscale, HA2 = fear of uncertainty subscale, HA3 = shyness with strangers subscale, HA4 = fatigability and asthenia subscale).
SOURCE: Adapted from Kramer et al. (2012).

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

sion of fear in different domains, as manifested in self-report, that might hang together separately for other reasons than the physiology of those characteristics.” That is, this “dispositional fear” can be thought of as the degree of defensive reactivity exhibited in different psychological contexts; these contexts may seem separate in personality models based on self-report, but from a neurobiological standpoint, the behavior in each context is influenced by variations in dispositional fear. Patrick added that he and his colleagues have followed up on this quantitative modeling work to develop fine-grained scales for measuring the general fear versus boldness dimension of the model using self-reports. They are also interested in measuring the construct behaviorally with various tasks as well as with physiological measures. “Again,” he reiterated, “we are choosing these indicators because of their relationship to a neurophysiological indicator,” that is, to the fear-potentiated startle response. Thus assessment techniques framed around this model are grounded in neurophysiology.

Applying his ideas to real-world scenarios, Patrick said the trait characteristic of low dispositional fear or “boldness” was well illustrated by the recent article, “Fearless Dominance and the U.S. Presidency: Implications of Psychopathic Personality Traits for Successful and Unsuccessful Leadership” (Lilienfeld et al., 2012). The authors had expert presidential biographers rate the presidents on facet-level traits of the Big Five personality traits model. Based on prior work linking Big Five personality traits to measures of psychopathy, the authors then used these facet ratings to estimate scores for the presidents on factors of psychopathy, one of which, “fearless dominance,” is very similar to his own concept of boldness, Patrick said. Among the U.S. presidents considered in the article, Theodore Roosevelt was rated highest in boldness.

In his book The Antisocial Personalities, David Lykken writes, “The hero and the psychopath are twigs on the same genetic branch” (Lykken, 1995). The idea, Patrick explained, is that some people may possess the temperament of a psychopath, but because of other factors they do great things. “Lykken talked about the bold tendencies of Winston Churchill as an example of someone who had what he saw as the temperament of a psychopathic individual, but expressed in a more benign sort of adaptive way.” What is the difference between someone who expresses this characteristic of boldness adaptively versus maladaptively? Part of the answer, Patrick said, may be found in the second trait he has been studying—inhibitory control.

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

Inhibitory Control

The second construct on which Patrick’s research focuses is inhibitory control, which is defined as “the ability to restrain or modulate impulses.” This trait relates to the degree to which people can modulate their tendencies and their behavior under conditions that require a certain amount of flexibility and foresight. The presumed neural basis for inhibitory control lies in individual differences in the functioning of anterior brain circuitry, including the prefrontal cortex subdivisions and the anterior cingulate cortex. The operational model for inhibitory control is what he refers to as the “externalizing spectrum,” or “trait inhibition-disinhibition” model (Krueger et al., 2007). The sample that Patrick and his colleagues used to develop the model consisted of students as well as prisoners “because we wanted to make sure that we mapped the full range of the continuum by including representations of individuals with very extreme externalizing tendencies.”

Again, as with defensive reactivity, the goals of this work were to develop an individual difference measure that had physiological correlates and to develop effective and efficient scales for measuring this construct through self-reports. The model they developed contains 23 facet scales that represent proclivities toward impulsiveness, aggression, rebelliousness, risk taking, and use and abuse of substances, all of which correlate with a broad inhibition-disinhibition factor (Krueger et al., 2007). In this case, the physiological correlates of the inhibition-disinhibition factor include the P300 response and the error-related negativity response, which are two types of brain reactions that can be detected and recorded using the technique of electroencephalography that records brain electrical activity from the scalp surface.

The Value of the Constructs

Patrick gave three reasons why the constructs of defensive reactivity and inhibitory control may be of interest to those developing assessments for military personnel: (1) adaptive performance, (2) direct brain referents, and (3) insights from psychoneurometrics.

Importance to Adaptive Performance

The first reason why these constructs may be of interest for military assessments is that they are important to adaptive performance. For example, a 2009 study found that individuals who were high in boldness were better able to maintain their focus on a task under threat of a shock (Dvorak-Bertscha et al., 2009). Thus it is reasonable to predict that high boldness would predict enhanced adaptive flexibility in a threatening situation.

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

Responding to a question regarding the distribution of boldness across the population, Patrick indicated that it shows a normal distribution very similar to the levels of intelligence within the population. He also noted that lack of inhibitory control has been shown to correlate with a propensity to develop post-traumatic stress disorder (PTSD) (Miller et al., 2006). He also speculated that, while individuals with high inhibitory control may still develop PTSD, the presence of inhibitory control is likely to produce a more focused and contained reaction to the specific event, rather than manifesting as generalized fear.

Similar to boldness, inhibitory control—which can be assessed via self-report—also correlates with performance in a variety of contexts. Miyake and Friedman (2012) showed, for example, that variations in general executive function contribute to success on a range of different cognitive performance tasks. A 2009 study of twins by Young and colleagues showed performance on tasks indicative of general executive function correlated with scores on the aforementioned inhibition-disinhibition factor (c.f., Krueger et al., 2007), and operationalized by the presence or absence of tendencies toward antisocial behavior and substance abuse. That is, the less inhibited participants were (as evidenced by clinical problems), the poorer they performed on the cognitive tasks related to executive function.

The Constructs Have Direct Brain Referents

The second point Patrick made about the constructs of defensive reactivity and inhibitory control is that they have direct brain referents. A major physiological indicator of boldness, for example, is fear-potentiated startle, which is defined as the increase in the magnitude of the natural defensive startle reflex (generally to a loud noise) that occurs when a person is doing something or viewing something that is scary. People who score high on boldness measures exhibit a reduced fear-potentiated startle, which indicates that they are not so likely to automatically mobilize their defenses in the face of a threat—a characteristic that may be valuable in contexts involving stress or uncertainty.

Similarly, there are various established neurophysiological indicators of disinhibition. One of these is the P300 response, which occurs in response to certain types of stimuli during tasks when a subject is asked to respond selectively to certain stimuli within a series—that is, to respond to some and not to others. The P300 has been studied since the 1980s as an indicator of alcohol problems, Patrick said, and more recently it has been shown to be an indicator of the inhibition-disinhibition dimension that undergirds impulse problems more broadly. Another brain activity-based indicator of inhibition-disinhibition is error-related negativity, a

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

negative deflection in brain activity that occurs when the subject realizes a performance error has been made. Highly disinhibited individuals have smaller responses of this sort, which suggests that they may not be as aware of errors as they occur—a factor that likely contributes to repetition of mistakes.

Patrick’s main message in describing this research is that it is possible to investigate—and in the process, clarify—the nature of these individual difference constructs through use of indicators that are purely neuro-physiological or by using physiological indicators together with indicators from other domains, such as self-reports or behavioral responses.

The Constructs Can Be Sharpened with Psychoneurometrics

The third reason that constructs of these types are interesting and valuable, Patrick said, is because they have one foot in the domain of psychometrics, the field of psychological measurements, and the other in the domain of neurobiology. This makes it possible to work back and forth between the two domains to sharpen the operationalizations of the constructs and to gain greater insight into their structures and relationships.

In closing, Patrick described a general research strategy that could be useful to achieve this sharpening of individual difference constructs, consisting of the following steps: (1) identify replicable neurophysiological indicators of psychometric measures of target constructs (e.g., disinhibition and trait fear measures); (2) evaluate the covariation among the neurophysiological indicators, that is, identify coherent neurophysiologi-cal factors; and (3) revise the psychometric measures and trait conceptions to cohere better with the neurophysiological factors. “The construct and the way we think about it are movable, as a function of what we learn about the convergence of the physiological indicators,” he explained. “This allows for psychological trait conceptions to be reshaped by physiological data. Then the revisions to the psychological trait conceptions in turn can help to reshape [one’s] conceptions of performance capacities to better accommodate physiological data.” The process can be carried out iteratively, sharpening both the psychometric measures and the corresponding physiological indicators.

WORKING MEMORY CAPACITY AND EXECUTIVE ATTENTION

While Christopher Patrick, the first speaker of the Emerging Constructs and Theory panel, focused on the use of neurobiology in measuring and refining constructs, Michael Kane focused on constructs derived from psychological theory. Kane, a professor of psychology at the University of North Carolina at Greensboro, described two such constructs—

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

working memory capacity and executive attention—that are measurable and predictive of a number of outcomes relevant to the military.

Working Memory Capacity

The construct of working memory capacity, Kane said, is derived from basic cognitive theory and, in particular, from Baddeley’s theory of working memory as a complex system that has several storage structures that hold specific types of short-term memories (the phonological loop for the sounds of language, the visuospatial sketchpad for visual and spatial information, and the episodic buffer for various other short-term memories) combined with a “central executive” that controls where attention is focused and coordinates different cognitive processes. The short-term memory structures are closely associated with the corresponding structures for long-term or secondary memory, which consequently have implications for performance. Figure 3-2 illustrates Baddeley’s model of working memory.

“The theory is very functionally based,” Kane said. “The idea is that working memory evolved for a purpose, which is to help us maintain access to memory representations in the service of ongoing cognitive activities, like comprehending language or solving multistep problems.”

image

FIGURE 3-2 Baddeley model of working memory.
NOTE: LTM = long-term memory.
SOURCE: Reprinted from Baddeley, A.D. (2000). The episodic buffer: A new component of working memory? Trends of Cognitive Science, 4(11):417-423. With permission from Elsevier.

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

According to Kane, the most compelling evidence for the functional importance of working memory capacity has come from research into individual differences.

For example, research by Daneman and Carpenter (1980) demonstrated that language comprehension capabilities were strongly predicted by students’ working memory span scores. A decade later, broad working memory capacity was shown to be an almost perfect predictor of Air Force recruits’ general reasoning capabilities (Kyllonen and Christal, 1990).

“Most of the modern research on working memory capacity uses some version of a working memory span task,” Kane continued. These are variations on the traditional short-term memory span tasks, which ask subjects to recall item lists in serial order, with the added feature that the items to be remembered are presented alternatively with a secondary processing task, such as judging sentences or verifying equations. The distinguishing feature of the working memory capacity test is that the subjects must “maintain ready access to the goal-relevant information—the memory items—in the face of massive proactive interference from prior trials and attention shifts away from those memoranda as they shift to the processing tasks.”

Executive Attention

Not surprisingly, there is a great deal of overlap between measures of working memory capacity and measures of short-term memory and various cognitive abilities. For example, a span task that involves equations will leverage mathematical ability in addition to working memory capacity, and a test that involves mental rotations will reflect spatial ability as well as working memory. By using a series of different types of tests, along with latent-variable analyses, it is possible to tease out working memory capacity from other factors, Kane explained.

In one key example of this type of study, Engle and colleagues (1999) looked at the relationships between memory and reasoning by having their subjects complete three verbal measures of working memory capacity, three span measures of short-term memory capacity, and two nonverbal measures of general fluid intelligence. In this way they could assess commonalities between working memory and short-term memory as well as distinctions between the two. They discovered that memory storage in itself was not correlated with general intelligence. “Rather, it was what working memory did independently, over and above the memory storage demands, that was the strongest predictor of general fluid intelligence.” From the point of view of Baddeley’s model, it was not the storage system that was key to general intelligence but rather the “attentional executive capability”—the attention-directing part of the system that came to the

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

fore in the memory tests involving multiple tasks—that was most closely associated with fluid intelligence. Kane referred to this capacity as executive attention.

Kane went on to describe several studies that demonstrate the roles that executive attention plays, including its relationship to working memory. In one study, he asked subjects to learn and then recall three lists of words, all from the same category, such as “animals” (see Figure 3-3). In some cases they were required to divide their attention by tapping a novel finger sequence on a keyboard over and over again, either while they studied each list (encoding load) or while they tried to recall each list (retrieval load) (Kane and Engle, 2000). Part a of Figure 3-3 shows the results for subjects whose attention was not divided. Subjects with high-working memory capacity (high span) and subjects with low-working memory capacity (low span) recalled approximately the same number of words on the first list, but a clear difference emerged on recall of words on the second and third lists. Both the high span and low span subjects recalled fewer words from the second list and even fewer from the third, but the drop-off was much more dramatic among the low span subjects. The explanation, Kane said, is that those with higher working memory capacity were better able to deal with the “interference” of having to memorize and retain the earlier groups of words.

Interestingly, when the high span subjects were asked to memorize or recall the lists of words while having to divide their attention, their performance (shown in part b of Figure 3-3) dropped to the level of the low span subjects, when not required to divide their attention (shown in part a of Figure 3-3). “Essentially,” Kane said, “dividing attention turns high-working memory subjects into functional low-working memory subjects.” Thus the test is not just assessing memory. “It is attention that really seems to matter,” Kane observed.

This difference in ability to focus attention between subjects with high or low working memory was reinforced in a second study performed by Kane and colleagues (2001) that examined something quite different from memory. In this case high- and low-working memory subjects were tested on how quickly they could move their eyes in the proper direction after a stimulus. It is a standard assessment of executive control referred to as the antisaccade/prosaccade task. (A saccade is a quick movement of the eye.)

“In the prosaccade task,” Kane explained, “you stare at a computer screen, wait for a flash on one side of the screen, and look at it. Right there, there is going to be a letter that you have to identify. This is easy. The flash pulls attention toward the cue.” The antisaccade version is more difficult: When the light flashes, the subject is instructed to direct his or her vision to the opposite side of the screen to see the target letter. Because this antisaccade movement goes against natural tendencies, everyone is

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

image

FIGURE 3-3 Working memory capacity and executive attention in a word recall task.
SOURCE: Adapted from Kane and Engle (2000).

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

slower at this task than at the prosaccade task. The difference in response times between the prosaccade and antisaccade tasks gives a measure of executive control—in particular, how well a subject can control his or her attention.

In their study, Kane and his colleagues found that there was no difference in how quickly the low-working memory capacity (low span) subjects and the high-working memory capacity (high span) subjects responded in the prosaccade task. However, there was a sharp difference between the two on the antisaccade task. Both groups were significantly slower than on the prosaccade task, but the response time was much slower for the low-working memory (low span) subjects (see Figure 3-4).

To explore the phenomenon in greater detail, Kane’s group gave the subjects hundreds of trials on the antisaccade task and tracked their eye movements to see where they were looking—and, in particular, to see how often they first looked in the wrong direction, reflexively looking at the flash instead of in the opposite direction. The low-working memory subjects consistently looked in the wrong direction more often than the high-working memory subjects, even after many, many chances to practice and improve (Kane et al., 2001).

image

FIGURE 3-4 Working memory capacity and executive attention in prosaccade and antisaccade tasks.
NOTE: ms = milliseconds.
SOURCE: Reprinted with permission from Kane, M.J., M.K. Bleckley, A.R.A. Conway, and R.W. Engel. (2001). A controlled-attention view of working memory capacity. Journal of Experimental Psychology: General, 130(2):169-183.

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

“Remember, there is nothing to memorize in the antisaccade task,” Kane said. “There is no word list, it is hardly about memory at all. The only thing to remember is ‘look away.’ High-working memory subjects can do that better.” Thus the experiment reveals individual differences in the degree of executive control of behavior. High-working memory subjects have greater executive attention and control.

This has some very practical implications, Kane said. For example, the relationship between working memory and restraint over powerful responses has also been seen in both professional police officers (Kleider et al., 2010) and hockey players (Furley and Memmert, 2012). In laboratory-task simulations of real-world scenarios, those members of both groups who had higher working memory capacity made better decisions about whether to shoot or not shoot.

In general, Kane said, research has shown that variation in working memory corresponds to variation in a wide variety of attention-control tasks, including those involving the restraint of some habitual or prepotent response and also those involved in preventing oneself from being distracted by irrelevant stimuli in the environment. One of the more interesting correlations is with multitasking. Kane described a forthcoming study (Redick et al., unpublished) that found multitasking to be a single latent construct; that is, subjects who tested high on one measure of multitasking also tested high on different measures of multitasking, indicating that there is a single underlying ability related to multitasking performance in a variety of areas. Both working memory and attention control were related to multitasking ability, with people who scored better on tests of working memory and attention control also doing better on multitasking tests.

Mind Wandering

In his discussions of attention control, Kane noted that he had focused mainly on subjects who were dealing with external distractors or with controlling overt behavior. But, he said, “the control of attention also works internally, regulating the flow of thought from moment to moment. When that regulation fails, we experience mind wandering, or task-unrelated thinking.” So he and his colleagues have been studying mind wandering and how it correlates with working memory capacity and executive attention.

Laboratory studies have shown the tendency for mind wandering to be a stable characteristic (McVay and Kane, 2012; McVay et al., 2009). That is, the subjects who experience more mind wandering while doing one task will also experience more mind wandering while doing another task.

In one study of working memory and mind wandering, Kane and colleagues (2007) first tested a group of students on various working

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

memory tests, then they provided each student with a Palm Pilot personal digital assistant to be kept with the student at all times for one week. The Palm Pilot was programmed to beep eight times a day, at which point the students were asked to report on their conscious experiences and their ongoing activities. In particular, at each beep the students were first asked what they were thinking about at the moment of the beep—were they thinking about what they were doing, or had their minds wandered to something else?

The students were also asked to rate their mood and activities at the time of the beep on a variety of measures. Did they like what they were doing? Was it important? Was it stressful? Each measure was rated on a scale from one to seven. Kane was most interested in three particular ratings that were designed to reflect the cognitive demand of what they were doing: Were they trying to concentrate on what they were doing? Did the activity they were engaged in require a lot of effort? And was the activity challenging? “The idea is that we can get through a lot of life on autopilot,” he said. “We don’t need to bear down and regulate thought or behavior, except in some circumstances, presumably those characterized by these terms” (i.e., circumstances that are rewarding, important, stressful, or challenging).

When he compared the relationship between mind wandering and how people rated their activities, Kane found a significant negative correlation between working memory capacity and mind wandering—but only on those activities that were rated as challenging or requiring effort or concentration. “As subjects reported trying to concentrate more than usual, being more challenged than usual, or doing tasks that were more effortful than usual, the lower working memory subjects mind-wandered … more often than did higher working memory subjects.” None of the other contexts—which concerned such things as the subjects’ emotions at the time or interest in or importance of the activity—exhibited any interactions with working memory capacity (Kane et al., 2007). It would seem, then, that a higher tendency to mind-wander during challenging tasks indicates lower executive attention. Subjects with high-working memory capacity generally have greater ability to control their attention when they need or choose to. Thus, in situations requiring greater effort or concentration, they were better able to keep their minds from wandering than those subjects with less attention control.

Just as Kane studied how working memory capacity and executive attention interact on a variety of tasks, he also carried out a number of laboratory studies to examine the interplay of working memory and mind wandering to explain various types of performance. In one study, for example, he first tested his subjects on working memory capacity and then had them carry out a 40-minute go/no-go task in which they pressed

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

a button in response to a stimulus over and over again until, occasionally, they were prompted to stop in response to a different stimulus (McVay and Kane, 2009, 2012).

After about 60 percent of the no-go tasks, the subjects were shown a question on a computer screen that asked what they had just been thinking about: the task or a variety of off-task options. The frequency with which one of the options that was not about the task was selected provided a measure of mind wandering.

Also, by measuring the reaction time for each button-push, Kane could observe the consistency of the reaction time over the course of the task. The high-working memory subjects were much more consistent in their reaction times than the low-working memory subjects, who went from “really fast to really slow, all over the map” (see Figure 3-5 for an example of reaction time patterns of two randomly selected high-working memory subjects and two randomly selected low-working memory subjects).

The question then became whether the greater variation in response times among the low-working memory subjects was due to their greater propensity to mind-wander. Kane found that, while the tendency to mind-wander did explain a significant amount of the greater variation in response times, and much of this variation was shared with working memory capacity, working memory capacity also had an effect that was independent of the tendency to mind-wander (McVay and Kane, 2009, 2012). The lesson, he said, is that it is possible to learn more about what subjects are doing and why they are performing well versus poorly by looking at both measures—working memory capacity and tendency to mind-wander—rather than just one.

Similarly, Kane has studied the effects of working memory and mind wandering on reading comprehension. Previous studies had shown that people with greater working memory capacity tend to have better reading comprehension, but the natural question was: How much of that difference is due to differences in the tendency to mind-wander? Kane’s studies found that mind-wandering tendencies actually explain somewhat more about reading comprehension than working memory does, but, again, both play a role (McVay and Kane, 2012).

In a more recent, not yet published study, Kane and colleagues have done a similar analysis of working memory capacity, mind-wandering tendencies, and schizotypy, which is a “complex personality structure that confers risk for developing schizophrenia and related psychotic disorders in adulthood” (Kane et al., unpublished). Schizotypy is characterized, Kane explained, by such things as “magical ideation, weird perceptual experiences, difficulty understanding things and being understood by others, and feelings of paranoia and suspiciousness.” Kane’s studies have shown that mind-wandering tendencies have a significant positive cor-

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

image

FIGURE 3-5 High- versus low-working memory capacity in go/no-go tasks.
SOURCE: McVay and Kane (2012).

relation with schizotypy. And while working memory capacity also has a significant correlation with schizotypy (negative), it is much weaker. Looking more closely at working memory capacity, Kane found that the pure memory aspect of working memory capacity has no correlation with schizotypy at all; the correlation is completely with the attention-control aspect of working memory (that is, with the variance shared between working memory capacity tasks and lower level attention-control tasks, like the antisaccade task).

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

To conclude, Kane reiterated the major message of his presentation: “Cognitive psychology can now measure—and is now measuring—theoretically derived, theoretically tractable, and practically important constructs,” he said. Working memory capacity and executive attention, which reflect the control of behavior, perception, memory, and thought, are two such constructs, he concluded, and they are predictive of a variety of military-relevant outcomes.

THE AGENTIC SELF: ACTION-CONTROL BELIEFS

In his presentation, Todd Little, professor of psychology and director of the Center for Research Methods and Data Analysis at the University of Kansas in Lawrence, described the action-control model for understanding individuals’ beliefs about their own agency, or ability to perform successfully.

Referring to Fred Oswald’s presentation, which included a standard theoretical model of core self-evaluation that includes the traits of self-efficacy and locus of control (see Chapter 2), Little explained that his goal was to present an alternative model for thinking about how to measure those kinds of self-related beliefs and expectations about how the world works. From his perspective, he said, both self-efficacy and locus of control are ill-defined concepts, both theoretically and, particularly, operationally. By contrast, action-control theory offers concepts that are easy to operationalize (Little and Wanner, 1997).

To illustrate the difference between the two approaches, Little referred to the “little engine that could” from Watty Piper’s classic children’s book (1930). The engine’s famous line is, “I think I can, I think I can.” But, Little said, the line really should have been, “I know how one can, I know what I can; therefore, I think I can.” In other words, the engine’s belief is not a simple matter of believing it can do something, but instead it knows how to go about accomplishing the task, and it knows that it has the ability to do that particular thing, so it believes it can accomplish the task. According to Little, that is the action-control model in a nutshell.

Personal Agency

His work is grounded in an organismic perspective, Little said, so he treats most behaviors as volitional and goal-directed. The theory is aimed at understanding actions, and actions are taken as being purposeful, planned, and self-initiated, with a particular goal that the person is trying to achieve (Hawley and Little, 2002; Little, 1998; Little et al., 2006).

In particular, the theory sees individuals as agentic—in conscious control of their behaviors, working toward particular ends. “As agents, we

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

act on our needs and goals,” Little said. “We have our intentions, and we interpret and evaluate our actions and their consequences.” From watching their own actions, people develop beliefs about their own capabilities. This is the self-regulatory feature of personal agency, he said, “knowing what it takes and whether I’ve got it.”

The Action-Control Model

To illustrate the structure of actions according to his model, Little displayed a figure showing the relationship between the agent, means, and goal (see Figure 3-6). In essence, there is an agent who performs a certain action or means to attain a specific goal, and there are links between each of these constituent parts. The picture is slightly different for an individual thinking about his or her own actions than for an individual thinking about the actions of another.

Agency beliefs are the link between an individual agent and the means that are available to that individual—in the context of the pursued goal. The beliefs are, in essence, an answer to the question “Do I have what it takes?” By contrast, the link between the individual agent and the goal, the control expectancy, is the answer to the question “Can I do it?”

image

FIGURE 3-6 Action-control beliefs.
NOTE: Means include effort, ability, looks, personality, luck, teachers, parents, peers, etc.
SOURCE: Adapted from Hawley, P.H., and T.D. Little. (2002). Evolutionary and developmental perspectives on the agentic self. In D. Cervone and W. Mischel (Eds.), Advances in Personality Science. New York: Guilford Press. Copyright Guilford Press. Reprinted with permission of Guilford Press.

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

The theoretical definition that Little provided for “agency beliefs” is “Agent (A) has means (M) that is relevant to end (E).” The operational definition is “the person’s belief that he or she personally has access to, can use, can implement, or possesses a specific means that is relevant to achieve the outcome.” This is where action-control beliefs have an advantage over concepts like efficacy, Little said. “It is a very easy system to operationalize.” It is also straightforward to develop specific items to measure agency beliefs, he said. Once the specific context is known, one determines the kinds of means that would be useful in the particular context—being as inclusive as possible—and develops items to assess a person’s beliefs about those particular means. Examples of the sorts of items he has used in his own assessments include, “I can try hard,” “I am smart enough to do it,” “I am unlucky at it,” and “I can get others to help me” (see Little and Wanner, 1997).

It is important to note that these are all intrapersonal beliefs—effort, ability, and even luck. “When it comes to the side of agency, luck is an intrapersonal thing,” Little said. “It is something that I possess and own, just like my effort and my ability.” However, he noted, when an individual evaluates others, luck tends to be interpreted as a factor external to the individual.

Little has studied the relationship between these beliefs and an individual’s actual performance in various tasks. He emphasized the striking result that quite often there is no added value to knowing an individual’s control expectancy belief—the degree to which an individual believes he or she can achieve the specific goal—versus simply knowing an individual’s abilities (Little et al., 1995). “If I know what you possess, in terms of your effort and ability, those will always outperform just whether or not you think you will get it done. The ‘I think I can, I think I can, I think I can’ doesn’t buy us anything in terms of predictive ability.”

Similarly, there is generally relatively little correlation between an individual’s score on a self-efficacy measure and actual performance (Multon et al., 1991). The correlations between traditional measures of self-efficacy and performance are generally around 0.3, Little said. By contrast, in certain contexts he has seen correlations between action-control beliefs and performance that are greater than 0.7. The key, he said, is refining the measures to be very specific about the goal structure and the specific means that can be used to achieve the goal. The traditional measures of self-efficacy are generally at too high a level of aggregation to be very predictive.

Finally, Little described the causality beliefs, or means–ends beliefs in his model. “These are really contingent belief operations: my understanding of how the world works; my understanding of, will effort get me to this goal, will ability get me to this goal, will my looks, my personality,

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

teachers, parents, whatever, get me to that goal; and to what degree are they important for achieving that particular goal?” These beliefs develop in a variety of ways, he noted, and they are relatively trainable. “In the school setting, we teach kids what it takes to do well in school. In a military context, we can teach people what it takes to do well in this assignment.” Once a person knows what the assignment is, he or she can look for a match between agency beliefs and the causality beliefs.

The important feature here, Little said, is that it is generally possible to provide multiple means to an end, so if a person does not have the tools to do it one way, it can be done another way. For example, a person might think of it this way: “I am not really the sharpest pencil in the box, but I will work … and I will just go and go and go and go. I can get to the same goal that you can because you are smart. I can get there by effort.”

There are various ways that action-control beliefs are acquired, Little said. Some of them come from direct experiences with success and failure, while others are taught by parents, teachers, peers, and others (Little, 1998). Feedback on one’s own performance helps refine the various beliefs, as do vicarious observations: watching and seeing how others do things, in person or virtually.

One important influence on action-control beliefs is social comparisons: a person learning how he or she measures up to others. But the comparisons need to be accurate. In the United States, Little said, “when we ask kids about their agency beliefs for doing well in school, everybody believes they are above average because the teachers keep telling them they are great and wonderful.” By contrast, in Germany honest feedback is a “valued cultural aspect,” and agency beliefs are much more closely correlated with performance scores (Little et al., 1995; Oettingen et al., 1994).

According to Little, another important influence on action-control beliefs comes from symbolic actions—that is, by thinking through and rehearsing what it would take to carry out a certain task, without actually doing it (Boesch, 1991; Brandstádter, 1998). Using symbolic actions, people can develop action-control beliefs in a completely new context, one with which they have no previous experience.

Little closed by describing some of the differences that have been reported in the literature between agentic and nonagentic people (Hawley and Little, 2002; Little et al., 2006); that is, between those who tend to believe in their ability to control things in their lives and those who do not (see Box 3-1). Nonagentic people have little sense of personal empowerment, feel helpless when they are challenged, and tend to accept failures. They have low aspirations and perform poorly on most tasks. Agentic individuals are just the opposite. They have a greater sense of personal empowerment, they persist in the face of obstacles, and they learn from failures. They have high aspirations and perform well on tasks.

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

BOX 3-1
Nonagentic Versus Agentic Profiles

Nonagentic Profile

  • Has low aspirations
  • Feels helpless when challenged
  • Is hindered by problem-solving blinders
  • Performs poorly
  • Accepts failures
  • Has greater ill-being
  • Has little sense of personal empowerment

Agentic Profile

  • Has high aspirations
  • Persists in the face of obstacles
  • Sees more and varied options
  • Performs well
  • Learns from failures
  • Has greater well-being
  • Has a greater sense of personal empowerment

SOURCE: Little presentation.

DISCUSSION

Following the three panel presentations, committee members, the presenters, and other participants engaged in a roundtable discussion that included specific questions for the panelists as well as thought-provoking brainstorming on the future of measurements and assessments. The following section captures some of the more salient ideas expressed during the group discussion.

The first several questions to panelists addressed the distribution of various traits across the population and when certain traits might be more desirable than others. Participants debated in detail the importance of the context of the task in judging desirable traits. For example, when might it be better to be more likely to be distracted from a task (for example, if an emergency happens outside your area of attention) versus being highly focused on the task? In response, Patrick clarified the idea of fear versus boldness as a normal individual difference continuum: “To have a lot of fear is non-normative, but potentially adaptive in certain contexts,” he said. Likewise, he continued, to have very limited fear is non-normative but potentially adaptive in other contexts. That is, the existence of individual differences in fearfulness reflects an adaptive trade-off between exploratory types of behaviors and tendencies toward defensive withdrawal. Little agreed with this point, noting that it is not necessarily always the leader who survives; “followers survive, too, when they hook up with the right leaders” (see Hawley 1999; Hawley and Little, 2002; see also Buss and Hawley, 2010, for an edited volume on the evolution of individual differences).

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

The discussion of the many potential factors contributing to success at a specific task led Little to emphasize that any assessment of potential performance is a “multivariate problem.” “Working memory is clearly an important factor,” Little said, “but boldness is there, too. How does boldness work with working memory, under which contexts do we see an optimal one? We might find that boldness is selected for in some contexts, whereas memory is in others.” Several of the participants suggested that creating performance context, affective context, and other relevant contexts may be an interesting approach to assess the relationships and interactions of many different constructs in different situations. Later, Patrick took the discussion further by suggesting that not only should some things be assessed together but other things may need to be dissociated. Assessments may thus need to strategically separate constructs, to determine how they respond as indicators are manipulated. For example, Patrick added, “you are still holding onto something that is, say, callousness, but you are moving it away from disinhibition, so aggression is no longer an indicator.” Further proof of the need to both combine and disentangle constructs, according to Patrick, is the existence of suppressor relationships between correlated constructs: predictions improve as clear discriminate relationships emerge.

Stephen Stark, a committee member, asked Little about the context for the beliefs and traits assessed by his model. “There have been studies that suggest that contextualizing personality items seems to increase the predictive validity of the measures,” Stark noted, adding that one of the common concerns with generalized contextualization is that contextualizing the items too much may ultimately require different types of items for different situations. From a practical standpoint, this could be a challenge. Stark then asked Little to explain context with regard to his measures and the level of specificity necessary to achieve high validities relative to the broader, higher-level self-efficacy constructs and locus of control. In particular, Stark said that there are different occupational specialties in the military and that recent work suggests that different personality profiles are more or less predictive of performance across different jobs. He asked how specific the items would have to be in Little’s tool to be useful for at least families of jobs that are fairly similar in terms of their goals, characteristics, and environments.

Little responded that he believed his tool could be effective at the level of job families such as military occupation specialties. “I could see a level of aggregation for these beliefs, that you would still have predictive capability, but wouldn’t necessarily have to get down to specific job title.” He also thought it may be possible to have more refined tools that could determine which of two jobs in a particular job family a candidate might be best suited for. “You do the big sweep, you get your low-hanging

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

fruit first,” he said. “Then, you start doing follow-up assessments, after you can start moving people in the right direction, and then you can get to specific components.”

INTERESTS

In his presentation during the workshop’s second-day panel of individual differences and performance, James Rounds, a professor of psychology and educational psychology at the University of Illinois at Urbana-Champaign, further contributed to the workshop’s theme of emerging constructs and theory. Rounds began by suggesting that the future of individual performance prediction should revisit its history. In the 1920s and 1930s, psychologists paid a great deal of attention to personal interests in the belief that they would be predictive of many things, including how successful people would be in their careers. However, over time that attention faded as evidence seemed to indicate that interests had relatively little predictive power. However, Rounds argued that it is time to give interests a second look. Not only are interests surprisingly stable over the course of a person’s life, he said, but, when analyzed properly, they can also be used to predict performance and achievement.

Rounds then offered some background on how interests are generally approached by psychologists, describing two approaches to studying interests. One approach considers interests in terms of situations, that is, as context-specific “emotional states, curiosity, and momentary motivation” (Schraw and Lehman, 2001). This approach is associated more with educational psychology and with an experimental approach to interests. It is generally referred to as “situational interests” or sometimes “individual interests.”

The other approach, generally referred to as “dispositional interests,” considers interests more in terms of personal traits that reflect a person’s “preferences for behaviors, situations, contexts in which activities occur, and/or the outcomes associated with the preferred activities” (Rounds, 1995). In other words, interests are seen as expressions of underlying personality traits.

Generally speaking, Rounds said, researchers who study situational interests do not collaborate with those who study dispositional interests, to the extent that they might even be considered two separate scientific disciplines. They do not share data or ideas, and each group is generally unaware of the other group’s work. Thus, according to Rounds, some things have fallen between the cracks.

Some of the earliest work in the field was done by people in the dispositional interest camp. In the 1920s, Walter Bingham set in motion a program that eventually led to the development of nine different inter-

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

est inventories—sets of questions designed to identify interests in various areas. One of these inventories, developed by Edward K. Strong, Jr. (1943) is still used today. The underlying assumption in much of this work, according to Rounds, is that understanding interests would lead to a better understanding of performance.

However, in recent years the more predominant assumption has been that interests actually have relatively little to do with performance. Much of the change, Rounds said, can be traced to a paper published in 1984 by Hunter and Hunter, who reported that there was very little correlation between interests and performance—generally no more than about a 0.1 correlation. Other studies used the evidence from that paper to suggest that interests are not effective predictors of performance (for example, Barrick and Mount, 2005). The result, Rounds said, is that “you will not find anyone talking about performance and interests, period.” Furthermore, “you hardly find interests in textbooks anymore.” But Rounds believes that there is a great deal to learn from studying interests, and he offered three lines of evidence, summarized below, to support his contention.

Interests and Performance

Much of what has been written about interests, Rounds said, discusses interests as a motivational type of variable. It provides some sort of direction, it energizes a person, and it increases persistence (Nye et al., 2012).

There is also a parallel literature on person-environment fit (see, for example, Kristof-Brown et al., 2005, and Verquer et al., 2003). That literature suggests that the extent of compatibility between an individual and his or her environment can influence performance outcomes. Unfortunately, Rounds said, much of that literature ignores interests. Yet he believes that the literature on person-environment fit has the potential to show interests in a different, more compelling light.

To show why, Rounds described a meta-analysis that he and colleagues conducted on 60 studies published since 1934 (Nye et al., 2012). Of those 60 studies, 45 percent were published after the Hunter and Hunter paper appeared in 1984. The analysis included both studies that used interest scale scores and studies that used congruence indices reflecting the fit between a person’s interest profile and either the person’s job or the person’s occupational profile. The authors tested for correlations between the interest scores or congruence and several measures of performance, including task performance, organizational citizenship behavior, persistence in the workplace, and persistence and grades in an academic setting.

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

The analysis found a clear correlation between interests and performance, no matter how performance was measured. The overall correlation was 0.20—about double what Hunter and Hunter had reported nearly three decades earlier. What was most striking, however, was the overall correlation between performance and the congruence indices—about 0.36, which was very significant (Nye et al., 2012). Thus, while interests are moderate predictors of performance criteria, the correlations increase substantially when the congruence between the individual’s interests and the environment is considered.

The takeaway message, Rounds said, is that fit, particularly with regard to interests, really matters.

Interests and Career Success

A second study from Rounds’ laboratory looked at the incremental validity of vocational interests beyond personality and cognitive abilities in predicting academic achievement and career success (Su, 2012). It involved the analysis of data from a large-scale longitudinal study, Project Talent, which began in 1960. Five percent of American high school students in grades 9 through 12 participated in a full 2 days of testing, which included a comprehensive set of cognitive ability measures, 10 personality scales, and a large collection of interest items. After the original testing, the participants were surveyed 3 additional times—at 1, 5, and 11 years after their high school graduation—about their educational, occupational, and personal development.

Rounds’ graduate student performing the study, Rong Su, used regression analysis to determine the relative importance of interests, personality, and ability for various types of achievements, such as college degrees attained and income. Su found that ability was clearly important and that, indeed, it was the most important predictor of every achievement except income. Personality also played a role, but interests played a larger role than personality in every area, and interests were by far the biggest predictor of income (Su, 2012). The results are shown in Figure 3-7.

It is natural to assume, Rounds noted, that the correlation between interests and income can be explained by people selecting jobs in better-paying fields, such as science or business, but the data indicate it is not that simple. “The predictive power of interests for income does not just come from its influence on career choices,” he said, “but also comes from its influence on advancement in a career.”

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

image

FIGURE 3-7 Relative importance of interests, personality, and ability for educational and career success.
SOURCE: Adapted from Su (2012, p. 126).

The Stability of Interests

Interests have always been considered fairly stable, Rounds said, but no one had looked carefully at exactly when they become stable and how they might change, for example, from adolescence to young adulthood. To investigate the stability of interests, Rounds and his colleagues carried out a meta-analysis of 66 studies (Low et al., 2005). They found there were few studies that examined the stability of interests beyond age 40. However, they were able to access a significant amount of data concerning the stability of vocational interests from ages 12 to 40.

The analysis found that there was a big jump in stability of interests at about age 18, at which point they stabilize and stay about the same through age 40 (Low et al., 2005). This finding is important, Rounds said, because it indicates interests stabilize much earlier in life than had previously been thought. It is also useful information for predictive purposes, since one can reasonably assume that whatever measures of interest are obtained for subjects after age 18 are likely to remain fairly stable.

One surprising result of the study, Rounds said, concerned the stability of interests versus the stability of personality. Most researchers tend to think of things like interests and values as deriving from basic personality traits, and so it would seem that interests should be less stable than per-

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

sonality traits. However, the data indicated that interests are substantially more stable than personality traits (Low et al., 2005).

REFERENCES

Baddeley, A.D. (2000). The episodic buffer: A new component of working memory? Trends of Cognitive Science, 4(11):417-423.

Barrick, M.B., and M.K. Mount. (2005). Yes, personality matters: Moving on to more important matters. Human Performance, 18(4):359-372.

Boesch, E.E. (1991). Symbolic Action Theory and Cultural Psychology. Berlin, Germany: Springer-Verlag.

Brandstádter, J. (1998). Action perspectives on human development. In W. Damon (Series Ed.) and R. Lerner (Vol. Ed.), Handbook of Child Psychology: Vol. 1. Theoretical Models of Human Development (pp. 1029-1144). New York: Wiley.

Buss, D.M., and P.H. Hawley (Eds.). (2010). The Evolution of Personality and Individual Differences. New York: Oxford University Press.

Daneman, M., and P.A. Carpenter. (1980). Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior, 19(4):450-466.

Dvorak-Bertscha, J.D., J.J. Curtin, T.J. Rubinstein, and J.P. Newman. (2009). Psychopathic traits moderate the interaction between cognitive and affective processing. Psycho-physiology, 46(5):913-921.

Engle, R.W., S.W. Tuholski, J.E. Laughlin, and A.R.A. Conway. (1999). Working memory, short-term memory, and general fluid intelligence: A latent variable model approach. Journal of Experimental Psychology: General, 128(3):309-331.

Furley, P.A., and D. Memmert. (2012). Working memory capacity as controlled attention in tactical decision making. Journal of Sport and Exercise Psychology, 34(3):322-344.

Hawley, P.H. (1999). The ontogenesis of social dominance: A strategy-based evolutionary perspective. Developmental Review, 19(1):97-132.

Hawley, P.H., and T.D. Little. (2002). Evolutionary and developmental perspectives on the agentic self. In D. Cervone and W. Mischel (Eds.), Advances in Personality Science (vol. 1, pp. 177-195). New York: Guilford Press.

Hunter, J., and R.F. Hunter. (1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96(1):72-98.

Kane, M.J., and R.W. Engle. (2000). Working memory capacity, proactive interference, and divided attention: Limits on long-term memory retrieval. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26(2):336-358.

Kane, M.J., M.K. Bleckley, A.R.A. Conway, and R.W. Engle. (2001). A controlled-attention view of working-memory capacity. Journal of Experimental Psychology: General, 130(2):169-183.

Kane, M.J., L.H. Brown, J.C. McVay, I. Myin-Germeys, P.J. Silvia, and T.R. Kwapil. (2007). For whom the mind wanders, and when: An experience-sampling study of working memory and executive control in daily life. Psychological Science, 18(7):614-621.

Kane, M.J., T.K. Kwapil, and P.J. Silvia. (unpublished). Executive Control and Schizotypy in the Laboratory and Daily Life. National Institute of Mental Health-funded R15 project (MH 093771-01).

Kleider, H.M., D.J. Parrott, and T.Z. King. (2010). Shooting behaviour: How working memory and negative emotionality influence police officer shoot decisions. Applied Cognitive Psychology, 24(5):707-717.

Kramer, M.D., C.J. Patrick, R.F. Krueger, and M. Gasperi. (2012). Delineating physiologic defensive reactivity in the domain of self-report: Phenotypic and etiologic structure of dispositional fear. Psychological Medicine, 42(6):1305-1320.

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

Kristof-Brown, A.L., R.D. Zimmerman, and E.C. Johnson. (2005). Consequences of individuals’ fit at work: A meta-analysis of person-job, person-organization, person-group, and person-supervisor fit. Personnel Psychology, 58(2):281-342.

Krueger, R.F., K.E. Markon, C.J. Patrick, S.D. Benning, and M. Kramer. (2007). Linking antisocial behavior, substance use, and personality: An integrative quantitative model of the adult externalizing spectrum. Journal of Abnormal Psychology, 116(4):645-666.

Kyllonen, P.C., and R.E. Christal. (1990). Reasoning ability is (little more than) working-memory capacity?! Intelligence, 14(4):389-433.

Lilienfeld, S.O., I.D. Waldman, K. Landfield, A.L. Watts, S. Rubenzer, and T.R. Faschingbauer. (2012). Fearless dominance and the U.S. presidency: Implications of psychopathic personality traits for successful and unsuccessful political leadership. Journal of Personality and Social Psychology, 103(3):489-505.

Little, T.D. (1998). Sociocultural influences on the development of children’s action-control beliefs. In J. Heckhausen and C.S. Dweck (Eds.), Motivation and Self-Regulation Across the Life Span (pp. 281-315). New York: Cambridge University Press.

Little, T.D., and B. Wanner. (1997). The Multi-CAM: A Multidimensional Instrument to Assess Children’s Action-Control Motives, Beliefs, and Behaviors (Materialen aus der Bildungs-forschung, Nr. 59, ISBN #3-87985-064-x). Berlin, Germany: Max Planck Institute for Human Development.

Little, T.D., G. Oettingen, A. Stetsenko, and P.B. Baltes. (1995). Children’s action-control beliefs about school performance: How do American children compare with German and Russian children? Journal of Personality and Social Psychology, 69(4):686-700.

Little, T.D., C.R. Snyder, and M. Wehmeyer. (2006). The agentic self: On the nature and origins of personal agency across the lifespan. In D.K. Mroczek and T.D. Little (Eds.), Handbook of Personality Development (pp. 61-79). Mahwah, NJ: Lawrence Erlbaum Associates.

Low, K.S.D., M. Yoon, B.W. Roberts, and J. Rounds. (2005). The stability of vocational interests from early adolescence to middle adulthood: A quantitative review of longitudinal studies. Psychological Bulletin, 131(5):713-737.

Lykken, D.T. (1995). The Antisocial Personalities. Hillsdale, NJ: Lawrence Erlbaum Associates.

McVay, J.C., and M.J. Kane. (2009). Conducting the train of thought: Working memory capacity, goal neglect, and mind wandering in an executive-control task. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(1):196-204.

McVay, J.C., and M.J. Kane. (2012). Drifting from slow to “d’oh!”: Working memory capacity and mind wandering predict extreme reaction times and executive control errors. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38(3):525-549.

McVay, J.C., M.J. Kane, and T.R. Kwapil. (2009). Tracking the train of thought from the laboratory into everyday life: An experience-sampling study of mind wandering across controlled and ecological contexts. Psychonomic Bulletin and Review, 16(5): 857-863.

Miller, M.W., D.S. Vogt, S.L. Mozley, D.G. Kaloupek, and T.M. Keane. (2006). PTSD and substance-related problems: The mediating roles of disconstraint and negative emotionality. Journal of Abnormal Psychology, 115(2):369-379.

Miyake, A., and N.P. Friedman. (2012). The nature and organization of individual differences in executive functions: Four general conclusions. Current Directions in Psychological Science, 21(1):8-14.

Multon, K.D., S.D. Brown, and R.W. Lent. (1991). Relation of self-efficacy beliefs to academic outcomes: A meta-analytic investigation. Journal of Counseling Psychology, 38(1):30-38.

Nye, C.D., R. Su, J. Rounds, and F. Drasgow. (2012). Vocational interests and performance: A quantitative summary of over 60 years of research. Perspectives on Psychological Science, 7(4):384-403.

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×

Oettingen, G., T.D. Little, U. Lindenberger, and P.B. Baltes. (1994). Causality, agency, and control beliefs in East versus West Berlin children: A natural experiment on the role of context. Journal of Personality and Social Psychology, 66(3):579-595.

Piper, W. (1930). The Little Engine That Could. New York: Platt and Munk.

Redick, T., Z. Shipstead, M. Meier, J. Evans, K. Hicks, N. Unsworth, M. Kane, Z. Hambrick, and R. Engle. (unpublished). Understanding the Role of Working Memory Capacity in Complex Task Performance to Improve Sailor and Marine Selection, Classification, and Training. Office of Naval Research project.

Rounds, J.B. (1995). Vocational interests: Evaluation of structural hypotheses. In D. Lubinski and R.V. Dawis (Eds.), Assessing Individual Differences in Human Behavior: New Concepts, Methods, and Findings (pp. 177-232). Palo Alto, CA: Consulting Psychologists Press.

Schraw, G., and S. Lehman. (2001). Situational interest: A review of the literature and directions for future research. Educational Psychology Review, 13(1):23-52.

Strong, E.K., Jr. (1943). Vocational Interests of Men and Women. Palo Alto, CA: Stanford University Press.

Su, R. (2012). The Power of Vocational Interests and Interest Congruence in Predicting Career Success. Dissertation, University of Illinois at Urbana-Champaign. Available: https://www.ideals.illinois.edu/bitstream/handle/2142/34329/Su_Rong.pdf?sequence=1 [June 2013].

Verquer, M.L., T.A. Beehr, and S.H. Wagner. (2003). A meta-analysis of relations between person-organization fit and work attitudes. Journal of Vocational Behavior, 63(3):473-489.

Young, S.E., N.P. Friedman, A. Miyake, E.G. Willcutt, R.P. Corley, B.C. Haberstick, and J.K. Hewitt. (2009). Behavioral disinhibition: Liability for externalizing spectrum disorders and its genetic and environmental relation to response inhibition across adolescence. Journal of Abnormal Psychology, 118(1):117-130.

Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 21
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 22
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 23
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 24
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 25
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 26
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 27
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 28
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 29
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 30
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 31
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 32
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 33
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 34
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 35
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 36
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 37
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 38
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 39
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 40
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 41
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 42
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 43
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 44
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 45
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 46
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 47
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 48
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 49
Suggested Citation:"3 New Constructs for Assessing Individuals." National Research Council. 2013. New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18427.
×
Page 50
Next: 4 Emerging Understandings of Group-Related Characteristics »
New Directions in Assessing Performance Potential of Individuals and Groups: Workshop Summary Get This Book
×
Buy Paperback | $44.00 Buy Ebook | $35.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

As an all-volunteer service accepting applications from nearly 400,000 potential recruits annually from across the U.S. population, the U.S. military must accurately and efficiently assess the individual capability of each recruit for the purposes of selection, job classification, and unit assignment. New Directions for Assessing Performance Potential of Individuals and Groups is the summary of a workshop held April 3-4, 2013 to examine the future of military entrance assessments. This workshop was a part of the first phase of a larger study that will investigate cutting-edge research into the measurement of both individual capabilities and group composition in order to identify future research directions that may lead to improved assessment and selection of enlisted personnel for the U.S. Army. The workshop brought together scientists from a variety of relevant areas to focus on cognitive and noncognitive attributes that can be used in the initial testing and assignment of enlisted personnel. This report discusses the evolving goals of candidate testing, emerging constructs and theory, and ethical implications of testing methods.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!