7

Human Sciences

INTRODUCTION

The Panel on Human Factors Sciences at the Army Research Laboratory conducted its review of selected research and development (R&D) projects of the ARL Human Sciences Campaign and the Assessment and Analysis Campaign at the Aberdeen Proving Ground, Maryland, on July 14-16, 2015. The Human Sciences project areas reviewed were these:

• Humans in Multiagent Systems. These efforts aim to achieve critical technological breakthroughs needed for future Army multiagent, mixed-agent teams to effectively merge human and agent capabilities for collaborative decision making and enhanced team performance in dynamic, complex environments. The challenges for human sciences R&D are soldier workload, situation awareness, trust, influence, and cultural cognition.
• Real World Behavior. The objectives of the R&D in this area are to enable the collection, analysis, and interpretation of human behavioral data within dynamic, complex, natural environments. ARL conducts R&D in two areas (1) real-world complexity in human science experimentation and (2) assessing human behavior in the real world. A key focus of this work is the development of novel technology and methodologies and to collect and analyze these data in real world conditions.
• Toward Human Variability. The goals of this R&D are to enable high-resolution, moment-to-moment predictions of an individual soldier’s internal and external behavior and performance and the ways in which soldiers interact dynamically in mixed-agent team and social settings in both training and operational environments. Human variability R&D is conducted in two areas: (1) multifaceted soldier characterization to develop a comprehensive understanding of the factors influencing human variability and (2) brain structure function coupling to create a multiscale understanding of the relationship between the brain’s physical structure, its dynamic neurophysiological functioning, and human behavior.

The Panel on Human Factors Science also reviewed a component of the Assessment and Analysis Campaign portfolio on assessing mission capabilities of systems. The goal of this area, more or less, is engineering and acquisition decision support for current and future army human systems. Efforts are conducted in two areas, the first of which is human factors, which focuses on integrated human factors engineering (HFE) and system engineering (SE) assessments and analytic techniques to predict human, system, and mission capabilities early in the acquisition cycle. In this first area, HFE applications and tools are developed and refined to lower acquisition costs and improve design. The second area, soldier survivability, is concerned with analysis and engineering to increase the survivability of platforms and soldiers operating in combat environments.

Since the Board’s 2014 assessment, ARL has restructured its portfolio of ongoing and planned research and development to align with its S&T campaign plans for 2015-2035. These plans direct strategic shifts in emphasis that could transform the current program of work motivating new directions for future research. This transformation is in its early stages, and, as might be expected, the fit of some

ongoing work into the key campaign initiatives (KCI) in the S&T campaign plans for 2015-2035 is a bit uneasy.

The increased emphasis on understanding and managing real-world complexity within the KCIs is an important and notable development. Increasingly, complex sociotechnical systems that encompass interactions among multiple agents whose behavior is governed by intangible variables (e.g., emotion and culture) will pose many tough challenges for the “Army after next.” Research in this area will likely promote increased interdisciplinary collaboration with other campaign areas. Conceptual and theoretical breakthroughs from the growing adaptive complex systems engineering literature need to be leveraged to support this work.

In general, gains were evident in publication rates and in the establishment of collaborations and partnerships with relevant peers inside and outside ARL. The ARL human sciences work environments are exceptional in terms of their unique and advanced technological capabilities to support research. ARL has continued to successfully attract clever postdoctoral researchers from a diverse set of universities and disciplines.

Overall, ARL has been proactively responsive to prior concerns and recommendations from ARLTAB to good effect. For example, the materials provided in advance of the review meeting were generally more informative, presentations were more uniform, and statistical analysis has become more rigorous. Additionally, the efforts of ARL leadership to overcome constraints on conference participation by scientists and engineers are laudable. Overall, these are outstanding accomplishments and mark a visible advance over prior years.

ARL’s vision for 2015-2040 is compelling and raises expectations for an innovative program of research designed to be responsive to the needs of the Army after next. Unfortunately, this is not yet fully evident in the portfolio currently being assessed. Certainly, the reorganization of the portfolio into KCIs is promising in this regard, and it may take some time to transform and mature the program of work and its representation to consistently align with new critical paths. At present, however, some of the groupings appear forcibly merged, with the conceptual linkages between them unclear (e.g., how does the human–robotic interaction work relate to the social cultural work?). The relevant and high-value niches for human sciences research and development within the KCIs with respect to what others have done, are doing, and where ARL could best contribute is in serious need of clarification by elaboration. ARL needs to consider surveying the external communities in academia, industry, and other government agencies to establish strategic baselines for investments in these areas.

A general challenge, noted in past assessments by the ARLTAB, persists with respect to the critical need for access to military-relevant subjects in the work. Many of the real-world research questions that ARL is dealing with urgently require more effort with respect to subject populations and the representation of mission contexts. A number of studies presented drew upon ARL researchers as subjects to a worrisome degree. As another example, researchers at the soldier performance and equipment advanced research (SPEAR) facility waited eight months to acquire and run a limited number of soldiers as subjects. ARL needs to find a workable solution to this problem that threatens to compromise the credibility and impact of important research addressing vital Army needs.

On a positive note, statistical analysis at ARL has become increasingly rigorous; however, other analytical tools (e.g., mathematical modeling [optimization] and data mining) need further consideration in the program of work.

Several questions arose with respect to gaps in the disciplinary composition of the ARL human sciences workforce. Important core competencies such as systems and simulation engineering are absent; hiring is needed to grow these competency areas. There was also an expressed challenge and concern about the scant number of technicians needed to support the increasingly high-tech research and development within the human sciences campaign.

HUMANS IN MULTIAGENT SYSTEMS

Accomplishments

ARL human sciences work in multiagent systems addresses sociotechnical network operations, focusing on distributed collaboration and decision making in complex operational environments; sociocultural competency skills needed to support, inform, and influence operations in complex environments; human–robot trust; teaming of humans and intelligent systems; and human control of multiple robots. ARL appears primed to make important contributions to this paradigm and demonstrated a sound awareness of the key trends driving the research challenges in this area (i.e., the rise of the networked organization, increasing autonomy, and the need for cultural competency in military operations). The work in the area of human–robotic interactions and trust builds on the extant baseline of substantive theoretical and empirical work dealing with trust in automation. The conceptual organization of data-to-decisions at social cognitive, information, and communication layers allows for effective linkages and human–system integration with other multiagent systems activities. Hence ARL is applying a well-integrated systems approach to the study of purposeful social systems, addressing not only structure but also the role and function of sociotechnical Army systems involving people, information, and technology. The research staff, including postdoctoral researchers and interns with recent experience, appears competent. However, from a staffing perspective, this is an area that would benefit from growing interdisciplinary collaborations as the portfolio matures. Similarly, the modeling and simulation capability to support this work is good, but an expanded toolkit will be needed to enable substantive advances in the future.

Challenges and Opportunities

The potentially valuable work in this area faces inherent challenges, but overall there does not appear to be a coherent vision for how the research holds together and cumulatively builds to push the state of the art. This may be the result of a forced merger of ongoing activities into the new campaign organization, which is still somewhat immature.

The problem area as scoped is very broad-ranging, including human–technology interactions to human–human interaction (sociocultural interactions), and it is not clear how all of the pieces fit together. For example, the distinction between the human–robot interaction (HRI) research and sociocultural research is an important one, because these groups have different pre-theoretical ideas and histories, concepts, analogies, and terminology and very different subject matter and methodologies. Although there are connections that could be made between these areas, they are generally distinct subdisciplines, and the connection between them needs to be made clear in the present organization.

The emerging work on joint operations of robots and humans is an important area of work with a large body of extant research that needs to be strategically considered as ARL evolves its niche in this area. There are many directions to be considered, including multiple HRI configurations (i.e., multiple humans with multiple robots), threats to security or integrity of robots as potentially compromised agents, and the possibility of coadaptation and coagency, where either the human or agent can take control. The progress at ARL has been unidirectional (i.e., human to robots) and needs to progress to encompass bidirectional communication—that is, humans trusting robots and robots trusting humans.

Good use is made of modeling and simulation (M&S) in this area, although there is room for some improvement by expanding the toolkit of available techniques. For example, there are instances in which a statistical or event-driven simulation is used instead of physics-based M&S. M&S has emerged as a discipline in itself, as opposed to only an enabling technology, with several universities now offering degree programs. As such, ARL needs to consider recruiting more simulationists onto its research staff.

The emphasis on experimental work by the human factors group is commendable, but good mathematical and computational models of the robots, the interacting humans, and the feedback could

save a lot of trial and error, provide insight, allow testing of different control regimes and feedback strategies, quantify sensitivity, help with identifying worst-case scenarios and sources of instability, and provide guidance to the design of experiments. There are modeling opportunities, especially of human operators, of robots, and of human collaborators from the “‘viewpoint” of the robot. The development of such models can aid understanding about how humans and robots interact by using inexpensive simulations and sensitivity studies. These studies need to be done before undertaking costly experimentation with humans and physical machines.

The challenges of HRI are inherently interdisciplinary, and drawing upon multiple contributing competencies is necessary to have successful, relevant outcomes. For example, the investigators in this area have, for the most part, backgrounds in psychology, and they could benefit from the knowledge, alternative frameworks, and approaches of experts in robot navigation, control, and modeling and simulation. Multidisciplinary approaches can interrogate issues from multiple perspectives, promote the integration of insights, and facilitate the connection of ideas in novel ways. Over the years, ARL has apparently supported a body of work on robotic control and navigation and already has many experts on staff, including mathematicians, electrical engineers, mechanical engineers, and physicists. There now appears to be a great opportunity for joint development of models and algorithms supporting this work. The description of research in multiagent systems appeared to focus on the human factors aspects of the systems and did not describe collaborative interactions with robotics researchers. Such collaborations are essential for a full understanding of the robotics features of such systems.

There is a wealth of available knowledge to be leveraged on how robots identify where they are, map their environments, plan their movement in autonomous or semiautonomous settings, use beacons and milestones, identify obstacles and hazards, fulfill commands, navigate, and follow trajectories. There is an opportunity to use this knowledge in the design of joint human–machine action. Missing out on this wealth of knowledge means that the algorithms developed by the human sciences campaign may miss out on a variety of behaviors and constraints that are well studied and well-modeled by individuals with mathematical, mechanical, electrical engineering, and computer science training.

Another area where ARL research on joint robot–human action can benefit is the significant body of work on hard fusion. Also of increasing significance is the emerging field of soft and hard fusion. Soft and hard fusion appears to be a major opportunity for developing a framework for HRI scene interpretation, because it admits a large suite of heterogeneous inputs, including heat sensors and tweets. There are numerous individuals across ARL with relevant expertise in this area, but there was scant evidence of any collaboration with these individuals.

ARL identified a focus on sociotechnical systems, including data to decisions, decision support systems, human dynamics of cybersecurity, and network team performance. Although ARL identified human dynamics of cybersecurity as a component of ARL’s focus on sociotechnical systems within its Human Sciences campaign, ARL’s Human Sciences campaign did not present anything that was germane to cyber research. Given that there exist many opportunities for adversaries to compromise robots or influence people, this would seem a critical area for human sciences research. This work may already be ongoing under a different ARL campaign, but it is important to support proactive collaboration and engagement of the human sciences in this area.

The research on understanding sociocultural influences suffered from some serious methodological design flaws and weaknesses in analysis that completely undercut its potential usefulness and impact. Chief among the problems was the studies’ reliance on a subject population that was not representative of the target population. More specifically, subjects were uniformly Christian, white males who did not remotely reflect the religious, ethnic, racial, or gender demographics of the Army personnel targeted by the study, making it difficult and misleading to draw any credible conclusions from the findings of this study. The serious problematic design of this study suggests a need for preapproval quality vetting of proposed research designs at ARL.

REAL-WORLD BEHAVIOR

Accomplishments

The collection and analysis of human behavioral data within dynamic, complex, natural environments is an ambitious and challenging undertaking. Not surprisingly, the accomplishments in this area are only somewhat incremental given the immature state of the art and the groundbreaking challenges to develop needed enabling technology and methodology. Continued strategic investment to push advances in this area would yield significant payoffs for the Army and potential spinoffs of benefit to other government and private sector research and development.

Both the environment for auditory research (EAR) facility and the SPEAR obstacle course are outstanding, world-class facilities that have been brought in to this key campaign initiative. The research team using these facilities is well focused and has effective leadership. Ongoing hardware updates to the EAR facility will ensure that it is state of the art and easy to use for advanced studies of auditory perception. Since the ARLTAB’s 2013-2014 review, the research at the EAR facility has become more general and more relevant to the real world. Specifically, studies appear better designed, are less controlled than traditional psychoacoustic work, and address questions that can impact situations beyond the specific conditions tested. These advances could beneficially be pushed even further to raise the ante on research outcomes to an even higher level of importance and excellence. The collaboration with the neuroscience group to jointly measure behavior and neural signals, using electroencephalogram (EEG) technology, is laudable.

The research presented in this area appears focused on mission-relevant problems and contexts and draws on measures from multiple domains (e.g., biomechanics, cognition, and neurosciences), consistent with the goal of addressing real-world complexity. For example,

• The research using electrocortical activity to distinguish between uphill and level walking is apparently the first study to demonstrate that cortical activity changes while the human walks over different terrains. However, given the limited statistical power of the EEG signal to distinguish the terrain condition (level or incline), it would be necessary to consider more sophisticated analytics (e.g., source analyses for EEG studies) or other methods (e.g., signal enhancement methods that can reduce artifacts from electrode movement) to provide feed-forward control signals for exoskeletons. Regardless of outcome, this type of work is a good example of the foundational research needed to support the development of highly mobile sensing systems that could be useful in the field. Advanced measures for evaluating signal quality need to be developed, and the translational neuroscience (TN) group needs to ensure that it is aware of and understands the lessons learned from prior work in this area.
• The work on stretchable conductive elastomers for soldier biosensing was impressive and, if successful, could have applications well outside the military realm. This is also important foundational work, showing encouraging progress, to enable EEG measurement under real-world conditions where soldiers are moving, sweating, and otherwise burdened.
• The effort showing the effects of marching, rucksack load, and heart rate on shooting performance in the field is an applied study with considerable generality, because it is designed to understand the effect of work and fatigue on human performance. The study makes excellent use of the SPEAR facility and is very relevant to real-world Army combat, where soldiers may carry heavy rucksacks for extended periods of time.
• The research dealing with temporal and semantic coherence of sounds has implications for the presentation of multisensory data for training—for example, the possible effect of latency in presentation of initiating event and audio and visual stimuli generated in response to the event.

• The research on a novel measure of driver and vehicle interaction and research on eye movement correlates of behavioral performance in a simulated guard duty task showed interesting use of validating measures other than EEG. The novel driver–vehicle interaction metric and eye-tracking measurement used appeared less complex with respect to the collection and interpretation of data than the EEG measures that seem to dominate ARL’s real-world research paradigm.

Challenges and Opportunities

Measuring and understanding behavior in the real world is an audacious objective given the limits to the current understanding of nonlinear causal propagation and emergence in complex real-world systems and environments. True multidisciplinary projects are generally rare but are essential in order for this area to mature to yield meaningful and valuable outcomes.

Research in the real world requires giving up experimental control for the most part. It requires ecologically valid contexts and tasks and the means to capture and analyze the nonlinear causal interactions of multiple variables that may contribute to variance in performance and behavior. For example, it is not appropriate for real-world studies of auditory localization to assume fixed head and sounds unnaturally short that are turning on or off simultaneously. Expertise in ecological psychology and complex sociotechnical systems engineering is needed and could provide useful insights and research strategies under this initiative.

It may be possible to conduct some tightly controlled studies as a tool for assessing whether a new technique is feasible and then progress to a study in a more ecologically valid setting. Indeed, there may be a continuum of increasing complexity within which it may be possible to effectively derive valid causal assumptions. Therefore, based on findings, the driving–vehicle interaction work might be repeated for validation in more capable simulators before moving out to data collection in real-world driving in the field. Developing a methodology for moving research between the laboratory and real-world settings would be a major step forward and a significant contribution to establishing a methodological advance for real-world research.

The project on systems-based metrics of team states using communications data is somewhat ambitious, envisioning the collection of massive amounts of data on hundreds of individuals on a 24/7 basis on brigade-level mission command training exercises. Big data analytics will then be applied in an effort to explore the extent to which group efficacy is a function of variables such as shared understanding of command intent. The hope is that the data collection will be unobtrusive and that it will permit intervention before an existing or potential problem. The analysis of this massive volume of heterogeneous team communication data appears daunting and might benefit by collaboration with information technology or computer sciences subject-matter expertise resident elsewhere in ARL.

TOWARD HUMAN VARIABILITY

Accomplishments

Understanding and predicting human variability is an important and timely topic of investigation. Current systems are calibrated to the average performance of the average person in challenging circumstances; optimized adaptive systems might enable better use of human capacity when situations and states permit. Advances in this area reflect the availability of increasingly sophisticated behavioral and brain measures and the development of new analytic and statistical methods that may enable adaptive systems in operational settings.

The human variability research at ARL addresses the variability in performance between different individuals and the variability within one individual at different times and particularly in different states.

Two components of this research initiative were presented: (1) cognitive neuroscience research and development with a focus on real-world measurement designed to illuminate the connection between brain states and behavior and their potential implications for brain–computer interfaces and (2) a proposed initiative to understand and predict variations in performance with measures of human behavior and physiological state over long periods, using relatively unobtrusive immersive measures interspersed with laboratory-based tests and measures.

Overall, the work in this area seems to be of exceptional quality. The majority of the research efforts used EEG (or in some cases fMRI) indexes of brain responses associated with visual or auditory processing or with motor responses. Several of the newer projects used either EEG sensor data or fMRI activity to identify patterns of coactive brain circuits that are characteristic during different stages of sensory, motor, or cognitive tasks. Novel work was presented aimed at advancing the state of the art of dynamic analysis of brain signals to identify active brain circuits and show how activity within these brain circuits differs within individuals and between individuals across the time course of responses in typical operator-relevant tasks. The research infrastructure in EEG, eye movements, and (through university collaboration) fMRI is superb and well suited to address the target issues.

This group has recruited an exceptionally strong set of researchers, including well-qualified postdoctoral and early-career scientists representing different technical backgrounds, with some gender and ethnic diversity. A number of the presentations featured early-career scientists. Overall, the presentations indicated good mentorship of these early-career researchers.

The quality, productivity, and scope of coverage in journal publications and presentations continue to be exceptional. The recent peer-reviewed publication record indicated between 5 and 10 journal articles per year since 2012, including 6 to date in 2015, with 10 additional articles submitted. The focuses of publications are well distributed over the scope of the initiative and are directed at cutting-edge issues in human variability. For example, a number are focused on EEG and/or survey methods (e.g., the Big Five inventory) and a number evaluate intra-individual gender measures. The majority of the verbal and poster presentations for current and completed work were focused on brain states and behavior in a variety of operational situations.

An informal network of shared resources and shared knowledge is developing that transcends organizational boundaries. In particular, there appear to be considerable cohesiveness and teamwork that extend across ARL laboratories. The level of awareness and cooperation across the laboratories is impressive, the resources needed to advance the work are available, and the collective expertise can rival that in top academic institutions, where a more insular attitude is commonplace. Cohesiveness can be a powerful tool, and ARL deserves kudos for maintaining this excellent environment.

Challenges and Opportunities

Many of the projects presented appeared to be at formative stages at both empirical and theoretical levels, and the range of problems being worked on seemed somewhat vague, diverse, and unconnected. However, the implicit focus was clarified somewhat during small group discussions with the research staff, who described the focus as the identification of potential brain–behavior signatures for brain–computer interface translation.

The development of practical adaptive systems based on assessment of operator state is an important research objective and would benefit from a focused set of research priorities using a well-defined set of high-value operator tasks and operational contexts. The current effort appears devoted to testing and gaining expertise with various biometric measurements (e.g., field testing of EEG and multimodal sensors). However, the diverse array of biosensors and the extent of potential behavioral components require some winnowing of possibilities. In addition to high-fidelity brain measurements examined in the laboratory, a range of other measures such as multiple cue or cue fusion approaches needs to be considered, with the goal of identifying potentially lower fidelity neurocognitive measures of brain state. Fused cue approaches use multiple data modalities (e.g., heart rate measures, eye movement

behaviors, simple EEG indicators, and other behavioral indexes) to achieve possible benefits of cue integration in the classification of human states (e.g., attention, vigilance state, fatigue). This could, in turn, be foundational to the notion of precision performance, whereby inputs, decisions, and scheduling demands can be tailored to the individual, including the mental state of the individual.

One caution is that the ARL studies on individual differences focus almost entirely on developing neurophysiological measures, with little or no incorporation of more traditional behavioral and psychological measurements. As a counterexample, ARL’s reported research has shown that a relationship exists between the spatial abilities of operators and their performance on robotic tasks. This finding illustrates how much information on individual differences might be missed if the focus is entirely on biological sensors. Advice on the selection of appropriate psychological measurements depends on the parameters of a particular targeted behavior (e.g., precision), but their incorporation could be given a careful and more complete consideration in the overall research strategy.

The planned new initiative in unobtrusive immersive measurement of behavior is potentially an enormous problem domain. The findings and best practices from other related major data projects need to be leveraged to identify the research niche with the highest potential return on investment for ARL. As an example, there is a program of the U.S. Department of Transportation designed in part to characterize state precursors of traffic events. Other potential opportunities for leverage include advances in detecting and correlating attention or fatigue states with human performance in smart-home projects or monitoring system efforts associated with medical issues or adaptive support for aging populations.

One of the core challenges faced by this work—as in other initiatives addressed earlier—is the availability of suitably representative human subject populations for research testing. These might include civilian populations where those are appropriate or selected access to military personnel, where data need to be representative of the populations to which the research is expected to be applied. Suitably representative populations are especially important for credible research on interindividual variation.

Another core challenge is the need to engage appropriate information systems and computational resources where needed. The scope of data that could be collected in the immersive measurement for understanding and predicting individual variation in performance has the potential to be massive. The measurements planned in the immersive workplace (possibly including recording facial expression, direction of gaze, all keystrokes in work projects, and some simple physiological measures of heart rate) suggested the generation of large amounts of data. While preliminary and pilot testing of such data in smaller units may be feasible without special arrangements, the large data demands of ongoing immersive projects of this kind are likely to require specialized data management plans and adapting existing algorithms or developing new algorithms for data mining and data analysis. This would necessitate engagement of more expertise in information systems and computational resources. While the information systems and computational resources were not part of the research program under review, it was indicated that arrangements were under way for broader collaboration with other ARL researchers and staff with expertise and resources in this area.

The cognitive neuroscience group has recruited a number of early-career researchers and collaborators from academia. Although there was evidence for strong mentoring of these individuals, the expansion of the group, combined with the fact that key individuals are moving up into administration, suggests a need to develop expanded procedures for mentorship and for the identification of a set of mentors.

For the planned research into characterizing and predicting human variability, a series of workshops is being planned to discuss relevant research topics to enable program development. This is laudable, and a beneficial expansion of this effort (perhaps under ARL’s open campus initiative) could include short-term visitors from academia or industry to broaden intellectual engagement around planned initiatives such as that for large-scale immersive measurement and individual variability.

Research on individual variability is one of the most difficult problems in the field of psychology, and there is much motivation to understand the phenomenon. High-quality and high-impact research in this area, focused on measurement, estimation, and prediction, could position the group for a broadly recognized leadership role. Unique high-impact studies of individual differences and intraindividual

variability as it relates to stress, fatigue, and other psychophysiological states in targeted operational environments could also provide significant benefits, beyond those envisioned by ARL, to external university or industry research efforts that generally do not have direct access to these operational contexts.

ASSESSING MISSION CAPABILITY OF SYSTEMS

Accomplishments

The work in this area is comprised of human-centered engineering and decision support methods, models, and tools supported under ARL’s Assessment and Analysis campaign. Most of the projects that were presented in this area are responding to important and specific Army customer needs, such as the efforts dealing with human modeling, field assistance in science and technology, human behaviors negatively affect engineering solutions, and fire suppressant effectiveness. Soldier surveys conducted by ARL to identify and characterize problems with equipment and systems used in the field yield valuable feedback that can, if effectively acted on, save lives, promote mission effectiveness, and potentially provide long-term cost savings to the Army.

Progress and advances are evident in the integration of human factors and systems engineering tools. A prime example of this is in the integration of SysML activity diagrams as input to Improved Performance Research Integration Tool (IMPRINT) models. The facilities, tools, and test equipment used in this area (e.g., the renovated obstacle course) appear fully up to date and are exceptional.

Commendable efforts are under way at ARL to advance assessment science by developing new models, tools, and metrics to support the acquisition and fielding of effective human systems responsive to emerging missions and threats. For example, improving the integration of human factors and systems engineering tools in the soldier decision framework model is a significant step forward. Advances in technology, system and mission environments, and changes in soldier roles and the nature of the work they do require attendant and anticipatory advances in assessment science, methods, and tools.

Challenges and Opportunities

The Army continues to be a DOD leader for human systems integration as an integral part of DOD systems acquisition and engineering. As such, it is an important challenge and responsibility for ARL to push the science of human systems evaluation, not just to be the evaluators of record. ARL needs to lead the envisioning and development of new methods and models that are militarily relevant to future technologies, missions, and threats and that will support the acquisition and fielding of effective human systems. As an example, ARL leverage of advances in cognitive engineering can support diagnosis and assessment of complex human systems, human technology interfaces, training programs, and work redesign with methods and tools (e.g., cognitive task analysis) that can identify the mental demands (e.g., workload) and cognitive skills (e.g., situation awareness, decision making, and planning) needed to complete a task or accomplish a mission.

Another major challenge is to improve the timeliness of human systems assessment and analysis to ensure that it is not cast aside, to the detriment of soldier and mission effectiveness, in the rush for rapid fielding due to war. The process of evaluating new systems in wartime has to allow a rapid response while not ignoring the human science factors. The problem is how to best tailor human systems processes to support wartime (rapid fielding) versus peacetime system acquisition. Enhanced use of modeling of system and mission environments needs to be considered for its potential to accelerate these processes.

ARL needs to be in the feedback loop, both sending and receiving on human systems integration (HSI) issues from system conceptualization, acquisition, and design, through fielding to prevent future problems and failures with the process and to ensure long-term improvements. For example, problems

identified prior to deployment that were rectified need to be tracked to determine their impact in the field. Similarly, problems in the field that were identified might warrant changes in process to assure they are captured in the future. Field data on adherence to usage recommendations or human systems effectiveness of equipment need to be systematically fed back to modifications or redesign. This would enable continual improvement of the ARL assessment and analysis processes and their impact on future Army systems. An additional benefit would be a documented audit trail that could support validation of the cost-benefits of the process for use in future baselines.

ARL could use customer-funded projects to motivate human sciences S&T research ideas and directions and to provide pilot data. Because research is fundamental to ARL, even customer-funded projects could push the research agenda. A continual focus on using contract work to provide pilot data would enhance research and identify meaningful data or problems. If the linkage between ARL human sciences research and HSI applications were more transparent, it could facilitate investment by external operational customers in the work. For example, ARL researchers indicated strong desires to connect research-based findings with field-based operational effectiveness and customer-responsive work.

Researchers making connections with military personnel in the field to gain soldier perspective has the potential to be transformative. It can inform ARL research, motivate scientists and engineers, and lead to rapid and/or dramatic changes that can have significant impact on soldiers. Programs that enable ARL investigators to collect field data have been cut, thereby reducing and/or eliminating experiences and data that are extremely valuable to the HSI effort. Assuring these opportunities for research and assessment in the field continue and expand is a key priority. In a similar vein, the Army provides a course whereby new employees are introduced to Army systems and procedures. There would be value in making this real-world sensitization standard procedure available for all ARL civilian scientists and engineers.

OVERALL QUALITY OF THE WORK

Effective human system performance is essential to Army mission effectiveness, and ARL’s investment in quality research and development in the human sciences has potential for significant impact on the present and future Army. In general, the quality of the research presented, the capabilities of the leadership, the knowledge and abilities of the investigators, and proposed future directions continue to improve. ARL has done an excellent job by hiring highly skilled postdoctoral researchers, who are being groomed to become next-generation researchers. The increased exposure of ARL in publications and conferences is commendable. This will further improve S&T quality, encourage interactions with researchers who study the same issues elsewhere, provide ARL personnel with invaluable networking, and prompt researchers from other agencies and from universities to offer their expertise to ARL. The ARL facilities are, for the most part, superb. Collaboration with the broader scientific community is generally good and would be amplified by effectively networking with and capitalizing on the great academic programs that many of these postdoctoral researchers have been drawn from.

While there has been good progress at growing ARL basic science, more needs to be done. Unique contributions to science in the portfolio presented are not always evident. The key exception is the initiative on individual variability, which has effectively demonstrated the potential to make important contributions to the science of brain state measurement and individual differences. Overall, the ARL neuroscience laboratory, with its emphasis on high-quality R&D and peer-reviewed publications on par with distinguished university peers, is an exemplar on which to grow and sustain basic science across the human sciences KCIs.

The new emphasis on advancing assessment science in the human factors area of assessing mission capability of systems is commendable; it needs to be brought into balance with the assessment-for-hire field assistance in this KCI, to position ARL to effectively support the Army after next. The relationships built through good customer-driven work can be leveraged to gain support for the science and technology needed to advance this capability.

ARL may already possess the in-house expertise to effectively deal with many of the complex challenges anticipated to confront the Army after next; it is important to assure that beneficial synergies that exist across the KCIs are systematically leveraged. Three areas in human sciences might benefit from broader ARL exposure, engagement, and collaborations: Big data analysis, autonomous or semiautonomous systems, and EEG-based studies in real-time (or near-real-time) systems.

Linking on-site research to field-based applications work and developing a unique set of capabilities relevant to Army needs, military operations specialties, and contexts potentially represents a unique strength and feature of the ARL human sciences. More work needs to be done here. The natural tension between the comfortable technology pull of customer-driven work and the disruptive potential of innovating through technology push needs to be better balanced to best impact the Army after next. The KCIs are promising in this regard; it may take some time to transform the current program of work and its representation to consistently align with this new core competency structure that links expertise across ARL directorates.

Taken individually, each area of the human sciences campaign (humans in multiagent systems and real-world behavior and human variability) can be cast as a grand challenge, because it is complex, multidisciplinary, and involves many unknowns, requiring multilevel focuses on theory, data, modeling, and engineering to meet stated and implicit goals and objectives. For example, research on individual variability is one of the most difficult problems in the field of psychology; the planned new initiative on unobtrusive immersive measurement of behavior is potentially an enormous problem domain; and measuring and understanding behavior in the real world is an audacious objective given the limits to our current understanding of nonlinear causal propagation and emergence in complex, real-world systems and environments. These grand challenges are further complicated by the interdependence and overlap of these areas of the campaign. A coherent strategy and vision, not yet evident, are critically needed for how ongoing and planned research holds together, capitalizes on useful existing theoretical and empirical baselines, and cumulatively builds to achieve goals and objectives over the near, medium, and long terms.