Summary

The statement of task that guided the work of the Army Research Laboratory Technical Assessment Board (ARLTAB) is as follows:

During the 2017-2018 assessment, the ARLTAB is being assisted by seven panels, each of which focuses on a portion of the ARL program conducted in ARL’s science and technology (S&T) campaigns: Materials Research, Sciences for Lethality and Protection, Information Sciences, Computational Sciences, Sciences for Maneuver, Human Sciences, and Analysis and Assessment.

This interim report summarizes the findings of the Board for the first year of this biennial assessment; the current report addresses approximately half the portfolio for each campaign; the remainder will be assessed in 2018. It should be noted that because a full spectrum of projects and programs within each ARL campaign and the interrelated mapping across all campaigns’ projects and programs was not provided to the ARLTAB, this report represents the Board’s assessment on only the projects and programs presented.

During the first year, the Board examined the following elements within the ARL S&T campaigns: (1) Materials Research—energy-efficient electronics and photonics, materials for soldier and platform power systems, and quantum sciences; (2) Sciences for Lethality and Protection—battlefield injury mechanisms, directed energy, and penetration, armor, and adaptive protection; (3) Information Sciences—sensing and effecting, system intelligence and intelligent systems, human and information interaction, and atmospheric sciences; (4) Computational Sciences—advanced computing architectures, data-intensive sciences, and predictive sciences; (5) Sciences for Maneuver—intelligence and control, machine-human interaction, and perception; (6) Human Sciences—humans in cybersecurity, humans in multiagent systems, human variability, and real-world behavior; and (7) Analysis and Assessment—ballistics survivability lethality and vulnerability, personnel survivability, and human systems integration. A second, final report will subsume the findings of this interim report and add the findings from the second year of the review, during which the Board will examine additional elements within the ARL S&T campaigns.

The mission of ARL, as the U.S. Army’s corporate laboratory, is to discover, innovate, and transition science and technology to ensure dominant strategic land power. In 2013 ARL restructured its portfolio of ongoing and planned research and development to align with its S&T campaign plans for 2015-2035. ARL has maintained its organizational structure, consisting of six directorates: Computational and Information Sciences Directorate (CISD), Human Research and Engineering Directorate (HRED),

Sensors and Electron Devices Directorate (SEDD), Survivability and Lethality Analysis Directorate (SLAD), Vehicle Technology Directorate (VTD), and Weapons and Materials Research Directorate (WMRD). The research portfolio has been organized into science and technology campaigns, each of which describes related work supported by staff from multiple directorates. Appendix Table A.1 shows the directorates that supported each campaign during the 2017 review. The ARL technical strategy document describes the portfolio of each campaign in detail.¹ ARL’s vision is compelling and raises expectations for an innovative program of research designed to be responsive to the needs of the “Army after next.” This is not yet fully evident in the portfolio currently being assessed. The reorganization of the portfolio into key focused campaigns is promising, but it may take some time to transform and mature the program of work to consistently align with new critical paths.

In general, the quality of the research presented, the capabilities of the leadership, the knowledge and abilities of the investigators, and proposed future directions continue to improve. Significant gains were evident in publication rates, numbers of postdoctoral researchers, and collaborations with relevant peers outside ARL. The research work environments were impressive in terms of their unique and advanced technology capabilities to support research. Overall, these are all outstanding accomplishments and mark an advance over prior years.

MATERIALS RESEARCH

ARL’s materials sciences span the spectrum of technology maturity and address Army applications, working from the state of the art to the art of the possible—“25 years out”—according to the ARL. Materials research efforts and expertise are spread throughout the ARL enterprise. As the ensemble of the materials discipline and capabilities, the area of materials sciences is one of ARL’s primary core technical competencies. In the larger context, the mission of ARL, as the U.S. Army’s corporate laboratory, is to discover, innovate, and transition science and technology to ensure dominant strategic land power.

Energy-Efficient Electronics and Photonics

Energy-efficient electronics and photonics are intended to address the size, weight, power, cost, and time (SWAPCT) of soldier technologies on the battlefield. The impact of advances utilizing optical equivalents, efficiencies realized through new radio frequency (RF) waveform and encoding strategies, and efficiencies for directed-energy applications are envisioned as being significant and important targets. Examples include escalation of electronic warfare technologies down to the individual unit and soldier in a continually contested RF environment, exacerbating current power challenges even more.

Materials for Soldier and Platform Power Systems

The research programs of materials for soldier and platform power systems (MS&PP) are motivated by soldier battlefield power needs both currently and in the future. The research supports the tactical unit energy independence (TUEI) essential research area (ERA), with focus on unburdening the soldier by making power lightweight, providing power on-site, and diminishing power needs, all essential enabling factors in supporting soldier welfare and effectiveness. Toward these ends, ARL is among the country’s top-tier research organizations. Its research portfolio includes a mixture of world-leading, established, innovative projects and recently initiated programs anticipating scientific trends.

___________________

¹ U.S. Army Research Laboratory, Army Research Laboratory Technical Strategy 2015-2035, Adelphi, Md., 2014, https://www.arl.army.mil/www/pages/172/docs/ARL_Technical_Strategy_FINAL.pdf.

Quantum Sciences

Quantum sciences is a new program area of high scientific quality and well aligned with the long-term goals of ARL’s mission to provide the Army of the future with clear tactical advantage. It is anticipated that quantum sciences will provide game-changing capabilities for command, control, communications, computers, intelligence, surveillance and reconnaissance (C4ISR) for the Army of the future. It is critical that ARL maintain and expand this research effort.

SCIENCES FOR LETHALITY AND PROTECTION

ARL’s research in the area of sciences for lethality and protection during 2017 ranged from basic research that improves our fundamental understanding of the scientific phenomena and technology generation that supports battlefield injury mechanisms in human response to threats and human protective equipment, directed energy programs, and programs that address weapon-target interactions and armor and adaptive protection developments to benefit the warfighter.

Battlefield Injury Mechanisms

The study of battlefield injury mechanisms are a relatively new area of research at ARL, and ARL has shown greatly improved coordination and focus over the past two years. Excellent progress on many topics was observed during the review. There has been considerable improvement in prioritization of projects, at least in the near term; while some of the specific goals and timelines of the remaining elements of the program need to be better developed and articulated. The ARL focus on the definition of biological injury in materials and engineering relevant terms is a necessary step in moving this critical area forward and may be unique in the field. The focus at ARL on identifying the critical size scale of injury is correct and the group emphasis on the translation of animal data to humans is necessary and positive.

Directed Energy

The ARL directed energy (DE) program focuses on radio frequency (RF)-DE and laser-DE. ARL leadership and research teams successfully implemented some of the recommendations of the previous ARLTAB report.² Specifically, the collection of presented projects demonstrated a coordinated strategy across the enterprise for the laser-related work, with indications of much greater collaboration with the Navy and Air Force. Internal to ARL, principal investigators (PIs) demonstrated greater awareness of work done with threat warning and countermeasures. The quality of the programs will continue to benefit from even deeper and more frequent collaborations both internal and external to ARL to foster rapid innovation with operational and contextual relevance.

___________________

² National Academies of Sciences, Engineering, and Medicine, 2017, 2015-2016 Assessment of the Army Research Laboratory, The National Academies Press, Washington, D.C.

Penetration, Armor, and Adaptive Protection

ARL continues to demonstrate a strong record of achievement in the fundamental and applied sciences and the engineering of penetration, armor, and adaptive protection. The ongoing work continues to highlight how ARL is building on its history of excellence to provide the knowledge basis for future Army needs in the area of warfighter protection. This is an absolutely critical and core competency that underlies Army capabilities.

A start has been made in the uncertainty quantification (UQ) area of research, but there is a long way to go. It was unclear what the level of effort of research is or the number of people involved. ARL needs to continue an emphasis in UQ. Further, ARL needs to pursue the integration of UQ into its data-to-decision workflow that includes modeling and simulation, experimentation, and design. To accelerate integration and given the complexity of their objectives, ARL scientists and engineers need to leverage software and methodologies developed at other Department of Energy (DOE) and Department of Defense (DOD) laboratories.

INFORMATION SCIENCES

ARL research in information sciences is focused on developing and enhancing science and technology (S&T) capabilities that allow for the timely acquisition and use of high-quality information and knowledge at the tactical edge, for both strategic operations planning and mission deployment. Included in this approach are technological advances that support information acquisition, reasoning with such information, and support for decision-making activities such as collaborative communications.

Sensing and Effecting

Research projects covered thematic areas of nonimaging sensors (acoustic, electric, magnetic, seismic), radar sensing and signal processing, image and video analytics, sensor and data fusion, and machine learning. Noteworthy programs include electric and magnetic field sensing, research on the next-generation improvised explosive device (IED) and landmine detection platform, computational advances in electric field modeling, cross-modal face recognition, along with the development and dissemination of a cross-modal face recognition data set to the academic research community, and innovative approaches to fuse textual context with image features to improve machine learning of human activity.

The work in sensing and effecting (S&E) was assessed to be generally of high scientific quality, and there was a balance between theoretical and experimental work, as well as evidence of transition into practice or use by other areas.

System Intelligence and Intelligent Systems

System intelligence and intelligent systems (SIIS) research spans areas of information understanding, information fusion, and computational intelligence. This research has produced key results in areas relevant to Army needs, including the understanding and analysis of complex environments and streaming data, navigation, exploration, and mapping of the physical world. The work on unsupervised learning of semantic labels in streaming data, and the synergies between visual analysis and efficient exploration of environments is noteworthy. It has identified gaps in the state of the art and is being disseminated in leading venues. The approach of collaborating with researchers throughout ARL as well as on the outside to develop a continuum of work, ranging from information analysis (in SIIS) to decision support (in human and information interaction), will yield good dividends.

Human and Information Interaction

Human and information interaction (HII) is a new program in the Information Sciences Campaign, and has been in operation for about one year, bringing together researchers from disparate disciplines and technical backgrounds. “Interaction” is what distinguishes HII in the information science and human science research space. The objective of HII research at ARL is to develop models, methods, and understanding of data and information generated by humans and intelligent agents in a complex, multi-genre network environment. It further examines tools to respond to user information needs with due consideration of user variability and mission constraints, and thereby to develop timely and accurate situational understanding.

Atmospheric Sciences

The research portfolio of the Battlefield Environments Division (BED) seeks to improve environmental understanding of the planetary boundary layer (PBL) and processes that operate on small spatial and temporal scales, and on developing appropriate environmental intelligence tools for deployed soldiers to use in austere, complex operating environments. The research projects reviewed by the panel during this cycle included detection and characterization of chemical aerosols, acoustic and infrasound sensing, development and fielding of a meteorological sensor array at White Sands Missile Range (WSMR), and advances in small-scale atmospheric model development, verification, and validation.

COMPUTATIONAL SCIENCES

The computational sciences panel examined projects in advanced computing architectures, data intensive sciences (AI and machine learning), and predictive sciences. In each area, there has been substantial progress since the last review.

Advanced Computing Architectures

Alongside operating and managing high-performance computing (HPC) systems to serve the processing needs of the broader DoD community, the group has evolved to focus on tactical HPC at the edge, in order to place innovative new systems at the points of need to support the soldier in complex operational environments. The research portfolio comprises projects for evaluating and advancing new and emerging architectures to enable artificial intelligence and machine learning applications that require real-time analysis of large-scale data at energy-constrained environments. There are also projects on models for quantum and classical networks for secure communications, where measurable progress has been made toward understanding the potential range of uses of quantum computing and networking through modeling and simulation.

Data Intensive Sciences

The data intensive sciences program has made progress in applied Machine Learning (ML), neuromorphic/low-power computing, and cooperative multi-agent control using deep reinforcement learning. The data intensive sciences overview and presentations on neuromorphic processing and cooperative reinforcement learning were excellent. The panel congratulates the data intensive team on its strong start in machine learning. ARL has not only hired new talent but also leveraged its existing researchers and external collaborators. The panel recommends considering a range of low-

power/neuromorphic computing frameworks and the development of practices for leveraging the expertise of external collaborators to enhance the research of ARL scientists.

Predictive Sciences

Current predictive sciences work at ARL focuses on developing the various “multi-“capabilities that are required for accurate computational analysis, which includes multi-scale, multi-disciplinary, and multi-fidelity analysis. In each of these regimes, ARL researchers have a stated goal of including relevant verification and validation (V&V) and uncertainty quantification (UQ) capabilities in their computational analyses. A related R&D goal is to leverage results of prior computations toward developing various surrogate reduced-order models that retain the accuracy characteristics of the original models while requiring significantly lower computational time. The panel observed a general improvement in the quality of predictive sciences R&D from 2015, and in particular, noted a decreased variance in quality across the range of efforts presented.

SCIENCES FOR MANEUVER

In general, research presentations and posters were professional, logical, content-rich, and useful. Clear growth in knowledge content by ARL researchers and support staff was demonstrated. Significant advances in the use of analytical and simulation tools were observed. The collaborative interactions—for example, the Collaborative Technology Alliances (CTAs) and Collaborative Research Alliances (CRAs)—continue to be productive. The Board noted the various director-level responses to previous Panel recommendations. These positive responses are also reflected in the continuous improvement in Campaign research performance.

Several research programs were observed to be outstanding. Three such research programs stand out—research on low-ranked representation learning of action attributes (flexibility and extensibility) in focusing on human action attributes; research on autonomous mobile information collection using a value of information-enriched belief approach (projected functional stochastic gradient based-approach with teams of robots); and research and simulation work on the Wingman Software Integration Laboratory, which has a clear path to Army-relevant static and dynamic scenarios and multiple-machine and multiple-human interactions.

Intelligence and Control

The overall technical quality of the intelligence and control effort is good, and has shown continual improvement—particularly since the 2015-2016 assessment by the ARLTAB. The group has benefited from the hiring of highly skilled postdoctoral researchers, some of whom are being groomed to become full-time ARL employees. Publication in peer-reviewed journals and participation at professional conferences has continued to grow, coupled with increasing participation in other professional activities. Collaborations with peer communities and reputable academic groups appear to be healthy, and provide the researchers with invaluable networking opportunities and options to leverage quality research elsewhere. The investment in quality R&D, especially in areas less likely to be pursued by academia, has increased the potential for impact. The connections between the individual research projects and the CTA and CRA programs are very useful and are highly commended. While it would be a mistake to expect all basic research to be tied into the CTAs, the CRAs provide rich sources of data and research problems, and ready platforms for integration and testing in a research-friendly environment. The CTAs and CRAs may naturally serve as a starting point for the benchmarks.

Machine-Human Interaction

The research generated by the machine-human interaction (MHI) group is generally of high quality, and is focused on important MHI areas such as the development of algorithms that will allow a machine to more effectively communicate and act as a teammate to humans. ARL’s MHI research is largely comparable to university-led research in this area. In particular, the posters and presentations typically contained acceptable technical content, experimental methods, presentation of data, and statistical analysis of results. The research reflected a broad understanding of the science, and references to related work indicated knowledge of research conducted elsewhere. The qualifications of the research teams were well matched to the research problems and employed acceptable and often state-of-the-art equipment and models. The research typically utilized an appropriate mix of theory and experimentation to arrive at well-reasoned conclusions. The Wingman Software Integration Laboratory was identified as a promising project potentially resulting in outstanding data and knowledge that could ultimately be transitioned to the field. The project is focused on an important topic, necessary for the deployment and implementation of human-machine teams with automated targeting. ARL has a strong set of well-qualified MHI researchers addressing important, Army-related problems. These researchers have a unique opportunity to generate mission-critical data from a population of specifically trained human subjects. Doing so would increase the impact and applicability of the research while also helping the researchers better understand the needs of the population they serve.

Perception

Overall, perception research is addressing cutting-edge problems, with meaningful and relevant results. The group demonstrated an appropriate mix of theory, computation, and experimentation. The group’s publications list and strategy spans the gamut from respected, application-based conference venues to well-regarded academic conferences and publications. There is an opportunity for the group to extend and enhance key projects to yield publications in the field’s very best journals with some regularity. When considering the collective portfolio of researchers at individual, leading universities or laboratories, the work achieved by ARL is comparable in scope and outcome. Together, the perception group’s projects reflect an understanding of relevant state of the art, while demonstrating a commitment to pursuing key open questions of Army relevance. It is clear that ARL has attracted well-qualified research staff and provided them with excellent facilities for conducting cutting-edge research in perception. Several of the projects were particularly well presented and showed strong promise to transition to Army use. One such project is the online gyro calibration algorithm; another is the embodied training project. These projects demonstrated solid understanding of the tactical ARL end point while bringing together the proper theory or practice, as needed.

HUMAN SCIENCES

Real-World Behavior

This initiative is an ambitious and promising entry into the challenging field of the measurement and analysis of real-world behavior (RWB). Overall, the technical quality of the work is high. In particular, the group has worked to identify technical and theoretical gaps and to align resources to solve specific needs. The technical quality of enabling technology and instrumentation was especially high. In general, the group uses strong experimental techniques and appropriate modeling approaches. There has been a continued improvement in research products, including published papers, chapters, technical reports, and conference papers. Still, as new research areas are broached, the work would benefit by consultation with appropriate experts.

Human Variability

The analytical abilities and techniques of the Human Variability program in general are strong. The electroencephalogram (EEG)-related technical expertise is excellent. The source localization methods being developed are interesting and a good approach to go beyond simple subtractive methods of analysis.

The Human Variability projects have made progress since the last review by continuing to publish findings in the scientific literature and present findings at conferences.

Humans in Multiagent Systems

Overall, the technical quality of the work is good, and methodologies that are used to explore the research questions are appropriate. Throughout the description of the research in this area, there were many examples of good interdisciplinary collaboration to support broad-ranging questions among computer scientists, cognitive scientists, and human factors psychologists and engineers, such as in the work on trust in robotic transportation systems. The Humans in Multiagent Systems team leverages the foundational (6.1) research of their colleagues in the Human Variability and Real World Behavior programs to inform their applied (6.2) work as well as collaboration with operational warfighters, at Fort Bragg, for example. The identification of the Cyber Human Integrated Modeling and Experimentation Range Army (CHIMERA) lab as a target of opportunity for research into the human aspects of cyber security, represents foresight into outreach and collaboration with other organizations, and it leverages investments made elsewhere. The researchers demonstrated a clear understanding of the importance of measures of performance and measures of effectiveness, that achievement of the former does not always translate into achievement of the latter, and that this disconnect needs to be addressed in their research.

One area that lacks depth is qualitative research. The work on sociocultural influences in particular seems heavily dependent upon the use of qualitative methods, but the methods used for conceptualizing research questions and analyzing and presenting data need some additional expertise to be brought up to the standards of similar academic research.

Another area where more depth is needed is in the area of teams research. In some areas—particularly the work on teams in cybersecurity—the lack of deep expertise on current literature is leading to slow progress in the research. The opinion of some ARL researchers that nothing is known about the area of cybersecurity teams reveals a lack of knowledge of other teamwork research. Abstracting the issues of cybersecurity teams to think in terms of complex, fast-paced decision making in the face of adversarial pressure would reveal some relevant and useful literature to build upon.

A related area that could benefit from more expertise is in multilevel theory and analysis. Ultimately, to translate the findings across campaigns into actionable conclusions will require integrating findings from individual-level research into teams and higher levels of analysis. Much of the research presented was at the individual level of analysis; the few examples of teams research made no use of information on individual differences, which would undoubtedly affect how teams operate.

Human Cyber Performance

Because this line of research is in its infancy, it is very difficult to assess the technical quality of the work at this time. With that said, it was clear that the researchers are beginning to come up to speed in cybersecurity as they continue to collaborate with their technical counterparts who have a deeper understanding of the technical components of the cyber mission. This collaboration will be essential as the team advances its vision to develop a human science of cybersecurity.

ANALYSIS AND ASSESSMENT

The ARLTAB assessment of this campaign is different from that of most, if not all, other campaign assessments, since the Analysis and Assessment Campaign is intended to be more of an analytically focused, crosscutting, activity rather than being research focused. As a result, the criteria are different from those of research-focused campaigns. Nevertheless, the work needs to have technical depth.

The quality of the technical staff and the quality of the work reviewed was generally outstanding. Nevertheless, current analysis and assessment (A&A) efforts are falling further behind in incorporating the complexities of the technologies and environments needed for A&A. This could be due to lack of resources—either financial, personnel, or both. In several areas, the team is only one deep, so there is a lack of personnel, likely driven by funding. Resources need to be made available to address the requirements to analyze and assess complex technologies in complex environments.

The modeling work is generally of high quality, but rigorous verification and validation is not always included in model and tool development; this is especially the case in human systems integration (HSI).

Of the core technical ballistics survivability, vulnerability, and lethality (BSVL) efforts reviewed, the projects on underbody blast modeling and the collection of data that shows the impact of multiple hits on armor panels were of very high technical quality and the teams were highly educated and skilled to conduct these efforts. However, limited resources, either funding or personnel, resulted in important effects being modeled well after this information was needed. A lack of validated models has led to the need to carry out a very costly experimental program, illustrating that it is important to stay ahead of the need.

Three of the Key Campaign Initiatives (KCIs), “Framework for Complex Multidomain Analysis,” “Analysis & Assessment of Congested & Contested Operational Environments,” and “CEMA Analysis and Assessment Methodology for Congested and Contested Environments,” have significant overlap, are well outside the current mission space of ARL, and are so broad that the outcomes cannot be clearly seen in the 15-year time frame. More definition and work is needed on these initiatives. The fourth KCI, “Virtual Interactive Simulation Analysis and Assessment,” is a long-term follow-on effort from the immersive demonstration that served as a proof-of-principle for these visualization techniques. These KCI efforts would all benefit with near- and long-term deliverables defined.

The area of analysis and assessment is a very important area to the Army. The analyses and assessments prepared by ARL support Army decisions at all levels of the Army. The products of these assessments and analyses are used by the Army Evaluation Command in preparing recommendations to Army leadership up to and at the Secretariat level. A&A is an important activity that is very underresourced and falling further behind in meeting mission goals, which puts ARL at risk to losing this activity to another Army organization. It is important to keep this activity in ARL to link the 6.2 tool development with 6.6 tool application.

BSVL has historically led the way in modeling ballistic survivability, lethality, and vulnerability for the Army, DOD, and international allied community. There is no competition for leadership in this area—it is ARL’s mission. The movements to embrace HPC to speed computations and support the Army community needs for A&A are commendable efforts. The emerging methodology for underbody blast and multihit survivability analyses will be exceptional contributions to Army A&A.

The finite element modeling of underbody events on vehicles is top-notch work. The finite element modeling team has a clear understanding of the fidelity required and is advancing state-of-the-art tools and contributing to their validation.

The approach for the physiological experimental work combining high-speed X-ray imaging with state-of-the-art and exploratory sensor technologies is an example of outstanding work that is leading this field. It is the best high-speed X-ray capability that has been observed in this area.

In HSI, the human physical accommodation models and soldier performance and workload modeling and simulation tools developed and employed by ARL are first rate and have provided the Army and industry with an excellent capability to assess soldier integration into complex systems. Current tools, including the Improved Performance Research Integration Tool (IMPRINT) and digital clothing and

equipment models, provide analytical capabilities that can be cost-effectively applied early in the acquisition cycle as well as later during system development.

CROSSCUTTING RECOMMENDATIONS

Based on the 2017 reviews whose assessment is summarized in this interim report, ARLTAB offers eight recommendations.