4
Information Sciences
The Panel on Information Science at the Army Research Laboratory (ARL) was charged with reviewing ARL research in the broad areas of computational sciences, information sciences, and atmospheric sciences. A 2-year cycle of review has been adopted for this purpose with the focus in 2015 on reviewing activities in computational sciences and on that portion of the Information Sciences Campaign related to work in system intelligence and intelligent systems (SIIS) and in sensing. The review in 2016 was focused on research related to networks and communications, cybersecurity, human information interaction, and atmospheric sciences.
ARL research in information sciences is focused on developing and enhancing intelligent systems for the analysis of information and knowledge. Included in this approach are technological advances that support information acquisition, reasoning with such information, and activities, including collaborative communications, that support decision-making. Research in these areas falling under the broad categories of SIIS and sensing were reviewed in June 17-19, 2015, at Adelphi, Maryland. The research program in atmospheric sciences includes projects that belong to these categories and were reviewed on June 1-2, 2016, at the ARL Laboratory at the White Sands Missile Range in New Mexico. Additionally, research related to networks and communications (NC), human and information interaction (HII), and cybersecurity were reviewed on July 19-21, 2016, in Aberdeen, Maryland.
Research projects in the NC area concentrate on understanding and exploiting the interactions between information and socio-technical networks—in particular, on communications and command and control networks. Areas of specific endeavor include channels and protocols, network control and behavior, and network-based information processing. The research is primarily aligned with the ARL key campaign initiative (KCI) of taming the flash-floods of networked battlefield information, which aims to realize analytical approaches that better describe, characterize, and explain complex, dynamic, multi-genre networks and the data generated by these networks. The research is also aligned with ARL core campaign enabler (CCE) of networking and communications in contested and austere environments.
The Army faces significant and growing challenges in the area of human and information interaction. The growth in use of unmanned aerial vehicles, robots, bots, and social media is creating a challenging communication environment for the Army of 2040 in which humans and machines communicate, but often without the ability to determine whether the communication is from and to a human or a software agent. There is a need to understand how to communicate effectively in this environment and to accurately understand, and in some cases predict, the communications of other actors, so as to make timely and accurate decisions. The need to operate effectively in this environment is creating a need for a new transdisciplinary science of human and information interaction. Human and information interaction is central to the third offset strategy1—particularly the areas of human-machine teaming, cyber warfare, and the development of new operating concepts.
Research in cybersecurity is focused around theoretical advances and model development related to cyber-threat detection, recognition, and defeat mechanisms. The emphasis is on both detection and defeat of highly sophisticated attacks that use techniques very different from those encountered in commercial or civilian settings. The combination of strategic and tactical networks in use by the Army creates unique challenges in the domain of cybersecurity. The dual nature of ARL’s cybersecurity role, operations, and research create a truly unique capability for cybersecurity research. ARL is one of the few cybersecurity research organizations with continuing access to real-world data, the importance of which cannot be overstated. Additionally, ARL researchers interface with a deployed operational environment. This gives ARL an ability to focus on research that actually matters. Taken together, real-world data and access to operators positions ARL’s cybersecurity research to be exceptional. Given that it addresses a constantly evolving environment of threats, cybersecurity research tends to be more applied than pure theoretical advances; the ARL portfolio of research in this area contains an even balance between the theoretical and applied components.
The Army faces significant challenges in both the characterization of the battlespace environment and prediction of optimal conditions for engaging an adversary with overwhelming force. The Army is likely to find itself engaged in a number of challenging environments, from complex terrain to sprawling urban areas. There is a need, therefore, to characterize these diverse environments and develop accurate, relevant, and timely predictions of their future state on spatial and temporal scales useful to Army operations. The related need to collect and process accurate, relevant, and timely environmental characterizations in austere conditions, and translate that data into actionable environmental intelligence for field commanders, will also pose new challenges to the computational sciences community. The research being conducted by the Battlefield Environments Division is addressing these challenges and includes a mix of analytical, computational, and experimental projects in its atmospheric sciences program.
SYSTEM INTELLIGENCE AND INTELLIGENT SYSTEMS
Research projects in SIIS were presented in three thematic areas: information understanding, information fusion, and computational intelligence. Collectively the work addresses technical challenges in the use of sensors, communication, and computing to provide the soldier with new levels of tactical intelligence and the automated support needed for missions. Areas of specific endeavor include language translation, information extraction, semantic analysis, understanding of human trust networks, fusion
___________________
1 This is a Department of Defense initiative. A core component of this initiative is the formation of a Long-Range Research and Development Planning Program that will target several promising technology areas, including robotics and system autonomy, miniaturization, big data, and advanced manufacturing, while also seeking to improve the U.S. military’s collaboration with innovative private sector enterprises.
of conflicting information, integration of video and text analytics, anomaly detection, reasoning under uncertainty, robotic control and path planning, and models of cognition and tactical decision making.
Accomplishments
The research portfolio contains an appropriate mix of theory, computation, and experiments and is clearly of value for the support of Army missions. The research projects were generally, but not uniformly, of a good quality. The strongest work has been published in elite conferences and archive literature, and additional opportunities exist to disseminate other work at respected venues. It was difficult to identify a unified theme or common basis for the multiple projects, and the principal challenge for each of the three areas of research is to fuse the collection of projects into a coherent whole.
Information Understanding
Research in this theme focuses on the development of critical methods and techniques for transforming data so as to provide useful information to the soldier. The work represented solid but incremental advancement in several interesting and potentially important areas such as language translation, information extraction, semantic analysis, and understanding of human trust networks. Some of the projects appear to have potential for short-term applications in the field. The research staff ranges from experienced researchers with doctoral degrees to students pursuing master’s degrees or internships. Laboratory resources appeared to be adequate to support the research agenda.
The work on temporal information extraction focuses on methods for extracting temporal relationships from text for constructing knowledge networks. The proposed approach is technically solid and particularly well grounded in the available literature. Temporal relations are an important area for future work by search engine companies, and the long-term applications to the Army are clear. This work has been published in a top conference—Association for Computational Linguistics (ACL)—an indication that it is first-rate quality work. The impact of the work is already evident in that other researchers at ARL are making use of this work.
Ongoing research on influence in social networks is closely related to activities in the network science Collaborative Technology Alliance (CTA) and pertains to identifying mechanisms for trust formation in human networks. The effort has emphasized identifying main factors, but the work has not addressed this issue in the context of networks. Most results to date are based on an analysis of data from the online microfinancing company Kiva and other lenders and are related to microfinancing and corruption perception indexes. The opportunities for extending this work to intelligent systems research for the Army were not readily apparent. The project would benefit from additional awareness and understanding of the literature in the area of online trust.
Also stemming from work related to the network science CTA is research related to agent-based semantic analysis in information retrieval. This research focuses on how to manage information dissemination across constrained channels based on the trade-off between size and accuracy. Rather than posing the task as a constrained optimization problem, the proposed approach implements a fuzzy logic model in order to include model uncertainty or fluidity in the attributes. The work requires a clearer articulation of what additional insights into operational questions are available from the proposed methodology. Further, to establish the advantages of the approach, it would be helpful to compare results against those obtained through more traditional optimization techniques. If such optimization techniques have inherent limitations or are not applicable in this particular problem setting, statement of the problem could better articulate those techniques.
An example of a 6.2 (applied)-level research program addressing a real-world need is the project whose objective is to perform language translation in an automated fashion with a goal of enhancing translation capabilities for low-resource languages. The approach involves the construction of a language model based on entropy minimization that selects those sentences that need to be translated by humans in order to construct a best overall training corpus. The novelty of the work centers on methods for constructing the language model.
Information Fusion
Research in this area pertains to the fusion of data from disparate sources to produce timely, actionable information for the soldier. The research has advanced the state of the art in several interesting and potentially important areas such as fusion of conflicting information, integration of video and text analytics, and anomaly detection. The quality of the research was generally good, ranging from archival quality results publishable in top journals to preliminary but promising work. In some instances there was an apparent lack of full awareness of critical literature and alternative approaches. The researchers engaged in the projects were well qualified and possessed a range of experience. While laboratory resources appeared to be adequate to support the research agenda, there were not enough experienced staff to fully develop the ideas.
Research on estimating credibility by fusing subject opinions was of good technical quality and was disseminated in a reputable conference. This research focused on fusing multiple inconsistent and potentially conflicting information sources. The work treated inputs as propositions with truth-values and used subjective logic in a manner that took into account prior experience with the sources. It was well grounded in statistical machine learning approaches, but it could benefit from consideration of prior work in the human computation literature.
A project that seeks to develop an observer model for helping soldiers determine salient targets draws upon recent advances in image processing and neuroscience. The approach is to train a saliency model on the basis of experiments with experienced soldiers and then adapt that model for automated training of less experienced soldiers. The work emphasizes anomaly detection in saliency models. It is well grounded in previously published work and could be advanced by research considering information from a network of devices.
In a related vein, research is also being conducted into integrating complementary or contradictory information into a fuzzy model in order to determine the value of the information. The underlying idea is to take multidimensional values of information metrics and to project them onto a single dimension. This kind of dimensionality reduction facilitates subsequent sequence-based analyses. It is not clear that such analyses could be performed without unnecessary loss of information. This work would benefit from considering the results of previous research in indexing, feature reduction, and information theory.
Several research efforts in the thematic area of information fusion were in the early stages of investigation. An effort dealing with intelligent information management in the battle environment focused on bandwidth management during distribution of information to end users. The approach is based on linear scoring functions with weights established by the users. Its effectiveness is based on questionable assumptions about human capability, and the investigators might want to consider alternative approaches in the literature that deal with learning user preference and modeling utility in humans.
Another early research effort that could potentially have important impact is based on a hypothesis that learning from video and text could benefit from being done together when video includes or is associated with text. The improvement in the quality of the results comes with the cost of implementing joint learning methods, a much harder problem to solve. The next appropriate steps would include
a literature review that examines relevant advances in optical character recognition, video retrieval, and object recognition in videos.
Computational Intelligence
Research in computational intelligence examines the development of intelligence in systems to support highly automated or autonomous operations in support of Army missions. The research efforts have yielded significant advances in several interesting and potentially important areas such as reasoning under uncertainty, robotic control and path planning, and models of cognition and tactical decision making. Researchers engaged in the effort are well qualified, but the team comprises a disproportionate number of early-career researchers. Some of the research efforts lack the staff that would be needed to fully develop the ideas. In particular, projects related to information for robot navigation and message delivery and those requiring expertise in psychology would all benefit from strategic recruitment in targeted areas of expertise.
A research project focused on improvement of visual classification for navigation purposes uses an approach of implementing partially supervised discovery and labeling the navigation domain. The research aims to reduce the effort associated with human labeling of terrain and objects in images for the purpose of supervised learning of objects in images. Over-segmenting the image and then using clustering techniques on the resulting segments accomplish this goal. Human labeling based on the clusters is much faster but less accurate. However, this loss of accuracy does not significantly degrade robot navigation when using the new visual system. This research is expected to have an important impact and has yielded promising initial results.
Another promising research project uses a semantic vector space for reasoning in the presence of uncertainties. This work seeks to take advantage of two different types of semantic models with a goal of augmenting a curated knowledge base by reasoning through analogies based on statistical representations. Both the ideas and the proposed methodology contain novel elements, but the work is still in an early phase. A number of complexities have to be resolved, including those arising from multiple meanings for words. The work is well grounded in the literature, and the researchers are aware of related efforts in the research community. With continued support and application to meaningful problems, it has the potential for publication in top journals.
Investigations into the concept of robust distributed communication relays for minimizing message latency in a vulnerable and uncertain operating environment are also ongoing, albeit at a preliminary stage. The research is focused on resource-constrained, dynamic environments, where it may be advantageous to use kinetic or nonelectronic means to augment electronic information distribution. The formulation results in a very hard computational problem based on vehicle routing. The approach to handling the computational difficulty derives from an integrated heuristic/high-power computing (HPC) methodology and is being tested in a simulation environment. While the pertinent literature has been explored and cited, additional benefits in computational efficiency may be gained by using recent work on probabilistic spanning trees.
Another example of an early research project investigates strategies employed by humans in situations similar to those experienced by soldiers facing trade-offs between short-term and long-term actions. Initial results in two-player games, which have been obtained using computer agents, demonstrate that there is no dominant strategy. The underlying hypothesis for the work is that these games will provide an environment for investigating how humans move between strategy, tactics, and actions. Although at a preliminary stage, the work addresses an important problem. It is critical to place this research in the context of the literature on human cognition, training, game theory, and computation intelligence in
games. Another effort at modeling human cognition was based on varying the volume and velocity of data presented to shooters who have to make a choice of targets. This allows investigation of the effects of data attributes (in classic big data terms) on human ability to make decisions. This topic appeared to be narrow research toward a master’s degree, and its future is unclear.
Ongoing work at a more advanced level looks at the use of an information-gain metric to design paths for autonomous mobile robot movement. Entropy minimization concepts are used to develop optimal routes for mobile ground robots. This work has evolved to an advanced demonstration level; there is a good plan for future work that would make use of recent advances in combinatorial optimization methods and that would include constraints on power and available onboard computational bandwidth.
The concept of episodic memory consolidation and revision, or robotic dreams, was used to identify precursors to events such as the deployment of an improvised explosive device. The research is premised on the hypothesis that this computationally demanding problem is intractable in a real-time computation environment but is more amenable to large-scale background processing using otherwise idle computing resources. While this problem is of significant importance, it was unclear what level of computational efficiency was added through the episodic memory consolidation approach vis-à-vis simply devoting additional computational resources to the problem directly. It was not readily apparent how the approach is distinct from traditional work-stealing methods for background processing during low computational loads.
Challenges and Opportunities
The most significant challenge facing the SIIS program is the development of a coherent and unifying thrust for the research program. Overall, there was little evidence that the whole would be greater than the sum of the parts that were reviewed. The portfolio of research is broad, with many lightly staffed projects. A challenge lies in a better integration of the work to pursue fewer, but more significant and more ambitious projects. Such prioritization is important to focus the limited staff on strategically selected projects so as to more fully develop the ideas. It is possible that the above impression is due to the selection of the topics presented to the panel. ARL leadership noted that projects had been selected to provide a broad sample of the research portfolio rather than to explore its depth. If there are elements that more coherently unify the research projects, then it is important to highlight such integration, not just in the top-level overviews but also in the project descriptions themselves.
Intelligent systems research is increasingly dependent on harnessing vast quantities of data, and the fields of scalable machine learning and big data analytics are advancing rapidly. A number of important techniques and tools have been developed to address these emerging challenges, many in the open source community. Some of the projects reviewed in this report would benefit from understanding and using these new tools. Such an approach is being practiced in research related to cybersecurity and other areas that were not reviewed in 2015.
Some of the reviewed projects were particularly strong because they combined pertinent approaches—for example, the combination of statistical/machine learning (ML) methods with linguistic rules—to address research challenges. Beyond such innovative and opportunistic approaches, some impactful research projects demonstrated clear integration with other activities and research at the ARL and other external entities. Other opportunities for such integration exist and could be more broadly embraced.
New advances in automation have changed the roles of humans and machines in intelligent systems to a degree that humans and machines need to collaborate in task activities. Therefore, the human systems perspective has to be fully integrated into any development to ensure usability and robustness of the results. The ARL presentations did not include the work on human interactions, making it difficult to assess the impact of some of the systems intelligence research.
SENSING AND EFFECTING
Research projects in sensing and effecting covered thematic areas of non-imaging sensors, image understanding, sensor and data fusion, and radar signal processing. There was a strong focus in the area of acoustic sensors that collectively examined new materials applications alongside the development and implementation of better signal processing capabilities. Cross-modal face recognition was an important thrust in the image-understanding research. The use of long-wave infrared polarimetry to facilitate discrimination of manmade and nonmanmade objects was another thrust in this research. The sensor and data fusion focus was largely on approaches for dynamically adapting information to situational changes and appears to comprise engineering advances as opposed to exploring fundamental science. Research in radar signal processing was both relevant and of good technical quality, and it focused on signal processing in congested and cluttered environments.
Accomplishments
The research portfolio contains a mix of projects in areas that are well aligned in support of future Army missions. The research was generally of a good quality but not uniformly so. The research projects emphasized applications, as opposed to the foundational science. Some of the work is being published in top venues, but researchers could be given additional guidance and encouragement to present their work at leading conferences and then pursue publication in top archival journals. In general, the research staff were well qualified and demonstrated a good understanding of the important challenges in their work. The long-term research vision provides a natural framework for integrating the individual efforts. However, the connection between these individual activities and this overall framework was not clear. Finally, the time horizon for some of the projects can be shortened (to 3-7 years) to demonstrate proof of concept; the pace of technology advances in the field dictates this necessity.
Non-Imaging Sensors
Research in this domain is primarily on the application of acoustic sensors for Army-relevant tasks, including identifying helicopters through acoustic signatures, localizing gunfire, and detecting vehicles or weapons at long range. The Army has a long history of using such sensors, but new technologies and new signal-processing techniques could extend the range and the precision of these sensors, reduce complexity, and minimize power requirements.
Research related to the application of new materials in the design of windscreens for acoustic sensors has yielded positive outcomes. Acoustic sensors covering a range of frequencies, from infrasound to ultrasound, were explored in this study. Ongoing efforts are evaluating a range of porous materials and are characterizing their efficacy relative to baseline materials used in traditional microphones (e.g., foam). The work is based on results from partners at the University of Mississippi. The results have been presented at good conferences but have not been submitted for publication in archival journals which needs to be done; internal technical reports document the progress of this work. ARL needs to consider the submission of this work for publication in archival journals. There is an opportunity to use foundational theoretical knowledge to guide in the selection of optimal materials for this application.
In research related to enhancing the effectiveness of acoustic sensors, the work on identification of helicopters through the use of innovative signal-processing techniques has yielded promising results. The signal-processing algorithm takes advantage of the relatively invariant blade speed to determine the Doppler shift and to calculate appropriate motion compensation, which is then used to auto-focus. Papers describing the approach have been published in recognized journals.
Research in this domain is also exploring sensors alternative to the more traditional microphone arrays. In particular, a research project is directed at investigating the use of microelectromechanical systems (MEMS)-based, three-dimensional acoustic particle velocity sensors. A commercially available sensor was adopted for this work, and the ARL research is focused on developing signal-processing techniques better than those provided by the vendor for applications that include localization for small-arms fire and triangulation of continuous waves. These devices have significant potential for reducing complexity, weight, and power requirements. Their robustness in field applications continues to be an area for research and development. Findings of this research have been presented at recognized conferences and documented in internal technical reports.
Image Understanding
The image understanding research program was generally of high quality. The work is being published in high-quality journals and shows a comprehensive understanding of research conducted elsewhere and how the ARL research fits into the broader research landscape. The cross-modal face-recognition work represents an excellent example of an appropriate applied-research topic for ARL, given the mission need for such an approach. The researchers were able to articulate the unique Army needs in these problems and were addressing them.
The research on polarization shows strong potential, and continued collaborations with camera producers would enhance capability. Researchers demonstrated an outstanding ability to summarize the significance and impact of the work. Additionally, the collaboration in this project related to sensor algorithms for polarization imagery was considered to be a positive outcome of the ARL open campus initiative.
The image understanding work was applied research rather than fundamental. Appropriate laboratory facilities were available to carry out this research. The work was judged to be relevant to Army needs and could ultimately be transmitted to the field. In particular, the manmade object discrimination work recently resulted in a patent, and a transition path is under way.
Sensor and Data Fusion
The research efforts in this area tended to be more incremental engineering advancements, as opposed to addressing fundamental questions. As an example, design concepts for dynamically adapting information to the situational changes in the utility of data attributes (e.g., accuracy, latency, reliability, data rate) could be a fundamental topic for investigation; instead, it was treated as a human factors problem rather than as a broader concept for automated management of data prioritization. Given the engineering development focus of much of the work, the researchers did not present the current state of the art in sufficient depth, and they appeared to lack access to the field data that would have allowed them to do so.
Research related to detection of vehicles, personnel, or targets is key to Army operations and requires the use of multiple sensor arrays. The fusing of information from many sensors observing similar objects (dependent data) is key to developing better inference. The research focuses on developing a means to fuse correlated information.
The work on dynamic belief fusion applies a sound and seemingly straightforward approach. It integrates the outputs of different object detectors by assigning ambiguity levels derived from previous performance to each detector. An approach of dynamically allocating probabilities based on prior performance has been developed that demonstrates the improvement in detection accuracy over conventional fusion methods.
Another area of research is aimed at developing and enhancing tools to reduce the time between data gathering and making decisions. While important from a practical perspective, the research effort is simply an extension of a fuzzy logic–based tool to assess value of information (VoI) through the use of additional membership functions. The work is in an early stage, and the fundamental technical advances of the approach are not readily apparent.
Across the entire research endeavor, access to actual field data would further enrich the research effort and would help to distinguish the work from that in the outside research community. In the area of dynamically adapting information to situational changes, for example, research in the commercial applications arena focuses on certain performance objectives that do not meet the requirements in the military context. It is important to build in recognition of ease of disruption and cost of errors (e.g., of human life) as explicit considerations in the performance objectives as field data become a critical component in evaluating such criteria.
The laboratories and infrastructure support were appropriate to support the research; the access to the E/H-Field laboratory is particularly beneficial to ongoing research. The researchers were academically well prepared to undertake the work.
Radar Signal Processing
The research reviewed in this domain was of good technical quality and showed promising results. The research problems were well defined, the methodology was explained at an appropriate level, and the results were well organized. Detecting moving personnel under tree cover on the basis of frequency-modulated continuous-wave (FM/CW) radar is an important challenge. A full-wave approach, realized through the use of a parallel three-dimensional finite-difference time-domain algorithm, is deployed for this purpose. Likewise, the ability to detect targets obscured by artifacts in ultra-wideband imagery is also relevant and important from an Army perspective. Mission requirements dictate effective performance in the presence of uncontrolled radio frequency (RF) transmitters, and RF interference notch-filtering techniques are widely used in such applications. An alternative approach based on sparse representation and recovery of signal was proposed that does not have the shortcomings of notch-filtering techniques.
The work related to nonlinear radar methods is also timely. It focuses on the use of nonlinear harmonic radar to achieve greater sensitivity across a narrow frequency band. These approaches have been in the literature for a few years, and early applications in the field represent an important next step. Based on these experiments, a 3- to 7-year time horizon for transitioning to full-scale developments in this area is a real possibility.
HPC facilities at the ARL are a key enabler for this work, especially in applications to detect moving personnel under cover. In a similar vein, the significant use of radio equipment indicates adequate infrastructure to support the research effort.
Two of the projects employed computer simulations to generate input data that were used to demonstrate improved receiver processing techniques. While adequate for purposes of showing proof of concept, the use of real data would also significantly benefit further studies. Simulated data might lack some of the complexities in real data and could lead to false validation of the proposed methodology. ARL might be able to get such data from other Department of Defense (DOD) laboratories or from the industry. If necessary, simple experimental set-ups could be built to collect such data.
Challenges and Opportunities
The overall technical quality of the research in this area was good, albeit with a greater focus on technology development than on foundational research. Overall, researchers are aware of relevant work in their domain of interest, but this was not always the case. The research problems are important to the Army and are unlikely to be pursued in academic institutions or other government research laboratories.
From an engineering standpoint, the researchers seemed knowledgeable about identifying and adapting or creating new signal-processing techniques to the target domain and good at doing this. From a foundational standpoint, however, it was less clear how well the algorithms or ideas would transfer to other domains, or how knowledge would be gained to better understand the underlying physics of the domains.
In the area of image understanding, an increased emphasis on developing rich data sets and the quantitative improvement of performance for larger data sets—especially data sets that include mission-relevant variability—would enhance the impact of the program. In particular, a better understanding of how easily an adversary could defeat cross-modal recognition efficacy is of interest. The research emphasis could be expanded to embrace the fundamental science in order to better understand the physics that drives cross-modal features and the information-theoretic fundamental performance bounds. This would enhance the overall quality of the research and produce results with longer-term impact. Additionally, ARL could better connect with and provide intellectual leadership to the broader facial-recognition community by creating and curating open, standardized data sets as well as challenge problems for researchers to explore.
In the area of sensor and data fusion, dynamic automatic control of data attributes with operator support represents an important emerging area of research. This is especially true given the explosion in sensor technology and limitations in both communications bandwidth and human–computer interaction. A new opportunity would be to expand the focus to automatic data control rather than limiting control to the operator interface as the only basis for stimulating data control actions. While industry wishes to develop cryptographically solid authentication and authorization, the needs of the Army are more complex. While the authorization to manage parameters of a system resides with a few individuals, provisions have to exist for others to take over in the event of unauthorized access by others, or at least prevent it. One possible area for research is to design systems with redundancy, so that if one sensor net or dashboard is subverted, it would be immediately detected. Similarly, based on knowledge of the system, the ability to identify system behavior as illogical, and possibly compromised, would be hugely beneficial in such field operations.
The radar signal-processing work could benefit if the engineers working in this area had graduate-level training. There have been significant advances and new tools in the radar signal-processing area, and familiarity with these would elevate the research. It would also help to improve the success rate for presentations at leading conferences and publications in top journals. Access to advanced technical knowledge online or by other means can be explored. The work related to nonlinear radar shows promise for immediate transfer to a range of battlefield applications. Finding ways to transition this technology as rapidly as possible would be extremely helpful in meeting critical needs in Army operations.
Wireless (and other) communications is a critical part of Army operations today, and its importance keeps increasing. It would be beneficial for ARL to build some capability in this area, at least to support other activities in the laboratory. Lack of this communications technology skills set will increasingly become a disadvantage.
NETWORKS AND COMMUNICATIONS
Accomplishments
Research within NC addresses the technical challenges of communications in highly dynamic wireless and mobile networking environments populated by hundreds to thousands of networked nodes. The research is generally of high quality and relevant to the needs of the Army. The research portfolio is a good mix of foundational theoretical work, with a focus on applications and technology demonstration. In some projects, experimentation is effectively deployed to both validate theoretical models and to improve theoretical modeling. In general, the researchers demonstrate good awareness of related work in other research communities. For some of the interdisciplinary research projects, however, there is uneven awareness of the literature and accepted approaches across the constituent disciplines. Additional focus on formulating relevant research problem statements would be beneficial for these cross-disciplinary research endeavors. The research results are being published in archival journals and in respected conference proceedings. The ability of researchers to travel to key scientific conferences will have a continued positive impact on the overall research environment.
Channels and Protocols
In the area of channels and protocols, the research challenge is to explore new heterogeneous approaches to information delivery across a communication network using previously unexplored network configurations, channels, protocols, and allocation techniques. Research tasks in low very-high-frequency (VHF) range communications, ultraviolet (UV) communications, and on quantum methods for networked communications were evaluated. Research to evaluate the low very-high-frequency range for networked communications represents an effective applied research effort that focuses on both experimental validation and prototype communication node design. The program also reported on the development and demonstration of very small antennas that enable use of this frequency band with small nodes. While this is an applied research and advanced technology demonstration, it is nevertheless of good quality and highly relevant to Army needs.
Work on UV communications included an effective experimental program to validate theoretical propagation models for long link-distances that previously were not explored experimentally. This experimental program has resulted in improvements to theoretical modeling—a notable example of synergistically leveraging theoretical and experimental research for high impact. Finally, in the quantum area, work related to theoretical analyses on quantum key distribution and understanding of quantum entanglement was evaluated. This research represents sound theoretical contributions to the area. This research effort is modest in size when compared to much larger groups addressing these issues elsewhere, both theoretically and experimentally; nonetheless, the work was assessed to be of good quality.
Research related to the common sensor radio and associated medium access control (MAC) protocols were also reviewed. This is a notable example where a sustained and high-quality research effort has resulted in a significant impact. The program has evolved from an early radio design into a communications and networking system that is well suited to address unique Army and other DOD communication network needs, especially for ground operation in which the propagation loss is very high. This program is seeing multiple transitions to other military agencies and to the industry.
Information Delivery
A significant subset of the information delivery research topics included efforts focused on emulation and simulation tools for experimentation, as well as information theoretic foundations, trust in networks, and decentralized learning.
Several projects are aimed at developing emulation and simulation environments for network science experimentation. Many of these leverage the newly opened Network Science Research Laboratory (NSRL). The NSRL provides an open collaborative space for bringing together teams to address multidisciplinary networking challenges, provides flexible, configurable platforms for conducting experiments and studies, and can be used to provide realistic demonstrations of newly developed approaches. Research efforts in this area include video processing over networks, adaptive information query, distributed computing architectures and network protocols, controlled experimental testbeds to emulate field experiments and exercises, and multi-genre node interoperability in networks. The thrust on development of emulation and simulation environments in a research platform like the NSRL has yielded a powerful capability that will enable significant network and cross-coupled research investigations that address future ARL and Army research needs. It serves as a model of enabling capabilities for multidisciplinary collaborative research for complex problems.
The research on semantic information theory and on decentralized learning is of high quality and represents fundamental research aligned with future information systems needs and KCIs. The research on trust-based methods to address decision making while maintaining network efficiency and security is relevant to the NC research challenges and has been published in some leading venues and has received other external recognition. The significance of this research could be clarified by describing how it enhances the state of the art or relates to prior work, and why the trust models are appropriate to complexities of future battlefield communications.
Research on opinion dynamics would benefit from a stronger consideration of problem statement and prior work.
Control and Behavior
The research projects in control and behavior were primarily theoretical and considered algebraic topological approaches to address network coverage and topological persistence. This work is categorized as foundational, with strong theoretical underpinning, and identified as a core campaign enabler; it is more fundamental in nature and did not appear to directly address a KCI. Also reviewed in this category was work related to network topology control to maintain robust network performance in contested environments. The overall quality of this research is high.
Challenges and Opportunities
For much of the NC research portfolio, it is often unclear how the individual research topics address the scientific challenges of the campaigns—the KCIs and CCEs. The introduction of KCIs and CCEs to bring the research topics into focus is viewed as a positive step, yet in many cases the connections between the campaigns and the individual research projects are not clear. It would be helpful to see how the campaigns lead to a coordinated portfolio of research projects, and also how individual projects integrate to a comprehensive portfolio that successfully addresses the campaign objectives. Top-level overviews and project descriptions need to highlight elements that more coherently integrate the research
projects. Continued convergence between the campaigns and the individual research projects would improve the coherence, and thus the impact, of the overall research program.
For some of the cross-disciplinary research projects, there was uneven awareness of the literature and accepted approaches across the constituent disciplines. This was particularly noted for those projects that address human-centric topics such as trust, opinion, and value of information. To ensure high impact and to avoid compromising the overall quality of research, it is important for these cross-disciplinary research endeavors to establish strong, relevant research problem statements. Researchers pursuing such cross-disciplinary projects are encouraged to establish collaborations and strong mentoring relationships with experts in allied areas early in the problem-formulation stage. In areas where ARL has the quality research expertise in its existing workforce, this need can be addressed internally. The KCI structure will drive further cross-disciplinary research, and the need for cross-disciplinary collaboration and mentoring will increase. ARL leadership is encouraged to proactively support early engagement of these connections.
The concepts of value of information and quality of information are helpful ways to develop bridges between the NC research community and the HII community. There does not appear to be a common agreement on what these measures mean and how they are used. A common understanding and agreement on these measures would facilitate fruitful cross-disciplinary research endeavors that consider human effectiveness and network functionality jointly. The topic of resilience of networks was deemed to be of high importance for Army applications. The program and projects did address the issue of resilience at the application level. There was no evidence, however, of an explicit research focus in this area.
Intelligent systems research is increasingly dependent on harnessing vast quantities of data, and the fields of scalable machine learning and big data analytics are advancing rapidly. A number of important techniques and tools have been developed to address these emerging challenges, many in the open source community. Some of the projects reviewed in this report would benefit from understanding and using these new tools. More generally, the NC portfolio needs to include projects that address some of the challenges and opportunities that these vast quantities of data afford to Army application scenarios.
It would be useful if the NC portfolio included a mix of projects—some that are more narrow and focused, emphasizing specific contributions to a very well defined scientific or technical question, and some that are broader and that combine specific new contributions in a system-level effort, to solve a very important problem in an Army specific application.
HUMAN AND INFORMATION INTERACTION
Accomplishments
The Army finds itself today in a completely “informatted” environment, one where information and procedures are constantly updated on multiple cyber devices in use, and where this information can not only be retrieved but also affects or changes the state of the device. The efficiency with which humans can interact with such a deluge of information has given rise to research in the area of human and information interaction. Within ARL, HII is a new endeavor and has not had sufficient time to develop strong areas and lines of research. As characterized by the researchers and ARL leadership, the current research projects represent the following three major thrusts: (1) Dialogue—How can computational technologies help facilitate communications between humans and devices such as smart phones, robots, or even traffic lights, using natural language, utilizing images as appropriate, and in a manner that captures intent, even in a degraded information environment? (2) Immersed decision making—How do individual humans make decisions in a completely informatted cyber environment—where they are
immersed in information from the built environment, social media, traditional media, and a myriad of sensors? (3) Social computing—How do groups shape the information environment, gain information dominance, and take actions and affect appropriate response in a fully informatted cyber environment consisting of societies of bots, gangs of coordinated sensors, and crowds of humans—all connected and working in tandem to effect change?
HII is not human–computer interaction, a field that focuses on the one-person-one-machine interaction. In contrast, HII is concerned with groups—groups of people, software agents and machines, and groups themselves. HII is not social media or social media analytics either, because it deals with more than the content of the communication, and more than just understanding, interpreting, and predicting human and bot behavior in a completely digital world.
The area requires a truly interdisciplinary approach involving sociology, anthropology, linguistics, psychology, cognitive science, information science, and computer science. To build meaningful research in HII, ARL will need to leverage expertise of other researchers within the laboratory as well as the broader external scientific community.
HII is new at ARL, and it was not surprising to the panel that projects are in the early stages, of short duration, and with connections to more detailed 3- to 5-year plans. The status of research accomplishments may be classified as early stages in the HII context, but they build on more mature efforts in related areas. There are both opportunities and significant challenges in the selected projects; each has the potential of becoming a showcase project in this area.
A Lexically Informed Event Ontology
This project develops an a sense-making system for events that draws on existing ontologies such as wordnet and entities, relations, and events (ERE). Such a system could play a critical role in supporting communications in the new cyber environment. Access to an ontology expert would greatly benefit the research program; the investigators could consider taking an intensive course on ontology or include an ontology expert as a collaborator on the project. The use of ontology such as ERE is also limiting because it is not in the public domain. This limits the overall research productivity as well as the venues where the research may be published.
Intelligent Information Management for the Battlefield
In this research project, the investigators are using a policy-based approach to determining the value of information in order to reduce information overload. This is a significant project as it represents good science and is research that has potentially important impact. The research focus is transitioning from the technical side to the human interaction side. The challenges are related to bringing in appropriate information visualization capabilities and getting sufficient and appropriate experimental subjects. This could be facilitated by greater collaboration with those in information visualization (possibly in ARL and beyond) and with access to subjects earlier in the design process. Such an approach could be a template for other strong projects in the HII area.
Situated Human Modeling Via 3D Eye-Gaze Information
This project aims to improve the ability of autonomous agents to communicate with humans. The scope of the research was intentionally narrowed to focus on issues of using on-head eye-tracking glasses to allow autonomous agents to build models of humans. This narrowing of focus has enabled
progress to be made, but at the expense of limiting advances in the basic science. The key challenge lies in developing deep theory about humans—specifically, a theory of the mind for HII. This could be facilitated by collaboration with cognitive scientists working in this area.
Each of the aforementioned research projects is an example of HII science that could make an impact in the field. In each of these projects, the researchers demonstrated a sophisticated understanding of the problem, were familiar with some of the relevant literature and were able to draw upon it, and could clearly articulate either the problem being addressed or the value of the results. Each of these projects can be shaped to address an issue in at least two of the three main thrusts for the HII area. Having projects that address a combination of thrusts is critical in developing the overall area and the underlying science.
Challenges and Opportunities
There are a number of potentially important theoretical concepts being addressed by investigators in HII, such as trust, value of information, and quality of information. However, it was found that these terms were often used superficially and without the investigators having a deep understanding of the underlying phenomena. Relevant literature was often not cited, and there was a lack of understanding of the breadth of sciences that already use the words. In many cases, the investigators were unaware that the concept had been conceptualized in different and incompatible ways by different sciences. To make progress in this area, it is important that these key concepts are not just understood at a conceptual level but that they are based in theory and formalized in a consistent way. An example of how to do this well is the work presented to the panel on assessing the value of information in information graphics understanding.
It is important to recognize that HII needs to draw on a vast array of social sciences—including sociology, anthropology, organization theory, psychology, cognitive science, economics, political science, and communication theory. It is unreasonable to expect any one scientist to be aware of fundamentals in all of the social sciences, and collaborators from those fields are necessary. It is important to create a research environment where HII investigators have sufficient access to expertise in the relevant and related disciplines.
The third offset, and HII’s relevance to that, provides a tremendous opportunity and the framework in which HII could be particularly valuable to the Army. The breadth of activities is exciting but will require careful management. HII is at risk of having a small number of researchers superficially involved in many areas of research involving humans but not going in depth sufficiently to have a strong scientific impact. An example is autonomous systems—specifically, where could the locus of decision making in an immersive environment reside? To date, there is limited work in this area, and a question is whether greater attention is required in this problem area. Another example is determining which aspects of Army activities are significant—Could HII cover the complete suite of Army activities? Or could the focus be constrained to combat? Developing a challenge problem to focus research for 3 to 5 years would actually enable higher productivity and help focus this area. To be successful in this regard will require leveraging the larger scientific community beyond current collaborative technology alliance (CTA) collaborators and researchers brought in through ARL West.
To accelerate progress, HII needs a showcase problem. Currently, HII is the collaborator on a variety of topics, watches very few of the relevant subareas, and is not regarded as a leader in a given domain. Being the lead on a problem that crosses all three of the identified thrusts is critical for scientific advancement and organizational survival. Possible areas for such showcase problems include developing a theory of mind for HII or developing a theory of cyber-embedded social action. To enable these research efforts will require HII to support a distributed laboratory. A fully informatted distributed space is required—one that is structured to accept information from external information sources, including
social media, traditional media, and embedded sensors from different technologies. The ability of the laboratory to interact with this information, with laptops, personal electronic devices, digital agents, and so on, is important for facilitating studies involving groups (not just crowdsourced). Some features of such an environment already exist within the open campus philosophy and the network science research laboratory may serve an important role in this context. The emphasis, however, needs to be on a facility that supports interactions among groups of humans and robots in a truly informatted environment. The HII research program and research team would benefit by aligning with a showcase conference. Regular participation at such a conference will build links with a broader community of researchers to collaborate on problems of interest to the Army. One possible conference is SBP-BRiMS,2 which already has some military participation, is in the information science area, and draws participants from computer science, engineering, and social science.
ARL needs to focus on developing the right human capital for the HII program. Currently, the HII team is dominated by individuals with computer science backgrounds. Counting both collaborators and members of ARL, there appear to be eight computer scientists, three electrical engineers, three psychologists, and two general social scientists. There needs to be an emphasis in training or recruiting personnel with expertise in sociology, cognitive science, anthropology, or social psychology. Without some in-house expertise, it is challenging to evaluate potential collaborators or watch developments in relevant areas. It is also important to keep abreast of new developments in social sciences—there are a growing number of quantitative, mathematical and computational sociologists, anthropologists, linguists, and cognitive scientists that could bring the needed expertise for HII to be truly impactful. There is insufficient access to that the social science community through the current CTAs or ARL West. In the same context, it may be helpful to facilitate more training for the existing computer scientists and engineers in the relevant social science subareas through short courses and executive education programs. Early career professionals working on transdisciplinary projects also need to receive mentoring from the appropriate branches of computer science and social science. The use of mentors from outside of ARL is also a possibility. While there is evidence of support for training in new computational and mathematical skills, there is less focus on training opportunities in the social sciences. There are mechanisms in place to support collaboration, and these need to be actively pursued to support HII activities. An example of this need is in information visualization, a field where in-house expertise exists at ARL but does not appear to be appropriately engaged in the HII effort. There is evidence that the leadership supports collaboration, and additional attention is required to develop a platform that helps identify partners for collaborative endeavors.
For HII’s work to make an impact on the larger scientific community and to be of greater value to Army needs, it needs to have strong theoretical underpinnings that integrate computational and social theoretical constructs. An example of such an approach is the thrust on social computing. The current nascent work in this area is focused on social computing from a largely computer science perspective. This is unlikely to serve HII well because it would retain the existing computer science and engineering emphasis. A transdisciplinary approach that adds relevant and diverse social perspectives would produce the desired integrated theory—which is of greater value to the Army.
This is an exciting new area of research with huge opportunities and strong relevance to the Army. Researchers are enthusiastic about working in this transdisciplinary area but need time to get new training in the human and social sciences arena. The HII research field is vast, and to build prominence, it will be important for ARL to focus and to leverage the expertise in the larger community.
___________________
2 SBP-BRiMS is an International Conference on Social Computing, Behavioral-Cultural Modeling, and Prediction and Behavior Representation in Modeling and Simulation.
CYBERSECURITY
Accomplishments
The research projects chosen by ARL seem to be well within the capabilities of the laboratory. Where projects have need for different or more resources than provided by ARL, researchers have shown considerable initiative in teaming with other research institutions. As noted earlier, continuing access to real-world operators and operational data is the greatest resource ARL has in cybersecurity research. However, given their unique capabilities, there are other important cybersecurity areas that ARL is uniquely positioned to address. These are discussed in the challenges and opportunities section below.
The ARL cybersecurity researchers were scientifically competent and in some cases impressively so. Given the scarcity of cybersecurity professionals, this is noteworthy. There is good evidence of collaborative work with other organizations, particularly in the collaborative research projects detailed in the poster session. More importantly, ARL researchers were generally aware of other research in their areas of work. Given the wide range of organizations conducting cybersecurity research—government laboratories, University Affiliated Research Centers, academia, and commercial organizations—this is a difficult, but necessary task.
Stylometry Authorship Attribution for Source Code and Binaries
In cybersecurity, attribution of perpetrators and weapons is a difficult and elusive goal. There have been numerous research efforts to attribute a cyber weapon to a family of cyber weapons of common origin. Most of these efforts have been aimed at attributing source code to common origin. What makes ARL’s research novel is being able to carry the work over to binaries. Although the results are preliminary, ARL’s research shows promise and is unique among current approaches.
Interrogator Intrusion Detection System
Interrogator represents the best in leveraging ARL’s unique dual role in both cybersecurity research and operations. Interrogator is a deployed, widely used tool that has continued to evolve based on feedback from operational experience. Continuing work on Interrogator is encouraged for its contribution to operational excellence, but also for the threads of other research opportunities it uncovers.
Resource Conserving Signatures
The use of Bloom filters as a mechanism for compressing a large corpus of signatures in a network-constrained environment is ingenious. Preliminary testing of the approach with known data sets has shown an order of magnitude bandwidth savings in mobile tactical networks with false positive rates of less than 1 percent. This project is an excellent example of using ARL research to solve a problem that is particularly present in the Army warfighter environment. The approach is novel and the results encouraging. Several posters presented in areas categorized as threat defeat and resilience were of high quality.
Threat Defeat
The overall quality of research in areas related to threat defeat was very good, particularly in malware detection and containment. Even though caution needs to be exercised to avoid over reliance on
automated tools in the threat environment in which the Army operates, the ongoing research done in automated detection is considered to be particularly noteworthy.
Understanding the Cyber Threat
The work related to the use of extremely short rules to detect malicious activity and the aforementioned stylometry research to attribute malicious binaries was considered to be excellent and at the leading edge. There were research projects in this category that needed better definition, such as the work related to personality traits and behavior analysis research into aspects of cyber actors and victims. Also, as previously noted, several research projects in this group exemplified the best of the marriage of research and operational environments.
Automated Detection of Hostile Activities
Research related to automated detection is important and impactful. The work related to using Bloom filters to reduce signatures in a constrained network environment is exceptional. Also of particular importance was the research project in cyber-physical security. The security of cyber-physical systems is of immense importance to the Army and the country at large—especially considering the proliferation of devices within the Internet of Things. ARL needs to consider broadening its research efforts in this area, and the ongoing work appears to be a good start. The work in machine learning applications was well organized, but there is skepticism as to how useful the results will be. As an example, the work on grammatical and machine learning used machine learning techniques to address a computationally hard problem. Higher-than-ideal false positives were observed in the approach. Similarly, the work in machine learning for intrusion detection in mobile tactical networks was found to have limited applicability, constrained by the quality of the training set. Conversely, the result from ontological and cognitive modeling was promising and represents a path forward for extension to different threats and network features.
Prevention and Defeat of Hostile Activities
In contrast to the applied nature of research of the previous subsection, the work on preventing and defeating hostile activities was of a slightly more fundamental nature. The work was of good quality and well scoped. It included projects related to evaluation of platform migration defense as a strategy to enhance security of cyber systems. Using simulations, researchers were able to show the influence of platform migration rate and platform diversity on success in defeating attacks. Another project looked at reducing network security problems to graph problems that can be addressed through efficient O (poly) algorithms. Many of the results reported are from early stages of investigation but are nevertheless encouraging.
Resilience
The research projects in resilience appear to have a similar focus to work done at other institutions, but with an Army network constraint. It would have been helpful for the panel to see a typical Army network architecture to provide the context in which this research is applicable. Some of the efforts involve developing tools such as decision support systems that are currently deployed, and use operational feedback to help guide additional research. For example, one decision support system is a red/blue team organization that is called upon to assess the cybersecurity resilience of an organization and provide
training to new members. The consolidated virtual inspection program is a methodology to ensure that an existing cyber system is up to date with respect to revision levels of installed systems and existence of appropriate systems. The work facilitates inspection without the physical presence of inspectors and concomitant savings in labor, and has shown promise for widespread implementation. Even though these research projects are looking at deployed technology, it is obvious that applied research to drive the continuous evolution of these capabilities is a worthy goal.
Perhaps the most innovative effort in the resilience area is information security continuous monitoring. Cybersecurity needs ongoing monitoring and continual database updates to meet its operational requirements. Consequently, the database created would be able to scale over time as well as incorporate new metadata. This is required to stay one step ahead of adversaries. Information security continuous monitoring has most of these characteristics. However, the database is currently over the 2-petabyte range and needs to scale to exabytes and beyond. This would require research into scalability of the software and the mathematical algorithms used, as well as the architecture of the underlying hardware platform.
Challenges and Opportunities
For cybersecurity, it is important for ARL researchers to clearly articulate what is different in their environment as compared to a commercial environment. While there is considerable overlap, the differences are important and include issues such as bandwidth constraint, hostile operating environments, and ad hoc joining and departing of networked equipment. These differences could define ARL’s cybersecurity thrust. With the emerging Internet of Things, commercial networks are becoming more like the Army’s networks.
As noted above, research into the security of cyber-physical systems could be a priority of ARL. Many of the cyber-physical systems in the Army are unique to the Army, and no one else is likely to do research into their security—except perhaps an adversary. The security of these systems is of critical importance to the Army and the nation at large. More research focus is needed on defense against insider attacks. In this context, it is important to recognize how Army insider threats differ from those typically encountered in other organizations. The Army scenario also includes tactical overrun situations, replay attacks, and inside network malware detection. Again, this represents a threat almost unique to Army needs.
ATMOSPHERIC SCIENCES
Accomplishments
The research projects being conducted focus on challenges related to the collection and processing of environmental data from nontraditional observing platforms, the imaging and sensing of aerosols and objects in the battlespace environment, understanding complex atmospheric flows through a combination of field observation experiments and model development, and application of the observations into enabling technologies to enhance renewable energy use. Several of the projects also seek to adapt technology developed for other applications into the domain of atmospheric sciences. Observational field experiments and the proposed deployment of a meteorological sensor array (MSA) at the White Sands Missile Range (WSMR) complex are ambitious projects that seek to push the state of the art in characterizing atmospheric boundary layer (ABL) flows in different types of complex terrain and to eventually model them realistically and accurately. The overall scientific quality of the work is good, and comparable to research conducted at successful university, government, and industry laboratories. Researchers were certainly familiar with the underlying science and cognizant of research done elsewhere. In most cases, the researchers were aware of the potential challenges associated with their projects.
Long-Range Adaptive Passive Imaging System—Development and Field Results
Research on the long-range adaptive passive imaging system is focused on developing approaches to enable enhanced imaging. Specifically, the work seeks to account for the effects of turbulence on images so that objects may be positively identified at significant distances. The approach included experiments to validate and verify theoretical models that have been deployed in a prototype system. The proposed techniques appear to yield a doubling of the range capability, even under degraded conditions. This high-impact result for Army operations has an exceptionally high opportunity to improve in-theater tactical decisions. Collaborations exist with both national and international (e.g., Canadian) defense-related agencies. Even though customer-initiated with a clear application focus, the project entails significant basic science.
Crowdsourcing of Meteorological Data
One of the major challenges facing the Army of 2040 is the characterization of battlespace environments in remote, data-sparse areas where conventional observations are lacking. This research project ties directly into intelligence preparation of the battlespace and aims to leverage future sensor technologies that will be directly embedded within combat systems and on soldiers themselves. Such sensors can automatically collect and transmit real-time environmental data via command and control networks to servers located at a forward operating location that does not have the ability for “reach back” to conventional weather providers, either in rear echelons or in the Continental United States. The project is focused on the back end of the collection process (i.e., server) but utilizes technologies and crowdsourcing approaches that have previously been developed (front-end technologies) in order to obtain data inputs. The server-related research is in its early stages, but results have already shown the high value of the techniques in overcoming data-handling vulnerabilities. In addition, the front-end processes have resulted in one patent and two patent applications. This project is linked to distributed weather-decision support modeling, which concentrates on utilizing multiple sources of environmental information to produce automated guidance on optimal use of Army weapon systems on the battlefield.
Raman Spectroscopy of Atmospheric Aerosols
Given the complexity of the environment and the adequacy of current sensing technologies, there are considerable challenges in detecting chemical and biological agents on the battlefield. A dirty atmosphere can cause signal attenuation for certain weapon-tracking methods (e.g., lasers). The research is focused on developing a Raman spectroscopy-based approach to enable the characterization of supermicron-sized particles and uses an aerosol Raman Spectrometer (resource-effective bioidentification system [REBS]) developed at Battelle. Code was developed so that the REBS could be used as the sampling system for an aerosol Raman spectrometer. Proof of concept has been demonstrated through the identification of bio-like substances, organic substances with C-H stretches, and inorganic salts. The analytical tool was also modified to enable the identification of temporal changes in aerosol constituents. Plans exist to field test the Raman spectroscopy system, leveraging observations from the planned MSA.
The aerosol Raman spectroscopy technique is expected to have significant advantages over fluorescence-based systems for the analyses of biologically based aerosols. Specifically, the Raman-based technique enables positive identification of the chemical composition of bioparticles while simultaneously measuring inorganic aerosol particle composition, a process that is not available with fluorescence-based systems. The most established optical technique that is currently used for the analysis
of particles is Fourier Transform Infrared (FTIR) spectroscopy. Raman spectroscopy-based technique has the potential to extend the capabilities of the FTIR techniques by enabling the analyses of particles with compositions that either exhibit high absorption in the infrared range and therefore interfere with FTIR analyses, or may allow for the analyses of particle constituents that are infrared inactive. The Raman-based system has the potential to advance near-real-time optical analyses of particle composition. This work may be especially applicable to and useful for the analyses of bioparticles.
A paper on this work has been accepted for publication in a peer-reviewed journal, and four presentations at national conferences or universities have already been made. These dissemination efforts are an excellent indication of the larger impact of the work.
Advances in Optical Trapping of Aerosols
Real-time detection and characterization of atmospheric aerosol particles in the 0.01 to 10 µm range is an important capability for protecting soldier health and maximizing the effectiveness of battlespace aerosol particle-sensing capabilities. This research has produced highly innovative methods of capturing these particles using laser-trapping approaches. These methods allow real-time analyses of individual aerosol particle composition and optical scattering properties using laser-induced fluorescence and scattering measurements.
Three-Dimensional Polarimetric Imaging
This project exploits recent advances in the development and production of long-wavelength infrared cameras with polarimetric capabilities for real-time image analyses of an object’s Stokes parameters. The project currently employs commercially available 7 to 11 µm-band cameras. The initial application has demonstrated successful facial recognition capabilities, even in the presence of heavy makeup. Additional potential applications include complex welding inspections and other materials evaluations.
Phase-Ratio Imaging for Anomaly Detection
This research project is based on a collaboration between ARL and Ukrainian planetary remote sensing researchers. The research has revealed that evaluating the phase ratios of objects in two successive visible images with slightly displaced viewing angles can enhance the detection of surface disturbances. The work is in a preliminary stage and has potential applications in the enhanced detection and recognition of buried mines or improvised explosive devices (IEDs).
Mountainous Slope Transport and Diffusion
Mountainous Slope Transport and Diffusion (MASTODON) is a basic research project that entails a microscale atmospheric observational field experiment that represents a follow-on to the mesoscale field experiment program known as the Mountain Terrain Atmospheric Modeling and Observations (MATERHORN).3 That project, funded by DOD as a 5-year Multidisciplinary University Research Initiative, is described as the most densely instrumented complex-terrain field experiment in history. MASTODON addresses fundamental scientific questions regarding characterization of ABL flows in
___________________
3 H.J.S. Fernando and Collaborators, The MATERHORN: Unraveling the intricacies of mountain weather, Bulletin of the American Meteorological Society 96(11):1945-1967, 2015.
complex terrain under slope heating and cooling conditions, which will be used to improve both ABL and mesoscale numerical weather prediction (NWP) models needed for characterizing the battlespace environment in such areas.
One of the ongoing issues in ABL flow characterization is that traditional slope flow conceptual models under slope heating and cooling conditions do not agree. A number of researchers have investigated this and related scientific problems using a combination of laboratory-scale, thermally driven anabatic flow separation simulations, analyses of prior laboratory upslope flow velocity scaling experiments, and MATERHORN field observations. The laboratory and field data acquired support an upslope flow velocity scaling proposed by Hunt et al. (2003).4 This is a very promising result that will also have implications for ABL modeling.
There is a large international contingent of researchers, some of whom were involved with MATERHORN and will continue their collaboration in the next field experiment, set to begin in Perdigão, Portugal, in 2017. The Portugal site is unique in the configuration of the terrain with two parallel ridges in relatively close proximity. The experiment is being funded by the European Union as a wind-power exploration investigation and by the National Science Foundation as an investigation of microscale complex terrain processes. ARL has been involved in MATERHORN and will play a prominent role in the Portugal experiment. All of these efforts are pointing toward the technical communities’ involvement in the MSA system after its standup at the WSMR.
Distributed Weather Decision Support Modeling
This project can be considered as a front-end component of the crowdsourcing project described previously. This part of the project involves the development of a weather input system as well as a weather-based model on a portable device (e.g., a smartphone). This research and tool development is relatively mature, and has already been transitioned to other DOD agencies. The level of collaboration and technology transition is a clear indication of the profound impact that ARL’s weather decision support modeling and associated research has already had within DOD. A further indication of the novelty of the work and broader community impact is evident by the patents that have either been applied for or already granted. These patenting activities are impressive and serve as a clear indication of the exceptionally high level of use-inspired research that has been undertaken.
Weather Impacts on Microgrid Renewable Energy Ramping Event Modeling
A combined field experimentation and modeling effort is under way to enable the efficient use of renewable energy by incorporating weather data into the decision-making process. This novel and highly promising effort is specifically focused on a microgrid scale. Fieldwork was already undertaken in conjunction with collaborators to test the impacts of solar utilization and solar flux measurements on power output from photovoltaic systems. These tests were conducted using large solar panels and enabled the development of operational plans for additional tests as well as appropriate correlations of collected solar flux data for power simulation modeling with the incorporation of meteorological data. Well-conceived plans exist to test additional, flexible solar array systems and to correlate environmental parameters with power output. The work is recognized for its potential impact as ARL investigators have been invited to participate in multiple related scientific panels.
___________________
4 J.C.R. Hunt, H.J.S. Fernando, and M. Princevac, Unsteady thermally driven flows on gentle slopes, Journal of the Atmospheric Sciences 60:2169-2182, 2003.
The work has clear, positive benefits for Army operations, especially in reducing the frequency for and potential security challenges associated with liquid fuel re-supply convoys, which have been vulnerable to enemy attack. Additionally, the work addresses a critically important, evolving area of renewable energy research for the broader U.S. and international community, especially in under-resourced or remote areas of the world.
Microscale Modeling Focused on Dense Urban and Complex Terrain Domains for Future Army Battlefields
The need for environmental characterization at the microscale level (for the Army, this means spatial scales of 1 to 100 m and temporal scales of the order of minutes) exceeds the capabilities of today’s mesoscale NWP models. Previous work in this area has resulted in a diagnostic wind flow model known as 3DWF, which has no thermal forcing but can be run on horizontal scales small enough to include the presence of individual buildings. The 3DWF has been transitioned to several operational users but lacks more sophisticated physical processes such as thermally induced slope wind flow.
The focus is on developing a suite of atmospheric boundary layer environment (ABLE) models to predict mean wind, temperature, moisture, and turbulence over urban and complex terrain in near real time. Three following candidate models for ABLE are under consideration: (1) ABLE—Computational Fluid Dynamics, a finite volume model where the emphasis has been on the treatment of lateral boundary conditions in the model; (2) ABLE—Vortex Filament Scheme, which has no thermal forcing but the use of a “thermal bubble” perturbation appears to show some promise; (3) ABLE—Lattice Boltzmann Method (LBM), which employs LBM to retrieve macroscale features from the motions of individual molecules. The ABLE-LBM model can account for different heating perturbations and distributions of stability and can model the flow around complex terrain and buildings. It has undergone preliminary validation by use of elementary benchmark flows to determine if it reproduces the expected flow results.
Use of a New Hybrid Nudging/Variational LAPS Approach for WRF-ARW Initialization
The operational requirements for this project are driven by the need for high-resolution battlespace environmental characterizations in remote areas where conditions are not practical for conventional “reach-back” strategies to obtain weather forecast information from rear-echelon or Continental U.S. locations. These environments can be quite complex and variable—from rugged terrain to urban canyons—and provide the backdrop for the hybrid nudging/variational local analysis and prediction system (LAPS) project. The model chosen for development is a community-developed mesoscale NWP model known as the weather research and forecasting (WRF)—advanced research WRF (ARW). The WRF-ARW system, as modified by ARL, is known as the weather running estimate—nowcast (WRE-N); the system is expected to run over small geographic regions (100 to 500 km on a side) for very short forecast timeframes (1 to 6 hours). An additional requirement is for the WRE-N to be run every 30 minutes, making the assimilation of indirect and asynchronous observations from a variety of sources an issue of highest importance.
The investigators are working with other NWP modeling groups in the National Oceanic and Atmospheric Administration and the university community to address the unique requirements of assimilating such data in the computationally constrained environment of a forward operating base. The computational constraints do not allow the employment of more state-of-the-art data assimilation techniques, so the group is examining combinations of already-proven assimilation methods such as observational/ analysis nudging and variational data assimilation. Another unique aspect of this project is the approach
that variational LAPS uses, which proceeds from larger to smaller spatial scales and is computationally faster than more advanced methods such as Ensemble Kalman Filtering and 4-Dimensional Data Assimilation.
Several different variations of the assimilation technique, including variational LAPS alone, nudging alone, no assimilation, and combinations of variational LAPS and nudging or the hybrid scheme, were assessed in numerical simulations using existing recorded data. The hybrid scheme showed some promise when compared to the other initialization methods by better capturing several areas of deep moist convection, but the approach is not without its flaws.
Assessment of Storm-Scale Convective Forecasts
One of the most important considerations in battlespace environmental characterization and prediction is determining the quality of predictions. While commonly used statistical forecast verification scores capture important information, they do not provide insights into a model’s ability to capture specific weather phenomena accurately. It is well known that coarser-resolution NWP models can actually have better statistical verification scores than finer-scale models due to the “smoothing out” of fine-scale features that the higher-resolution models may capture, but they are deficient in predicting the correct location and/or time frame. The research seeks to address this deficiency by using an object-based verification scheme based on the method for object-based diagnostic evaluation (MODE), developed at the National Center for Atmospheric Research.
The MODE technique was tested against data for a 3-week period of High-Resolution Rapid Refresh (HRRR) forecasts in 2010. The MODE technique verified deep moist convection forecasts from HRRR by comparing simulated vertically integrated liquid (VIL), a commonly used radar-based diagnostic for deep moist convection, to observed VIL available at 15-minute intervals during the verification period. The VIL is a discontinuous field, and a threshold value was used to employ the MODE technique. The results of the verification study revealed some interesting characteristics about WRF’s ability to develop and maintain deep moist convection and how these differed by time of day, size of convective system, and geographic region. The plan is to adapt this technique to verify other atmospheric parameters from other model forecasts such as WRE-N. This is noteworthy but not without attendant challenges.
Design of Experiments for Verification and Assessment of Fine-Scale Atmospheric Forecasts
This research project seeks to determine the optimum configuration for running a mesoscale NWP model based on a statistical technique that has been used in other modeling and simulation applications. At present, the choice of numeric and physics options when configuring an NWP model run is subjective and largely based on anecdotal accounts of which schemes work best in certain geographic regions and climatic conditions. The research seeks an approach to quantify this information in order to produce a type of NWP model configuration “playbook” that can be employed in theater by deployed personnel who are not modeling specialists but nonetheless need the information that models such as WRE-N can provide. There appear to be potential synergies between this project and the object-based verification method discussed above, since the model forecast verification is key to proper employment of this statistically based model configuration scheme.
Sub-km Resolution WRF-ARW Modeling Studies During MATERHORN Intensive Observing Period 6
This is a project with connections to MASTODON and the two previous projects. In this investigation, WRF-ARW was run on a 0.5-km horizontal grid over the MATERHORN field experiment region during intensive observing period 6. The results of the model run were mixed. As the model was run without data assimilation (cold start), the complex terrain flow features during the first diurnal heating and cooling cycle were not captured adequately, but features such as the Playa breeze were partially captured during the subsequent model diurnal cycle. However, other types of complex terrain-induced flows, such as slope flows above the surface, were not reproduced accurately in the model simulation. Future plans include the adoption of a large eddy simulation transition approach for running the model at smaller horizontal grid spacing.
Challenges and Opportunities
In the long-range adaptive passive imaging system project, the broader fundamental scientific applicability and path for future project development are somewhat unclear. Indications of how the proposed effort might be integrated into other Information Sciences Campaign efforts would be ideal and may lead to further opportunities to extend the longer-term value of the existing effort.
Several opportunities exist to expand the Information Sciences Campaign’s efforts in the crowdsourcing projects. For example, questions of how uncertainty in data, data analyses, and subsequent error propagation would be handled are important in the development of broadly applicable research. Articulating how these issues of uncertainty and quality control would be handled or, alternatively, how fundamental research would progress in these areas is a larger issue that could be more thoroughly addressed in this research effort. Additionally, an ongoing challenge to the crowdsourcing effort is the identification, calibration, and deployment of compact, low-power, and robust environmental sensors that will fill its front-end data stream. There are numerous unknowns that may potentially exist with sensor quality and functionality, thereby resulting in data quality challenges. Early collaborative efforts in the simultaneous development of the sensors and data acquisition and modeling techniques may help to improve compatibility between data acquisition and data handling, as well as enhance quality control. Collaborative engagements are encouraged not only across ARL and other DOD agencies, but also with universities and private industry.
The Raman spectroscopy of atmospheric aerosols project is promising research but not without questions. First, it is unclear how this method compares to existing particle analyses techniques. In order to gain broader scientific acceptance of a Raman spectroscopy-based system, ARL staff are encouraged to engage in a thorough comparative study using other aerosol characterization methods and systems. Future work to expand analyses down to the submicron size range may present an additional opportunity to extend the applicability of the Raman spectroscopy-based analyses of particles. In particular, focusing on nano-sized particles would enable novel research in the area of nanoparticle toxicity. The latter is an important concern for the Army. Additional information on how complex matrices (that would be more representative of an actual battlefield environment) would impact the analyses can further extend the applicability of the project. A unique opportunity exists to further extend the work to characterize the oxidation of organic particles, and specifically the shifting of C-C or C-H bonds to C=O and COH bonds, to enable the identification of oxygenated compounds that are typical of secondary aerosol formation.
The advances in the optical trapping of aerosols project is currently at the successful proof-of-concept level and may very well become a useful laboratory tool for real-time detection and charac-
terization of a whole variety of aerosol particles. Possible evolution into a useful field measurement tool for force protection and situation awareness will require strong electro-optical, mechanical, and automated instrument operation and data analysis software advances, as well as instrumentation integration engineering activities.
The utility of the three-dimensional polarimetric imaging technique in the advancement of atmospheric science is not obvious. Over the past decade, NASA has sponsored some work in remote sensing from airborne or satellite-based polarimetric imagers to assess atmospheric aerosol shape parameters, using instruments in the visible and near-infrared spectral bands. While the phase-ratio imaging for anomaly detection technique may be useful in detecting disturbances in battlefield terrain, such as mine or IED burial scars, its utility for atmospheric measurement has yet to be identified.
The MASTODON project is one of ARL’s most ambitious—with potentially the highest payoff. It is linked to the installation and operating of the MSA, which has made considerable progress in terms of equipment funding and purchase since the 2013-2014 review.5 The MSA promises to be a cornerstone for the success of the Atmospheric Sciences Center at WSMR. However, a significant commitment of resources will be needed for continuous operations and maintenance of the sensor array, once installed, in order to attract world-class researchers to the facility to use the data and collaborate with ARL scientists. At the time of the panel’s visit, siting approval had not yet been received from the WSMR leadership.
There are significant leadership and collaboration opportunities for ARL by leveraging the project on weather impacts on microgrid renewable energy ramping event modeling. ARL project personnel are encouraged to continue engaging with the technical community, particularly in terms of incorporating the next-generation innovations in renewable energy technologies. This will enable rapid integration of new system parameters into the modeling effort. Innovations in the devices and techniques may, for example, enable more effective use of a wider range of the electromagnetic spectrum, thereby enhancing solar use efficiency and, ultimately, energy availability and power output. As renewable energy technologies change, the weather-renewable energy modeling tools will need to be flexible enough to adapt. Enough information on the modeling component of the project was not available for the panel to effectively assess how variations in renewable energy technologies would be incorporated into the efforts. It was somewhat unclear whether the project team has a plan to enable additional engagement from the renewable energy side of the project, although it was clear that efforts are under way to establish cooperative agreements with at least one university partner on the weather forecasting side. Cooperative agreements with university and industry researchers who are engaged in renewable energy projects are strongly encouraged. There are also numerous international institutions—particularly in Latin America—that have a strong interest in coupling environmental issues and renewable energy on the smaller scales that the ARL project team is undertaking.
While the use of elementary benchmarks for verification in the microscale modeling project is encouraging, this process will become more challenging as the developers move into more complex scenarios. Additionally, there is the question of how to model turbulence, which is of great importance in transport and diffusion of chemical and biological agents as well as smoke and other obscurants on the battlefield. The investigators mentioned the desire to use wind tunnel experiments for model evaluation, an approach that is to be encouraged. Opportunities may also present themselves in the area of code optimization. The LBM technique lends well to parallel processing, which will be important as the modeling system begins to approach readiness for technology transition. This is a relatively new approach, and close collaboration with researchers in the high-performance computing area is encour-
___________________
5 National Research Council, 2013-2014 Assessment of the Army Research Laboratory, The National Academies Press, Washington, D.C., 2015.
aged. An additional consideration is the use of the MSA data for model verification; the investigators need to take advantage of this unique data source.
The hybrid nudging/variational LAPS project has similar challenges to the microscale modeling project regarding validation and verification. Given the requirement to run WRE-N anywhere in the world, it will be important to have as complete an evaluation as feasible, given that the model cannot be pretested for all locations at all times of the year and in all climatic conditions. There is a great deal of qualitative and quantitative verification required for the hybrid assimilation scheme. For the test case where measurements from the 2013 Moore, Oklahoma, tornado were used for model assessment, the hybrid scheme did produce a better deep, moist convection forecast in comparison to the other methods. In several portions of the model domain, however, it performed poorly; for instance, it did not reproduce the convective system that spawned the Moore tornado itself. This brings up the question of accuracy versus realism. The model results show WRF-ARW has the ability to reproduce fine-scale features of a mesoscale convective system realistically, such as the strong updraft and hail-producing regions. However, for present and future Army needs, the model needs to be able to reproduce such features in the right location and at the right time. Additionally, the assimilation technique needs to be tested in more data-sparse areas and for other output parameters of interest to the Army (e.g., surface temperature, low-level winds). There is a great opportunity to use this modeling system to test the efficacy of future observing systems, such as soldier-borne sensors and various types of unmanned aerial systems, through the use of observing system simulation experiments. The investigators are aware of these challenges and opportunities, and their collaborations with other NWP modeling groups are very encouraging. An additional consideration is the use of the MSA data for model verification, and the investigators are encouraged to take advantage of this unique data source.
The assessment of storm-scale convective forecasts project has potential to add to the atmospheric science community’s knowledge of NWP model verification through the unique approach of phenomenon-based quantitative evaluation. The expansion of the project to other models (e.g., WRE-N) and other parameters will be challenging simply due to the nature of moving from one software system to another. A related issue will be the development of an appropriate protocol for choosing the right variables at the right locations, since it is not possible to prerun the model and verify it everywhere at all times and in all climatic conditions.
The design of experiments for the assessment of fine-scale atmospheric forecasts may be the highest-risk, highest-payoff project that was reviewed by the panel in the current cycle. The technique has never been applied to an atmospheric NWP model, and there is a dearth of literature on the subject of choosing numeric and physics packages in an NWP system based on how each influence the overall accuracy of the model predictions. As they attempt the translation of this proven statistical technique to the atmospheric modeling problem, it will be critical for investigators to reach out to both the operational and research components of the NWP community to get as much information as possible. Successful development of this technique has the potential to influence the operational NWP centers around the world; having a robust set of model-configuration guidance (playbook) for deploying forces to use will be a separate, but equally daunting, challenge.
The subkilometer-resolution WRF-ARW modeling studies project had some encouraging results, but like the hybrid nudging/variational LAPS project, more needs to be done in terms of model verification and potential enhancements that could improve the model forecasts. As stated for the latter project and the microscale modeling project, the use of MSA data for model verification will be crucial in advancing the goals of this research. It is important to get a sense of the ability of the model to reproduce the small-scale flow features that are critical to battlespace environmental characterization and prediction.
An overarching positive observation about the work in this area relates to the amount and extent of collaboration taking place within the laboratory (through the science and technology or S&T campaigns) and (very importantly) with knowledgeable research groups both in the United States and internationally. These collaborative efforts were highlighted numerous times by the project investigators. As an example, in the five modeling-related projects reviewed, collaborations involved federal laboratories and several universities. The observational field experiment projects similarly had notable collaborations with national and international universities.
OVERALL QUALITY OF THE WORK
The research portfolio of the Information Sciences Campaign reviewed in this current cycle was expansive and covered diverse areas. The projects reviewed range from those advancing fundamental science to those focused on enabling technologies and applications. The ongoing projects demonstrated relevance to the future missions of the Army and were generally of good technical quality. There are additional opportunities to further drive scientific innovation through enhanced integration and collaboration across campaigns.
ARL has focused on increasing the number of Ph.D. scientists on the research staff in critical areas of expertise, and this has had a measurable impact on the overall quality of technical work. In the 2013-2014 review, it was noted that efforts in the network science area would be enhanced by bringing greater technical diversity to the workforce—in particular, added strengths in the social and mathematical sciences. This diversity and added strength in the social sciences is also critical to the HII initiative. Among the ARL researchers, there was a good awareness of external research and connections to professional organizations and external research communities; research results are appearing in respected conference proceedings and in archival journals. There is room for even broader dissemination of these results to a larger scientific community. As noted in earlier reviews, the mission-oriented thrust helps differentiate the ongoing research from efforts pursued elsewhere and creates opportunities for impactful technical contributions. The impact of the work can be further enhanced by clear articulation of unique, cutting-edge research questions.
Given the vast range of computing available today, especially for data-intensive work, it is not clear whether ARL has the resources needed to carry on the current work as projects move past the proof-of-principle stage. It would be good to consider whether the resources available, or planned to be available, will be adequate for the next 2-3 years. It will be important to assure with confidence that the following questions are being asked and adequately addressed: Is there a systematic approach to estimating the need? Is there a need for occasional access to very large-scale parallel environments (such as a 10,000 or more node system—that is, a 250,000+ core system)? Does ARL need to develop partnerships with providers of very large scale, data-intensive systems, either with other federal agencies (e.g., NSF or DOE) or with private industry (e.g., Amazon or Google)?
Computing is going through a transition (including post-Moore’s law and the looming end of CMOS scaling, a revolution in data-intensive science, and the promise of quantum computing, which may only be useful for some niche areas but those include ones of great importance to ARL). This transition makes it difficult to make firm long-term plans, but it is essential to develop a strategy to address the possible directions in which computing may go. ARL needs to perform some research to better understand the consequences of these changes for 10 and 20 year strategic plans.
The research portfolio in SIIS includes a growing component focused on enabling technologies, the underlying science, and novel applications to intelligent and autonomous systems. In all cases the intelligent systems issues under study showed clear relevance to the future missions of the Army. The
projects were generally of good technical quality. Importantly, especially among the junior researchers, there was a good awareness of external research and connections to professional organizations and outside research communities; these are important for maintaining and growing the technical quality of the research. The research results are appearing in respected conference proceedings and archival journals. ARL has continued to demonstrate its responsiveness to the general comments and recommendations of the 2013-2014 review and is increasing the number of Ph.D. scientists on its research staff in key areas.
The opportunity for strong technical contributions and for differentiating the work from research conducted elsewhere, as well as enhancing the value proposition for the Army, lies in a mission-oriented thrust to the research. A number of projects in the research portfolio have just such a mission focus and the associated constraints (e.g., limits on the volume of prior information or on available network bandwidth), and these serve as a clear driver of the technical direction of the work. Other projects would similarly benefit by a focus on those areas where the special needs of the Army are not addressed in the basic research agenda being pursued outside of ARL, including the development of technology products related to this research.
The research thrust in sensing and effecting includes projects that address emergent needs of the Army. New theoretical advances and resulting tools have led to a rapid evolution of technology in this domain, and the 3- to 7-year horizons for ongoing research projects at ARL are quite reasonable. The research was generally of high quality, with a focus that is unlikely to be pursued by researchers at universities or at other federal or industrial research laboratories. As an example, recognition of ease of disruption and cost of errors (e.g., human life) needs special attention; field data become critical to the solution of this problem. The researchers generally demonstrated a good understanding of the problems being considered, were able to provide an appropriate statement of the research problem, and pursued appropriate methodologies. They demonstrated awareness of the state of the art and of the related research pursued elsewhere. The facilities required to support the research, including both instrumentation and the computational tools, were adequate.
Research projects in the area of networks and communications were generally of high quality and relevant to the needs of the Army. The research results are being published in archival journals and in respected conference proceedings. The recent relaxation in rules permitting travel to relevant scientific conferences will have a continued positive impact on the overall research environment. The research portfolio in NC has a focus on both theoretical research and on applications and technology demonstrations. The portfolio would have enhanced impact if, in addition to narrow and focused projects that seek answers to very-well-defined scientific or technical questions, it also included projects that are broader in scope that address a specific Army need. In general, researchers in the portfolio demonstrated good awareness of related work in other research communities. There was, however, in some of the interdisciplinary research projects, uneven awareness of the literature and of accepted approaches in constituent disciplines. Additional focus on formulating relevant research problem statements would be beneficial for these interdisciplinary research endeavors.
The research focus in the area of HII is very recent and includes projects that largely involve the work of collaborators. Existing projects in related areas have been transitioned and refocused on HII only recently. It is, therefore, too early to provide a definitive assessment of the overall scientific quality of the research. The work reviewed was comparable in effort, scope, and creativity to much of the work in universities and government and industry laboratories. However, with the exception of the projects identified above, the research that was presented was not well connected to much of the existing work that is being pursued elsewhere. Many researchers were unaware of the human-centered challenges, the relevant social, psychological, and cognitive theories, and the associated methods. They had difficulty articulating specific human-centered research questions and associated challenges. Many of the
researchers were unaware of the relevant theories and methods, and tended to make assumptions about human behavior based on personal experience rather than sound and verified theories. In many cases, the posters presented to the panel did not do justice to the research. They often did not provide information on the fundamental research question, the pertinent state of the art, whom the research would impact, and the key results. In other words, the posters rarely answered the Heilmeier questions.
The overall research program in cybersecurity was well aligned with the current and future needs of the Army. Notable progress has been made since the last review. The overall quality of research was mixed with some of the projects rated as truly excellent. The goals, plans, and approaches of the various projects were notably well defined, and the methodologies employed were appropriate for the research undertaken. The research projects appeared to be well conceived with well-defined goals and realistic projects plans. Both the posters and the presentations were generally clear and understandable. In particular, the projects seemed to be adequately scoped to the resources and available time. As expected of such a diverse collection of research topics, there were a wide variety of methodologies and scientific approaches evident in the project presentations. The majority of the projects showed suitable scientific rigor and practice. However, the methodologies in some of the research utilizing psychosocial aspects of adversary actions seemed ill-defined and unconvincing.
In work related to atmospheric sciences, the overall scientific quality of the research was good and comparable to research conducted at successful university, government, and industry laboratories. Good progress that has been made since the 2014 review, with multiple projects moving in the right direction. Researchers were familiar with the underlying science and cognizant of research done elsewhere. In most cases, the researchers were aware of the potential challenges associated with their projects and had given considerable thought regarding ways to address questions in that regard. For example, in the hybrid nudging/variational LAPS project, the performance of both data assimilation techniques is dependent on which problem parameters are chosen, such as the weighting coefficient in the nudging method. The choice of such methods is an art, and the researchers recognized it as such. In the microscale modeling project, the researchers understood the limitations of the various approaches that they are investigating and are pursuing creative and innovative solution strategies to solve a complex flow problem. Furthermore, researchers are exploring methods, such as the vortex filament approach and the Lattice Boltzmann Method, to capture complex physical flow phenomena and attain greater computational efficiency. The amount and extent of collaboration taking place within the laboratory (through the S&T campaigns) and with knowledgeable domestic and international research groups is truly commendable. Several of the collaborative efforts are a direct result of the open campus initiative, which has brought postdoctoral researchers into the laboratory, several of whom have been hired as term employees and are continuing their work and their research relationships with their respective alma maters.
