4
Information Sciences
INTRODUCTION
The Panel on Information Sciences at the Army Research Laboratory is charged with reviewing Army Research Laboratory (ARL) research in the broad areas of computational sciences, information sciences, and atmospheric sciences. A 2-year cycle of review has been adopted for this purpose with the focus in 2015 on reviewing activities in Computational Sciences and on that portion of the Information Sciences portfolio related to work in system intelligence and intelligent systems (SIIS) and in sensing. Research in information sciences related to networks and communications, cyber, and human information and information interaction will be reviewed in 2016.
The panel conducted its review of the ARL Information Sciences campaign on June 17-19, 2015. ARL research in information sciences is focused on developing and enhancing intelligent systems for the analysis of information and knowledge. Included in this approach are technological advances that support information acquisition, reasoning with such information, and activities, including collaborative communications, that support decision making. An important aspect of this work is information assurance and analysis of trust. The overall research effort falls within the broad categories of sensing and effecting, SIIS, human and information interaction, networks and communications, and cybersecurity. The first two of these broad areas were reviewed in 2015.
SYSTEM INTELLIGENCE AND INTELLIGENT SYSTEMS
Research projects in SIIS were presented in three thematic areas: information understanding, information fusion, and computational intelligence. Collectively the work addresses technical challenges in the use of sensors, communication, and computing to provide the soldier with new levels of tactical intelligence and the automated support needed for missions. Areas of specific endeavor include language translation, information extraction, semantic analysis, understanding of human trust networks, fusion of conflicting information, integration of video and text analytics, anomaly detection, reasoning under uncertainty, robotic control and path planning, and models of cognition and tactical decision making.
Accomplishments
The research portfolio contains an appropriate mix of theory, computation, and experiments and is clearly of value for the support of Army missions. The research projects were generally, but not uniformly, of a good quality. The strongest work has been published in elite conferences and archive literature, and additional opportunities exist to disseminate other work at respected venues. It was difficult to identify a unified theme or common basis for the multiple projects, and the principal challenge for each of the three areas of research is to fuse the collection of projects into a coherent whole.
Information Understanding
Research in this theme focuses on the development of critical methods and techniques for transforming data so as to provide useful information to the soldier. The work represented solid but incremental advancement in several interesting and potentially important areas such as language translation, information extraction, semantic analysis, and understanding of human trust networks. Some of the projects appear to have potential for short-term applications in the field. The research staff ranges from experienced researchers with doctoral degrees to students pursuing master’s degrees or internships. Laboratory resources appeared to be adequate to support the research agenda.
The work on temporal information extraction focuses on methods for extracting temporal relationships from text for constructing knowledge networks. The proposed approach is technically solid and particularly well grounded in the available literature. Temporal relations are an important area for future work by search engine companies, and the long-term applications to the Army are clear. This work has been published in a top conference—Association for Computational Linguistics (ACL)—an indication that it is first-rate quality work. The impact of the work is already evident in that other researchers at ARL are making use of this work.
Ongoing research on influence in social networks is closely related to activities in the network science Collaborative Technology Alliance (CTA) and pertains to identifying mechanisms for trust formation in human networks. The effort has emphasized identifying main factors, but the work has not addressed this issue in the context of networks. Most results to date are based on an analysis of data from the online microfinancing company Kiva and other lenders and are related to microfinancing and corruption perception indexes. The opportunities for extending this work to intelligent systems research for the Army were not readily apparent. The project would benefit from additional awareness and understanding of the literature in the area of online trust.
Also stemming from work related to the network science CTA is research related to agent-based semantic analysis in information retrieval. This research focuses on how to manage information dissemination across constrained channels based on the trade-off between size and accuracy. Rather than posing the task as a constrained optimization problem, the proposed approach implements a fuzzy logic model in order to include model uncertainty or fluidity in the attributes. The work requires a clearer articulation of what additional insights into operational questions are available from the proposed methodology. Further, to establish the advantages of the approach, it would be helpful to compare results against those obtained through more traditional optimization techniques. If such optimization techniques have inherent limitations or are not applicable in this particular problem setting, statement of the problem could better articulate those techniques.
An example of a 6.2 (applied) level research program addressing a real-world need is the project whose objective is to perform language translation in an automated fashion with a goal of enhancing translation capabilities for low-resource languages. The approach involves the construction of a language model based on entropy minimization that selects those sentences that need to be translated by humans in order to construct a best overall training corpus. The novelty of the work centers on methods for constructing the language model.
Information Fusion
Research in this area pertains to the fusion of data from disparate sources to produce timely, actionable information for the soldier. The research has advanced the state of the art in several interesting and potentially important areas such as fusion of conflicting information, integration of video and text analytics, and anomaly detection. The quality of the research was generally good, ranging from archival quality results publishable in top journals to preliminary but promising work. In some instances there was an apparent lack of full awareness of critical literature and alternative approaches. The researchers engaged in the projects were well qualified and possessed a range of experience. While laboratory
resources appeared to be adequate to support the research agenda, there were not enough experienced staff to fully develop the ideas.
Research on estimating credibility by fusing subject opinions was of good technical quality and was disseminated in a reputable conference. This research focused on fusing multiple inconsistent and potentially conflicting information sources. The work treated inputs as propositions with truth-values and used subjective logic in a manner that took into account prior experience with the sources. It was well grounded in statistical machine learning approaches, but it could benefit from consideration of prior work in the human computation literature.
A project that seeks to develop an observer model for helping soldiers determine salient targets draws upon recent advances in image processing and neuroscience. The approach is to train a saliency model on the basis of experiments with experienced soldiers and then adapt that model for automated training of less experienced soldiers. The work emphasizes anomaly detection in saliency models. It is well grounded in previously published work and could be advanced by research considering information from a network of devices.
In a related vein, research is also being conducted into integrating complementary or contradictory information into a fuzzy model in order to determine the value of the information. The underlying idea is to take multidimensional values of information metrics and to project them onto a single dimension. This kind of dimensionality reduction facilitates subsequent sequence-based analyses. It is not clear that such analyses could be performed without unnecessary loss of information. This work would benefit from considering the results of previous research in indexing, feature reduction, and information theory.
Several research efforts in the thematic area of information fusion were in the early stages of investigation. An effort dealing with intelligent information management in the battle environment focused on bandwidth management during distribution of information to end users. The approach is based on linear scoring functions with weights established by the users. Its effectiveness is based on questionable assumptions about human capability, and the investigators might want to consider alternative approaches in the literature that deal with learning user preference and modeling utility in humans.
Another early research effort that could potentially have important impact is based on a hypothesis that learning from video and text could benefit from being done together when video includes or is associated with text. The improvement in the quality of the results comes with the cost of implementing joint learning methods, a much harder problem to solve. The next appropriate steps would include a literature review that examines relevant advances in optical character recognition, video retrieval, and object recognition in videos.
Computational Intelligence
Research in computational intelligence examines the development of intelligence in systems to support highly automated or autonomous operations in support of Army missions. The research efforts have yielded significant advances in several interesting and potentially important areas such as reasoning under uncertainty, robotic control and path planning, and models of cognition and tactical decision making. Researchers engaged in the effort are well qualified, but the team comprises a disproportionate number of early-career researchers. Some of the research efforts lack the staff that would be needed to fully develop the ideas. In particular, projects related to information for robot navigation and message delivery and those requiring expertise in psychology would all benefit from strategic recruitment in targeted areas of expertise.
A research project focused on improvement of visual classification for navigation purposes uses an approach of implementing partially supervised discovery and labeling the navigation domain. The research aims to reduce the effort associated with human labeling of terrain and objects in images for the purpose of supervised learning of objects in images. Over-segmenting the image and then using clustering
techniques on the resulting segments accomplish this goal. Human labeling based on the clusters is much faster but less accurate. However, this loss of accuracy does not significantly degrade robot navigation when using the new visual system. This research is expected to have an important impact and has yielded promising initial results.
Another promising research project uses a semantic vector space for reasoning in the presence of uncertainties. This work seeks to take advantage of two different types of semantic models with a goal of augmenting a curated knowledge base by reasoning through analogies based on statistical representations. Both the ideas and the proposed methodology contain novel elements, but the work is still in an early phase. A number of complexities have to be resolved, including those arising from multiple meanings for words. The work is well grounded in the literature, and the researchers are aware of related efforts in the research community. With continued support and application to meaningful problems, it has the potential for publication in top journals.
Investigations into the concept of robust distributed communication relays for minimizing message latency in a vulnerable and uncertain operating environment are also ongoing, albeit at a preliminary stage. The research is focused on resource-constrained, dynamic environments, where it may be advantageous to use kinetic or nonelectronic means to augment electronic information distribution. The formulation results in a very hard computational problem based on vehicle routing. The approach to handling the computational difficulty derives from an integrated heuristic/high-power computing (HPC) methodology and is being tested in a simulation environment. While the pertinent literature has been explored and cited, additional benefits in computational efficiency may be gained by using recent work on probabilistic spanning trees.
Another example of an early research project investigates strategies employed by humans in situations similar to those experienced by soldiers facing trade-offs between short-term and long-term actions. Initial results in two-player games, which have been obtained using computer agents, demonstrate that there is no dominant strategy. The underlying hypothesis for the work is that these games will provide an environment for investigating how humans move between strategy, tactics, and actions. Although at a preliminary stage, the work addresses an important problem. It is critical to place this research in the context of the literature on human cognition, training, game theory, and computation intelligence in games. Another effort at modeling human cognition was based on varying the volume and velocity of data presented to shooters who have to make a choice of targets. This allows investigation of the effects of data attributes (in classic big data terms) on human ability to make decisions. This topic appeared to be narrow research toward a master’s degree, and its future is unclear.
Ongoing work at a more advanced level looks at the use of an information-gain metric to design paths for autonomous mobile robot movement. Entropy minimization concepts are used to develop optimal routes for mobile ground robots. This work has evolved to an advanced demonstration level; there is a good plan for future work that would make use of recent advances in combinatorial optimization methods and that would include constraints on power and available onboard computational bandwidth.
The concept of episodic memory consolidation and revision, or robotic dreams, was used to identify precursors to events such as the deployment of an improvised explosive device. The research is premised on the hypothesis that this computationally demanding problem is intractable in a real-time computation environment but is more amenable to large-scale background processing using otherwise idle computing resources. While this problem is of significant importance, it was unclear what level of computational efficiency was added through the episodic memory consolidation approach vis-à-vis simply devoting additional computational resources to the problem directly. It was not readily apparent how the approach is distinct from traditional work-stealing methods for background processing during low computational loads.
Challenges and Opportunities
The most significant challenge facing the SIIS program is the development of a coherent and unifying thrust for the research program. Overall, there was little evidence that the whole would be greater than the sum of the parts that were reviewed. The portfolio of research is broad, with many lightly staffed projects. A challenge lies in a better integration of the work to pursue fewer, but more significant and more ambitious projects. Such prioritization is important to focus the limited staff on strategically selected projects so as to more fully develop the ideas. It is possible that the above impression is due to the selection of the topics presented to the panel. ARL leadership noted that projects had been selected to provide a broad sample of the research portfolio rather than to explore its depth. If there are elements that more coherently unify the research projects, then it is important to highlight such integration, not just in the top-level overviews but also in the project descriptions themselves.
Intelligent systems research is increasingly dependent on harnessing vast quantities of data, and the fields of scalable machine learning and big data analytics are advancing rapidly. A number of important techniques and tools have been developed to address these emerging challenges, many in the open source community. Some of the projects reviewed in this report would benefit from understanding and using these new tools. Such an approach is being practiced in research related to cybersecurity and other areas that were not reviewed in 2015.
Some of the reviewed projects were particularly strong because they combined pertinent approaches—for example, the combination of statistical/machine learning [ML] methods with linguistic rules—to address research challenges. Beyond such innovative and opportunistic approaches, some impactful research projects demonstrated clear integration with other activities and research at the ARL and other external entities. Other opportunities for such integration exist and could be more broadly embraced.
New advances in automation have changed the roles of humans and machines in intelligent systems to a degree that humans and machines need to collaborate in task activities. Therefore, the human systems perspective has to be fully integrated into any development to ensure usability and robustness of the results. The ARL presentations did not include the work on human interactions, making it difficult to assess the impact of some of the systems intelligence research.
SENSING AND EFFECTING
Research projects in sensing and effecting covered thematic areas of non–imaging sensors, image understanding, sensor and data fusion, and radar signal processing. There was a strong focus in the area of acoustic sensors that collectively examined new materials applications alongside the development and implementation of better signal processing capabilities. Cross-modal face recognition was an important thrust in the image understanding research. The use of long-wave infrared polarimetry to facilitate discrimination of manmade and nonmanmade objects was another thrust in this research. The sensor and data fusion focus was largely on approaches for dynamically adapting information to situational changes and appears to comprise engineering advances as opposed to exploring fundamental science. Research in radar signal processing was both relevant and of good technical quality, and it focused on signal processing in congested and cluttered environments.
Accomplishments
The research portfolio contains a mix of projects in areas that are well aligned in support of future Army missions. The research was generally of a good quality but not uniformly so. The research projects emphasized applications as opposed to the foundational science. Some of the work is being published in top venues, but researchers could be given additional guidance and encouragement to present their work
at leading conferences and then pursue publication in top archival journals. In general, the research staff were well qualified and demonstrated a good understanding of the important challenges in their work. The long-term research vision provides a natural framework for integrating the individual efforts. However, the connection between these individual activities and this overall framework was not clear. Finally, the time horizon for some of the projects can be shortened (to 3-7 years) to demonstrate proof of concept; the pace of technology advances in the field dictates this necessity.
Non–Imaging Sensors
Research in this domain is primarily on the application of acoustic sensors for Army-relevant tasks, including identifying helicopters through acoustic signatures, to localizing gunfire, and to long-range detection of vehicles or weapons. The Army has a long history of using such sensors, but new technologies and new signal processing techniques could extend the range and the precision of these sensors, reduce complexity, and minimize power requirements.
Research related to the application of new materials in the design of windscreens for acoustic sensors has yielded positive outcomes. Acoustic sensors covering a range of frequencies, from infrasound to ultrasound, were explored in this study. Ongoing efforts are evaluating a range of porous materials and are characterizing their efficacy relative to baseline materials used in traditional microphones (e.g., foam). The work is based on results from partners at the University of Mississippi. The results have been presented at good conferences but have not been submitted for publication in archival journals; internal technical reports document the progress of this work. ARL needs to consider the submission of this work for publication in an archival journal. There is an opportunity to use foundational theoretical knowledge to guide in the selection of optimal materials for this application.
In research related to enhancing the effectiveness of acoustic sensors, the work on identification of helicopters through the use of innovative signal processing techniques has yielded promising results. The signal-processing algorithm takes advantage of the relatively invariant blade speed to determine the Doppler shift and to calculate appropriate motion compensation, which is then used to auto-focus. Papers describing the approach have been published in recognized journals.
Research in this domain is also exploring sensors alternative to the more traditional microphone arrays. In particular, a research project is directed at investigating the use of microelectromechanical systems (MEMS)-based, three-dimensional acoustic particle velocity sensors. A commercially available sensor was adopted for this work, and the ARL research is focused on developing signal-processing techniques better than those provided by the vendor for applications that include localization for small-arms fire and triangulation of continuous waves. These devices have significant potential for reducing complexity, weight, and power requirements. Their robustness in field applications continues to be an area for research and development. Findings of this research have been presented at recognized conferences and documented in internal technical reports.
Image Understanding
The image understanding research program was generally of high quality. The work is being published in high-quality journals and shows a comprehensive understanding of research conducted elsewhere and how the ARL research fits into the broader research landscape. The cross-modal face recognition work represents an excellent example of an appropriate applied research topic for ARL given the mission need for such an approach. The researchers were able to articulate the unique Army needs in these problems and were addressing them.
The research on polarization shows strong potential, and continued collaborations with camera producers would enhance capability. Researchers demonstrated an outstanding ability to summarize the significance and impact of the work. Additionally, the collaboration in this project related to sensor
algorithms for polarization imagery was considered to be a positive outcome of the ARL open campus initiative.
The image understanding work was applied research rather than fundamental. Appropriate laboratory facilities were available to carry out this research. The work was judged to be relevant to Army needs and could ultimately be transmitted to the field. In particular, the manmade object discrimination work recently resulted in a patent, and a transition path is under way.
Sensor and Data Fusion
The research efforts in this area tended to be more incremental engineering advancements as opposed to addressing fundamental questions. As an example, design concepts for dynamically adapting information to the situational changes in the utility of data attributes (e.g., accuracy, latency, reliability, data rate) could be a fundamental topic for investigation; instead, it was treated as a human factors problem rather than as a broader concept for automated management of data prioritization. Given the engineering development focus of much of the work, the researchers did not present current state of the art in sufficient depth, and they appeared to lack access to the field data that would have allowed them to do so.
Research related to detection of vehicles, personnel, or targets is key to Army operations and requires the use of multiple sensor arrays. The fusing of information from many sensors observing similar objects (dependent data) is key to developing better inference. The research focuses on developing a means to fuse correlated information.
The work on dynamic belief fusion applies a sound and seemingly straightforward approach. It integrates the outputs of different object detectors by assigning ambiguity levels derived from previous performance to each detector. An approach of dynamically allocating probabilities based on prior performance has been developed that demonstrates the improvement in detection accuracy over conventional fusion methods.
Another area of research is aimed at developing and enhancing tools to reduce the time between data gathering and making decisions. While important from a practical perspective, the research effort is simply an extension of a fuzzy logic–based tool to assess value of information (VoI) through the use of additional membership functions. The work is in an early stage, and the fundamental technical advances of the approach are not readily apparent.
Across the entire research endeavor, access to actual field data would further enrich the research effort and would help to distinguish the work from that in the outside research community. In the area of dynamically adapting information to situational changes, for example, research in the commercial applications arena focuses on certain performance objectives that do not meet the requirements in the military context. It is important to build in recognition of ease of disruption and cost of errors (e.g., of human life) as explicit considerations in the performance objectives, and field data become a critical component in evaluating such criteria.
The laboratories and infrastructure support were appropriate to support the research; the access to the E/H-Field laboratory is particularly beneficial to ongoing research. The researchers were academically well prepared to undertake the work.
Radar Signal Processing
The research reviewed in this domain was of good technical quality and showed promising results. The research problems were well defined, the methodology was explained at an appropriate level, and the results were well organized. Detecting moving personnel under tree cover on the basis of frequency-modulated continuous-wave (FM/CW) radar is an important challenge. A full-wave approach, realized through the use of a parallel 3D finite-difference time-domain algorithm, is deployed for this
purpose. Likewise, the ability to detect targets obscured by artifacts in ultra-wideband imagery is also relevant and important from an Army perspective. Mission requirements dictate effective performance in the presence of uncontrolled radio frequency (RF) transmitters, and RF interference notch-filtering techniques are widely used in such applications. An alternative approach based on sparse representation and recovery of signal was proposed that does not have the shortcomings of notch-filtering techniques.
The work related to nonlinear radar methods is also timely. It focuses on the use of nonlinear harmonic radar to achieve greater sensitivity across a narrow frequency band. These approaches have been in the literature for a few years, and early applications in the field represent an important next step. Based on these experiments, a 3-7 year time horizon for transitioning to full-scale developments in this area is a real possibility.
HPC facilities at the ARL are a key enabler for this work, especially in applications to detect moving personnel under cover. In a similar vein, the significant use of radio equipment indicates adequate infrastructure to support the research effort.
Two of the projects employed computer simulations to generate input data that were used to demonstrate improved receiver processing techniques. While adequate for purposes of showing proof of concept, the use of real data would also significantly benefit further studies. Simulated data might lack some of the complexities in real data and could lead to false validation of the proposed methodology. ARL might be able to get such data from other DOD laboratories or from the industry. If necessary, simple experimental set-ups could be built to collect such data.
Challenges and Opportunities
The overall technical quality of the research in this area was good, albeit with a greater focus on technology development than on foundational research. Overall, researchers are aware of relevant work in their domain of interest, but this was not always the case. The research problems are important to the Army and are unlikely to be pursued in academic institutions or other government research laboratories.
From an engineering standpoint, the researchers seemed knowledgeable about identifying and adapting or creating new signal processing techniques to the target domain and good at doing this. From a foundational standpoint, however, it was less clear how well the algorithms or ideas would transfer to other domains, or how knowledge would be gained to better understand the underlying physics of the domains
In the area of image understanding, an increased emphasis on developing rich data sets and the quantitative improvement of performance for larger data sets—especially data sets that include mission-relevant variability—would enhance the impact of the program. In particular, a better understanding of how easily an adversary could defeat cross-modal recognition efficacy is of interest. The research emphasis could be expanded to embrace the fundamental science in order to better understand the physics that drive cross-modal features and the information-theoretic fundamental performance bounds. This would enhance the overall quality of the research and produce results with longer-term impact. Additionally, ARL could better connect with and provide intellectual leadership to the broader facial recognition community by creating and curating open, standardized data sets as well as challenge problems for researchers to explore.
In the area of sensor and data fusion, dynamic automatic control of data attributes with operator support represents an important emerging area of research. This is especially true given the explosion in sensor technology and limitations in both communications bandwidth and human–computer interaction. A new opportunity would be to expand the focus to automatic data control rather than limiting control to the operator interface as the only basis for stimulating data control actions. While industry wishes to develop cryptographically solid authentication and authorization, the needs of the Army are more complex. While the authorization to manage parameters of a system resides with a few individuals, provisions have to exist for others to take over or at least prevent unauthorized access by others. One possible area for research is to design systems with redundancy, so that if one sensor net or dashboard is
subverted, it would be immediately detected. Similarly, based on knowledge of the system, the ability to identify system behavior as illogical and possibly compromised would be hugely beneficial in such field operations.
The radar signal processing work could benefit if the engineers working in this area had graduate-level training. There have been significant advances and new tools in the radar signal processing area, and familiarity with these would elevate the research. It would also help to improve the success rate for presentations at leading conferences and publications in top journals. Access to advanced technical knowledge online or by other means can be explored. The work related to nonlinear radar shows promise for immediate transfer to a range of battlefield applications. Finding ways to transitioning this technology as rapidly as possible would be extremely helpful in meeting critical needs in Army operations.
Wireless (and other) communications is a critical part of Army operations today, and its importance keeps increasing. It would be beneficial for ARL to build some capability in this area, at least to support other activities in the laboratory. Lack of this communications technology skills set will increasingly become a disadvantage
OVERALL QUALITY OF THE WORK
The research portfolio in SIIS includes a growing component focused on enabling technologies, the underlying science, and novel applications to intelligent and autonomous systems. In all cases the intelligent systems issues under study showed clear relevance to the future missions of the Army. The projects were generally of good technical quality. Importantly, especially among the junior researchers, there was a good awareness of external research and connections to professional organizations and outside research communities; these are important for maintaining and growing the technical quality of the research. The research results are appearing in respected conference proceedings and archival journals. ARL has continued to demonstrate its responsiveness to the general comments and recommendations of the 2013-2014 Assessment of the Army Research Laboratory by the National Research Council is increasing the number of Ph.D. scientists on its research staff in key areas.
The opportunity for strong technical contributions and for differentiating the work from research conducted elsewhere, as well as enhancing the value proposition for the Army, lies in a mission-oriented thrust to the research. A number of projects in the research portfolio have just such a mission focus and the associated constraints (e.g., limits on the volume of prior information or on available network bandwidth), and these serve as a clear driver of the technical direction of the work. Other projects would similarly benefit by a focus on those areas where the special needs of the Army are not addressed in the basic research agenda being pursued outside of ARL, including the development of technology products related to this research.
The research thrust in sensing and effecting includes projects that address emergent needs of the Army. New theoretical advances and resulting tools have led to a rapid evolution of technology in this domain, and the 3- to 7-year horizons for ongoing research projects at ARL are quite reasonable. The research was generally of high quality, with a focus that is unlikely to be pursued by researchers at universities or at other federal or industrial research laboratories. As an example, recognition of ease of disruption and cost of errors (e.g., human life) needs special attention; field data become critical to the solution of this problem. The researchers generally demonstrated a good understanding of the problems being considered, were able to provide an appropriate statement of the research problem, and pursued appropriate methodologies. They demonstrated awareness of the state of the art and of the related research pursued elsewhere. The facilities required to support the research, including both instrumentation and the computational tools, were adequate.
