6

Sciences for Maneuver

INTRODUCTION

The Panel on Mechanical Science and Engineering at the Army Research Laboratory conducted its review of ARL’s vehicle intelligence (VI) programs—human–machine interaction, intelligence and control, and perception—at Adelphi, Maryland, on July 8-10, 2015. This chapter evaluates that work, recognizing that it represents only a portion of ARL’s overall Sciences for Maneuver campaign.

HUMAN–MACHINE INTERACTION

The human–machine interaction (HMI) research at ARL reviewed by ARLTAB in 2015 was of top quality. The researchers presented studies with rigorous design, evaluation, and analysis, and all used appropriate metrics in their evaluations. They have a thorough understanding of related research and have built collaborations with others at ARL across campaigns and have connected with the right faculty and laboratories in academia through the Robotics Collaborative Technology Alliance (RCTA). The research team has benefitted from an expansion in the postdoctoral program and early-career hiring, which has grown the capabilities and collaborations of the team.

Each individual project defines its own appropriate algorithmic or experimental metrics, but the researchers did not uniformly communicate the higher-level success criteria—that is, what makes their project successful in the context of the larger HMI effort. Such metrics need to be defined at a programmatic level and also be communicated well to the research team; some researchers could not identify the larger goals of the program.

The science in the HMI program is technically sound, and the work is published in top journals, including Human Factors. The work needs to have broad exposure, which is achieved through presentations at conferences and meetings. The utility of the work appears to be recognized within ARL—for example, elements from the tactile feedback project will be incorporated into the next warrior experiment.

The use of soldiers in experiments is commended. The move toward more realistic warfighting vignettes and more real-life simulations that instantiate threats and hostile elements would help to establish the value of a technology in achieving a desired capability.

Some researchers were embedded directly with the soldiers for a few weeks, and their experiences directly led to the formulation of research topics. This type of exchange is of great value for the research program. Also, some researchers are permanently stationed at U.S. Army installations, where they have regular contact with active-duty soldiers and are able to recruit soldiers as participants in research.

The research presented will be shifting from one-person/one-robot studies to multiperson/multirobot scenarios. This shift in focus is appropriate as the Army moves to the use of more complex teaming architectures. This new direction will bring a need for additional research in trust and in HMI, raising questions about how one verifies software and validates systems to build confidence in the joint human–machine system, considering complex, emergent behaviors. This use case also highlights the

importance of providing the right information at the right time to the humans and to the robots, identified as a thrust of the HMI program.

Human–Robot Trust

In recognition of the changing role of people in a human–robot system, this topic investigates the relationship between trust and the design of vehicle autonomy. The goal is to look at design factors proactively in simulation concurrently with technology development, rather than retroactively; the design of human–robot interaction during early system development is important in order to create a robot system that will be used effectively and as intended. This project is commendable in that it is conducting evaluations and experimentation concurrently based on the applied robotics for installation and base operations system development at the U.S. Army Tank Automotive Research, Development and Engineering Center (TARDEC). The simulation models are based on real data from the system under development at TARDEC.

The milestones are clearly defined, appropriate, and feasible. This 6.2-funded work (early applied research) may lead to shorter-term application than the 6.1-funded work (basic research) described for most of the other HMI projects, though both may have longer-term implications. Trying to visualize what will be available in the way of automated vehicles in the longer term is challenging. This work seems to be a good stepping-stone to future interactions of humans with autonomous vehicles. Additionally, the close interaction with TARDEC on this project shows that the research is targeting a current Army need.

The researchers turned up more than 300 definitions of the simple-sounding word trust, a fundamental variable when working with people. They explained convincingly that trust is a critical factor to be considered when studying the interactions between humans and autonomous vehicles.

A three-factor model for trust, developed for the RCTA, served as the foundation of the researcher’s Ph.D. thesis on human–robot trust. The effort takes an analytic and empirical approach, deconstructing the factors involved in trust, into three categories: human, environmental, and robotic. One known factor that influences trust is system reliability. The effects of many other factors, including stress, workload, personality, trust propensity, and coping style, are still unknown. Quantifying this space is the basis and motivation for the project.

The equipment and tools that are being used are appropriate for this early investigation. The team is in collaboration with TARDEC as well as collaboration across ARL directorates, using the Control of Autonomous Robotic Vehicle Experiments (CARVE) and the Robotic Interactive Visualization Toolbox (RIVET) simulation tools. The trust theory was developed through the RCTA. The autonomous vehicle is under development by TARDEC, and current work uses computer-based simulation. The simulation is low fidelity—a reasonable first step for an early investigation. The simulation is based on data from the real system at TARDEC, and the simulation is updated as the actual system is developed further. Improvements in the simulation’s impact could be achieved with some affordable upgrades to the current desktop solution: for example, with wrap-around hardware.

There is some question about the effect of perception of risk on the experimental results, given the use of a simulated environment instead of a real one. Prior research in HMI has shown that the perception of risk, or lack thereof, influences the behavior of participants in studies with human subjects.

The work has been published in top journals, including Human Factors. Broad exposure of ARL’s work is necessary and is achieved through presentations at conferences and meetings.

The findings on how different people trust an autonomous system illustrate the complexity of the challenges faced by the new technologies. The project also demonstrates the effectiveness of the RCTA mechanism for conducting Army-relevant research in academic environments and the pathways that are created to hire researchers into ARL.

Multimodal Displays for Human–Robot Interaction

The motivation for the project is well-founded: Indeed, a soldier’s visual modality is overloaded. Additionally, constraints on auditory communication may pose a threat to soldier security and mission success. This project investigates the use of other modalities for communication with the soldier, including tactile and gesture. The project goal is to identify interaction modes (single or in combination) that are most effective for soldier-robot communication. The focus is on achieving external validity by synthesizing and translating theory-based predictions, meta-analysis, and experiments to determine whether results generalize to the field.

The project is commendable in that it conducted experiments with real soldiers in a realistic environment and context. The principal investigator is an outstanding practical researcher, and the effort is a clear success. The work was disseminated well at conferences and meetings; journal publications would be very valuable to the research community.

The experiments conducted would have been improved if they had explicitly included in the scenarios a robot as a critical element to provide meaningful information to the soldier, rather than focusing on the communication medium itself. Additionally, the information passed to the tactile belt could have been provided as easily from a human as from a robot, and so other scenarios involving human–human communication using the tactile belt could be performed.

The comparison of text messages, which remain on the screen, versus one-time tactile input, is questionable. It would be reasonable to allow the soldier to re-trigger the signal, which would provide a better comparison of the two methods in terms of the length of time for which the information is available.

Understanding how the technology supports the soldier in abnormal conditions is important, because there may be trade-offs in task performance and situation awareness. Additional insight into the effect of the tactile vest on the soldier’s behavior is expected if disruptive scenarios are incorporated into the experiments.

Overall, the project has developed a piece of technology that has the potential to improve soldier performance, particularly in terms of safely passing information in combat situations.

Human-Autonomy Sensor Fusion for Rapid Object Detection

The project conducts research on models of fusion between computer vision and neurophysiological responses. Recognition systems do not integrate humans explicitly; this project looks at an alternative architecture in which humans are peers to share the information with autonomous systems. The approach fuses human neurophysiological response to enable rapid real-time target detection. The objective of the work is to evaluate the hypothesis that joint human–autonomous system target recognition surpasses the target recognition capability of the autonomous system alone.

The project defined appropriate algorithmic and experimental metrics. The larger milestones and success criteria were less clear, although the project is part of a larger project with 40 researchers.

The project addresses challenging technical issues; neural classifiers are not yet fully understood. The push-button response (part of the human target recognition component) lags behind computer vision, and the neurophysiological recognition lag depends on human conditions. The goal is to produce overall better accuracy and efficiency with a human sensor and to demonstrate a path forward to relevant target tracking and engagement scenarios.

At this preliminary point, the results support the hypothesis that the human component improves object detection, but it is unclear whether the improvements are significant from a practical standpoint. While the research is young and the delta improvement is small in terms of making progress to a fielded, capable system, the payoff could be significant. One manuscript has been submitted for a journal publication and another was accepted and presented at the International Conference on Intelligent Robots

and Systems 2015 conference. This project is currently limited to object detection, but it could be expanded to explore similar approaches for object classification.

INTELLIGENCE AND CONTROL

The intelligence and control (I&C) research theme is focused on enabling the teaming of autonomous systems with soldiers. The overarching goal is to develop an intelligent autonomous system of robots that can perform effectively in an uncertain environment by optimally using limited resources. The topics investigated in this research area are aimed at generating the technologies needed to meet the challenges posed by future military operations that could occur in military-relevant missions and in military-relevant environments. To meet the goals of the research theme, six research programs have been initiated:

• Control focuses on the low-level processes and closely couples sensing and action (actuation) of individual elements of the vehicle;
• Planning and guidance focus on the mid-level, vehicle-centric layer of the control architecture, with immediate path-planning objectives;
• Abstract reasoning focuses on the cognitive element of the architecture, with a special emphasis on human–robot teaming occurring at this level of the architecture;
• Teaming and coordination focus on the interaction of multiple homogeneous or heterogeneous entities to achieve a specified goal, including coordination and communication;
• Behaviors focus on actions of a vehicle built from a hierarchy of elemental tasks and capabilities to achieve one or more specified goals; and
• Learning and adaptation focus on employing key cognitive features of an intelligent vehicle to enable control of its actions and/or behavior to successfully achieve goals in dynamic and/or unrecognizable scenarios and environments.

Projects in the teaming and coordination and learning and adaptation project areas were not presented for review.

The focus of the I&C theme is developing software and algorithms that enable the vehicle to approach a higher level of cognition, enabling the teaming of autonomous systems and soldiers. The I&C theme has tight couplings with the RCTA program and the Micro Autonomous Systems Technologies (MAST) Collaborative Technology Alliance program, with some specific ARL focuses. The higher level cognition that the I&C theme focuses on is aimed at enabling autonomous assets to work in environments of relevance to the military—caves, subterranean spaces, jungles, undercanopies, megacities, and urban environments. Specific I&C research topics are targeted to leveraging state-of-the-art approaches and expanding them to address the uniqueness of the military environment and missions. To identify specific research projects that address the theme, the Army needs, which emphasize high-level capabilities, are deconstructed into specific project areas. The following projects were presented for review.

Abstract Reasoning: Spatial Reasoning in Uncertain Conditions

The focus of this research was on determining how to characterize information collected from field data represented as the collection of uncertain (or incomplete) information. The primary issues this research is trying to resolve are how to build an initial knowledge base and how to expand it autonomously when needed. The state of the art in this domain has been achieved by Microsoft (Bing), Google (Knowledge Graph), and IBM (Watson). The primary limitation of current approaches is that they

are trying to extract all information prior to deployment (versus at query time). Overall, the principal investigator (PI) demonstrated broad understanding of the field and possesses the skills necessary to perform this research. The PI employs adequate tools and methods for this research and has several papers published, including in well-respected venues such as the Association for the Advancement of Artificial Intelligence.

Opportunities

This is an analytical study that is still a work in progress. It was not completely linked to specific military-relevant applications and did not involve any real-world experiments. Issues that might impact the success of the work include how to determine whether outcomes are reliable and correct; how to get algorithms and operations to run on the robot processor (along with the other processing components); and understanding key elements of the development path necessary to get to that point. Only limited results were presented. Estimates of the algorithm accuracy were not provided; more effort is required to provide evidence of feasibility. Experiments would also be helpful. Growing the PI’s team might help in addressing these challenges.

Planning and Guidance:
Autonomous Mobile Robot Exploration with an Information-Gain Metric

This research applied an information-based approach to the mobile robot exploration problem, based on probabilistic and entropy concepts. Effective robot exploration is important for intelligence, surveillance, and reconnaissance efforts. It was shown that the developed algorithm could be used to more effectively detect improvised explosive devices (IEDs). Probabilities or weighted probabilities could also be used to reflect prior information. As such, the problem being addressed is particularly relevant to the mission of enhancing the warfighter’s capabilities via intelligent robotic teams. The algorithm presented was compared to a baseline (non-information based) algorithm. The PI was well aware of the state of the art in this domain and is well qualified to pursue these important issues. The PI’s collaborative efforts, which led to a number of publications, are excellent.

Opportunities

While the information-based entropy algorithm was shown to outperform the baseline (greedy) algorithm, work needs to focus on describing the pros and cons of each algorithm, comparing the algorithms presented to other algorithms considered in the literature, examining the effect of local topology on the algorithm’s temporal and spatial performance, examining the effect of a priori probabilities and weightings on algorithm performance, its reflection of real-world concerns (e.g., uncertain communications, terminal hazards), and extending the work to include multiple robots.

Behaviors: Scene Consistent Visual Saliency

This research dealt with how to better define visual saliency in a dynamic environment. The project was first envisioned because ARL researchers were examining images from a moving robot and realized that present definitions of saliency did not give consistent metrics for saliency—for any given feature––as the moving camera passed a feature and dynamically changed its field of view. The goal was to define visual saliency in a manner that was independent of the view in a general way. As a step toward this, the research considered images obtained from a fixed camera as it panned, tilted, and zoomed to give

different views. This was the first step toward defining a more general definition for a camera with a moving base. The approach began with a bottom-up method of determining saliency, in which one looks for whatever jumps out of the background. Later, this approach was merged with the top-down approach of looking for specific objects. Using a consistency metric, it was realized that consistency for the saliency of a given feature in the surroundings can be obtained only if various images (from different views) are merged into a composite mosaic that casts the set of pixels as a unified field of view. From this concept came a consistent definition of saliency of the visual image. The project is well thought out and has demonstrated that the new definition of saliency can give consistent answers for saliency over a range of camera angles and zoom parameters. The scientific quality of the work is excellent, and the researchers are qualified to do the work. The researchers understand the positioning of this work with respect to other work in the field and have published in appropriate journals and conference proceedings.

Opportunities

The approach represents a first step in addressing a larger problem. It does not work well when the focus is changing, so additional improvements in methodology are needed. This work could also be improved by use of a depth parameter via the stereoscopic effect (once the base is allowed to move). Researchers could then better understand how the work connects with other human–machine interaction work in ARL. The researchers need to identify how the work addresses an important need—that is, why is it necessary to measure saliency consistently and how good does saliency consistency need to be for images to be useful for the Army mission? An additional opportunity could include integrating eye-tracking devices to try to determine what a solider thinks is salient, inferring what the solider is trying to convey based on eye-gaze, and providing that information to the robot or informing the robot planning algorithms.

Control: Autonomous Self-Righting for a Generic Robot with Dynamic Maneuvers

In this research, the problem of self-righting a fallen robot was examined. This situation occurred when the robot fell despite all that had been done to prevent toppling, so the problem is an important one. A potential energy method was used to address the problem. The method is based on an approximation approach that was shown to be potentially useful for robot design, minimizing the number of states from which acceptable recovery is not feasible. The PI has provided metrics and validations to evaluate the accuracy of the approximation. The associated publications of the PI are also good, and a related patent has been filed.

Opportunities

The work could examine how well the energy approach taken by ARL works in practice and a more precise (albeit computationally more burdensome) measure of acceptable recovery (e.g., a measure that includes spatial/temporal constraints). Although the focus is on righting the robot from a static overturned position, a better understanding of the dynamics would come from a more general examination of a broader set of related issues, including stabilizing and destabilizing factors before overturn and the ability to right itself while falling and rolling before coming to rest.

Behaviors: Autonomous Navigation Analysis

This project had the objective of evaluating the performance of three different Army robots navigating from a fixed starting point to a number of predetermined global positioning system way points in an outdoor environment. The evaluation was at the system level—that is, the overall performance of the robots, including their sensors, controllers, and traction/power train systems, were included. The project was designed to expand on research that had been conducted indoors by Microsoft and apply it to the more Army-relevant situation occurring outdoors, in unfamiliar surroundings and with various obstacles that cause significant difficulties for the robots. The experiment was conducted at the Aberdeen Proving Ground facility, and the errors, difficulties, time, and success rates of the robots were evaluated. Further experiments are planned over the next year to enrich the comparison of system performances. The referenced work done by Microsoft was an appropriate starting point. The researchers were qualified to perform this research and they conducted the research as a team and used state-of-the-art facilities to conduct their analyses.

Opportunities

This project was not a sophisticated research effort, but it was a good start in evaluating the performance of a system. Future work could focus on developing methods to predict system-level failures and understanding what causes them. It could conduct more detailed experimentation and analysis to better grasp the effect of the various system components on the robot’s performance.

Abstract Reasoning: Robotic Dream—Episodic Memory Consolidation and Revision

This research focused on the development of a memory system that allowed a robot to retain knowledge from previous experiences. There was some uncertainty about how long this particular project has been pursued. It was listed as a 2004-2020 effort, but it was unclear whether the project was ending or continuing through 2020. There was mention of a path forward, including transition to CTA partners, although this was not defined. It would be helpful if the objectives included more metrics-oriented information and were more clearly set forth.

Storing and processing only exciting events and precursors is an elegant and logical contribution, and there is also a process to code the events into a set of simple low-memory symbols. Memory access time savings also seem intuitive, and they are dramatic.

One PI indicated that no similar research was being done elsewhere, but this was belied by the numerous references cited in a recent journal paper of the PI, which had a sufficient scholarly section of related research. The PIs could continue to strive to publish work in mainstream journals in the field, such as those of the Association for Computing Machinery or the Institute of Electrical and Electronics Engineers.

Opportunities

Since this research has been in existence for a while, there needs to be a stronger linkage, at this point, to military-relevant scenarios.

Overall Intelligence and Control Accomplishments

The I&C team is developing and supporting a suite of forward-looking technologies, algorithms, tools, collaborations—all of which are important for the warfighter effort. ARL has brought together a number of different viewpoints and different skills from different disciplines to start thinking about these problems and tackling them in innovative ways. Collaboration with universities and other agencies and industry has been active and of high value, and ARL has invested in the research through the people it has hired. In general, PIs recognize related research and methods and leverage them to push forward improvements to address the uniqueness of military-relevant problems. While unifying demonstrations, milestones, objectives, and capabilities could better motivate the specifics being developed and elucidate how they will be integrated, the developments being pursued are for the most part essential.

Overall I&C Challenges and Opportunities

The I&C team needs to move on to the next step—bringing the different research projects together synergistically to successfully address broader problems faced by the Army. The I&C team has a good start on this but needs to formalize the process for integrating and knitting together the various research pieces necessary to transition to the next step. Developing quantitative milestones for gauging progress and performance would help in this endeavor. By using milestones, research projects can be redirected where appropriate to better achieve the overall mission goals. It would also be good to see the process whereby desired future capabilities or goals are broken down into a sequence of achievable (realistic) short-term capabilities and goals.

There also needs to be a focus on the big picture: an understanding of where ARL is going and how the projects fit into the bigger picture. To quantify general progress and application-specific performance, more specific connections to the literature could be made in the course of baselining or benchmarking. All projects could make sure to reference the related literature (baselines, metrics, or benchmarks) and understand how they fit and compare. Through mentoring, guidance, and appropriate milestones, quicker progress might also be made toward integrating projects and more effectively contributing to solutions to important problems. Mentoring can come from both internal and external experts. ARL’s open campus concept can be used to bring in external experts.

Challenges to I&C research focus on determining (1) how to deal with trade-offs in order to determine which research to continue; (2) how to effectively integrate outcomes from the individual projects and develop a methodology for this integration; (3) how to share the overarching systems perspective and relay that vision to the research projects; (4) how to identify and validate the process of getting from high-level capability or needs to research tasks (and evaluation or benchmarking of whether they comply with needs); (5) how to appropriately delineate between basic and early applied work; (6) how to balance and integrate top-down and bottom-up-driven processes; (7) how to compare the research against the standard baseline data sets (when available) and how to identify standard metrics for validating whether the research has achieved the stated goals of the proposed work; and (8) how to transition the research from work on simplified problems that facilitate analysis to actual scenarios that are germane to the Army’s unique problems and characteristics.

PERCEPTION

The ARL aspires to be the nation’s premier laboratory for land forces. The perception group at ARL has made significant headway toward achievement of this goal. It has succeeded in establishing relationships with top university laboratories, has attracted some outstanding personnel, especially new Ph.D.’s, and is undertaking interesting and relevant work on a par with academic departments. The ARL perception group is well aware of the current trends in the research community through participation in

top, highly competitive conferences in the field. ARL’s open campus policy appears to be making a positive difference in the quality of the work.

The perception group did not articulate clearly the context within which it works and did not clearly differentiate its approach from the approaches in other areas. Research work on robotics perception needs to be linked to the power needed for the robot or the materials of which the robot is constructed. A systems approach is needed here. Therefore, based on the material presented, the following were inferred:

• The perception group, for the purposes of this evaluation, works in the context of three scenarios:
  • — Microautonomous systems and technology (MAST), where soldiers use microrobots to explore;
  • — Robotics collaborative technology alliance (RCTA), where soldiers use robot/human teams to penetrate the built environment; and
  • — Applied robotics for installations and base operations, where robots perform functions on bases, relieving the warfighters of these functions.
• Most of the perception group’s efforts are basic research in one of the three scenarios. The goal of each is as follows: Within the next 5 years, perform relevant basic science; 5 to 10 years from now, inspire concrete modules for the three integrated scenarios; and 10 to 15 years from now, apply these modules to support integrated experiments, which will subsequently enter the Army Research, Development and Engineering Center as processes to be matured by 2040.

The perception research was assessed according to its achievement of the above-inferred goals.

Autonomous Squad Member

This project attempts to detect changes in the tactical situation by observing the motion of individual soldiers. It is an interesting and important problem as the Army integrates robots into small unit operations. It was not made clear whether there is a roadmap from the broad concept to achievable steps in that direction. Is it going to be practically feasible to deploy a robot with sufficient capabilities in the next 10-15 years? Will the robot participate beyond being a team member? In 2040, a robot may have unique capabilities—will it then be a more active participant in activities?

Data-Driven Learning and Semantic Perception

The core of this work is using video analysis to categorize the actions a human is performing using machine learning with a hierarchy of action templates. This is good, focused work using state-of-the-art methods. Like similar methods, there are still questions about its broader applicability. It would be informative to learn how this work might relate to or affect other projects in the group.

Efficient Discovery and Labeling of Environments for Visual Classification and
Autonomous Navigation

This project is also using machine learning methods to examine video data. In this case, the intent is to segment the scene into regions such as trafficable areas, vegetation, buildings, and sky. This is a good combination of theory and practical application and has great potential. The work could become a framework for developing and testing other new ideas and scenarios.

Dynamic Belief Fusion

Dempster-Shafer theory is a well-known framework for reasoning about uncertain events when, because the categories might overlap, the sum of probabilities need not add up to one. The method introduced here updates Dempster-Shafer for object detection. The results indicate that the new method outperforms existing methods, but the theoretical basis has not yet been fully explored.

Immersive Display of Robot Lidar Imagery

ARL has its own design for a lidar that has excellent performance, generating 256 × 128 pixel depth images. This project takes that three-dimensional (3D) data and displays it on an Oculus Rift head-mounted display for visualization. The project is building the tools needed to enable future research efforts.

Real-Time Optical Flow

The basis of optical flow—measuring the apparent motion of a scene as the observer or the object moves—has been established for many decades. Typical optical flow techniques break down with large changes in illumination (e.g., the sun going behind a cloud) or with cluttered scenes (e.g., trees blowing in the wind). The work shown here demonstrates a new approach, which is much less sensitive to the absolute brightness of the scene and is capable of differentiating the optical flow from humans moving in the scene from the flow caused by vegetation. While the parts of the algorithm are not novel, the combination of these subunits shows promise.

Overall Perception Accomplishments

The perception group presented a broad spectrum of projects, including understanding group behavior, perception for mapping and navigation, and basic research on dimensional reduction and clustering. Many of these projects are of high quality and are informed by the goals and needs of today’s Army. The projects are well defined, and the researchers are aware of the state-of-the-art of computer vision. The group actively participates in important international conferences, which guarantees their awareness of the current and relevant activities in computer vision. Most of the work focuses on specific scenarios relevant to the Army. While it may not be as broad or groundbreaking as the best university research, the work is appropriate to the ARL context.

Through the MAST and RCTA collaborations, the perception group has access to many of the best computer vision researchers in the country, including those at University of Illinois; University of California, Berkeley; University of Southern California; John Hopkins University; and Carnegie Mellon University. Although these collaborative projects at the universities were not within the scope of this review, it seemed clear that they are a good chance to enhance the visibility and quality of the science at ARL. These collaborations need to be exploited to continue to strengthen the group.

The facilities, equipment, and approaches are well-targeted at the group’s goals. The laboratories appear to have created a good computing and experimental environment, and the test site at Fort Indian Town Gap provide realistic scenarios for testing against the three scenarios.

The researchers seemed to work smoothly together, with encouragement from their management to cross organizational boundaries both within ARL and with other national laboratories. At the same time, it was not apparent what mechanisms exist to encourage sharing information; a regular seminar series might be a good addition.

While the quality of the work overall is very good, there was no single project that stood out as especially promising and ready for accelerated deployment. The closest is the work on weakly supervised segmentation for mobility. This project is significant for several reasons: an interesting vision/science result was published at a major conference; it is an integrated end-to-end project that demonstrates the value of the research; and it is an external collaboration with a university. These are all indicators of the project’s scientific value and intrinsic contributions. The more projects that exhibit those characteristics, the stronger the perception effort will be.

It would have been helpful to present, in a concise form, a list of publications, awards, and other data that would help to convey the recognized accomplishments of the group.

Overall Perception Challenges and Opportunities

Perception is a rapidly evolving area. The tool sets, the approaches, and the benchmarks on performance change yearly (sometimes monthly). It is also an area in which it is highly competitive to hire, placing a premium on providing the best resources, colleagues, and opportunities to attract the best people.

The ARL has done a very good job of building a strong perception group and giving them the opportunity to perform very credible single-investigator basic research. To move to the next level, ARL needs to think about a few audacious challenges that go beyond the extant state of the art—so-called grand challenges. These challenges would provide an exciting context for the group and would provide a point of focus for collaborative efforts. This is not to say that the work might not continue to be basic research. Semantic labeling of scenes, for example, would contribute both to the various integrated scenarios and to the international perception community.

More domain-specific challenges for teams, such as off-road mobility, would speak to the broader ARL mission. There are many other possibilities, but it is important for the perception group to aspire to one or more audacious projects that speak to Army needs.

Another opportunity would be to pursue some common platform/sandbox concepts that could be built upon. For example, the project related to weakly supervised segmentation deployed on a robot could be driven in a wide variety of directions. Its current application for driving is already interesting, but perhaps that could be expanded to become a platform for testing other segmentation methods (e.g., motion-based segmentation) and human body tracking for teaming. Again, this would become a point of cohesion for the group.

Most of the work presented focused on relatively traditional RGB vision.¹ However, there is no reason to limit the activities to vision. It may be relevant to consider multispectral sensing, range sensing, and contact sensors, such as temperature, force, and pressure.

OVERALL QUALITY OF THE WORK

In each of the three pillars of the Vehicle Intelligence (VI) program—intelligence and control, perception, and human–machine interaction—the research quality was generally high. Research results are published in high-quality journals. Collaboration with other government agencies, industry, and universities continues to yield positive benefit. Internal personnel advancement, including hiring new, well-qualified Ph.D. researchers, strengthens the capability of the Sciences for Maneuver VI research and development program.

_______________________________

¹RGB is an additive color model in which red, green, and blue light are mixed in various ways to reproduce a broad array of colors.

Each of the three pillars of the VI program has demonstrated significant progress in advancing its research and development objectives to support the warfighter in increasingly complex environments. The research and development activities in each pillar were consistent with its defined objectives. Opportunities in multiperson/multirobot scenario simulation, teaming of autonomous systems with soldiers in uncertain environments, multispectral sensing, range sensing, contact sensing, and immersive display of robot lidar imagery may allow ARL to take the lead in this research and offer greater benefit to the soldier.

The I&C pillar employs innovative approaches in developing and supporting advanced technologies, algorithms, and tools in support of the warfighter effort. There are high-value collaborations with top universities, industry, and other government agencies. This pillar invests in advancing the effectiveness and efficiency of its research personnel.

The perception pillar benefits from high-quality collaborations with top universities that enable successful hiring of outstanding personnel at the Ph.D. level. Current trends and research vectors were observed. Research personnel participate in highly competitive technical conferences and publish in top research journals.

The human–machine interaction pillar’s research and development program is of high quality, based on rigorous design and appropriate metrics. It benefits from a substantial increase in external and internal collaborations. Through early retirements and expansion of the postdoctoral program, qualified Ph.D. personnel have been hired.

The VI research and development program is correctly constituted and resourced with workforce and facilities. In general, the VI team demonstrates good awareness of the scope and direction of research and development in each of the pillars. Cognizance of related activities in industry, government, and international research and development enable meaningful goal setting and tactical adjustments in specific program advancements.

Within VI research and development programs, research quality is generally of high quality. Based on rigorous design, useful evaluation and analysis, appropriate metrics, thorough understanding of related research, workforce interaction at critical open conferences, and publishing VI investigations in top journals, VI is well positioned to maintain and improve the quality of its research products.

Recently hired researchers within VI appear to be well qualified to conduct leading research and development in VI. These new additions to the VI workforce have been educated and trained by leading faculty in the three pillars at top-ranked U.S. academic institutions. At ARL new personnel are exposed to effective mentoring. The VI principal investigators are well prepared and energetic.

VI collaborations with U.S. industry, government, and academe appear to be extensive, very effective, and enabling in advancing the VI research and development mission. These collaborations are important components driving VI awareness, leading to the establishment of meaningful goal setting and tactical program adjustments. Similarly, the collaborations feed the energy of the VI principal investigators.

Inclusion of U.S. soldiers in VI field experiments is commendable. Usage of more realistic vignettes and real-life simulations in experiments would be very beneficial. In particular, the use of realistic warfighting vignettes, where researchers are in the field with soldiers, provides opportunities to test and evaluate research hypotheses more thoroughly, including the revelation of previous unknowns.

Within VI the emerging shift from one-person/one-robot studies to multiperson/multirobot studies merits sustained attention. This shift exposes VI to more complex teaming architectures, concomitant realistic field environment, and potential improvement of the validity and applicability of the research results.

Some strategic goals and tactical milestones for VI research and development programs could be made more apparent. To help quantify general progress and application-specific performance, more efforts need to be made in terms of baselining and benchmarking. A process whereby desired capabilities and goals are broken down into a sequence of achievable (realistic) short-term capabilities and goals would be beneficial. Unifying demonstrations, milestones, objectives, and capabilities could help to better motivate the specifics being developed and how they will be integrated.

Application of more systems integration principles across research projects and pillars would strengthen the overall impact of VI research products. Similarly, connectivity between individual principal investigators could be improved.

The VI program is well positioned to maintain and improve the quality of its research products. To move to the next level, VI needs to undertake carefully chosen, audacious, grand challenges that go beyond the extant state of the art. Resulting activity and research products would provide leadership in research and development. This would yield inherent advantages in framing VI problems to achieve solutions that benefit the Army.