National Academies Press: OpenBook
« Previous: 7 Human Sciences
Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×

8

Analysis and Assessment

The Panel on Assessment and Analysis at the Army Research Laboratory (ARL) conducted a review on July 11-13, 2017 at the Aberdeen Proving Ground in Maryland. The review was on three of the ARL’s Analysis and Assessment core campaign enablers—ballistics survivability, vulnerability, and lethality (BSVL), personnel survivability (PS), and human systems integration (HSI). This chapter provides an evaluation of that work.

ARL’s Analysis and Assessment (A&A) Campaign provides tools that increase awareness of material capabilities, assesses the survivability and lethality of Army systems, and both improves and simplifies the Army’s decision making. The work in the BSVL program provides the analysis and assessment capability to develop efficient means to understand and influence the factors that reduce vulnerability and increase lethality of Army ground and air combat systems. The work in the PS program develops and applies methodologies and tools to model, analyze, and predict effects of weapons against personnel and soldier protective systems. The work in the HSI program provides methodologies for the assessment of cognitive and physical human performance trade-offs and workload in support of human systems integration.

BALLISTICS SURVIVABILITY, VULNERABILITY, AND LETHALITY

Accomplishments and Advancements

The BSVL team is a high-performing group with a wealth of experience and capabilities. The team is a key contributor to the Army’s system analysis, acquisition, and test and evaluation communities. It has a large set of responsibilities that it performs to a high level within the limits of the resources provided. Specific accomplishments that stood out in the work were: (1) improvements in visualization and augmented reality; (2) computing efficiency efforts; (3) underbody blast advancements; and (4) automation of manual data collection.

BSVL studies generate a large set of data when assessing various threats, aspect angles, threat ranges, threat aim points, response variability, and so on. Previously, study results had been presented in difficult to understand and explain formats. By using the increased computing power and advances in data display tools, the BSVL team has developed visualization and augmented reality methods that can more effectively support Army decisions. The demonstration of augmented reality in a ballistic vulnerability analysis is a good example of applying existing technology to this area to make the outputs more easily understood by analysts, customers, and decision makers.

The work on profiling the modular UNIX-based vulnerability estimation suite (MUVES) and distributing the code over parallel processors to gain a factor of 2-8 decrease in computational time is a significant advancement, particularly as more complexity will have to be added to this code. Two areas demonstrated significant efficiencies in preparing inputs for the MUVES model—scanning of the systems to be analyzed in order to automate portions of the building of the geometric target descriptions; and the

Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×

tablet-based range-data acquisition computer toolkit. These both represent significant reduction in human labor and make the process more efficient.

BSVL studies are computationally intensive, so historically it typically took weeks or even months to complete a set of BSVL estimates for an Army customer. Calculations are beginning to take advantage of the Army high-performance computing capability, which together with advances in parallel computing, will substantially reduce the time required to perform BSVL studies. For example, high-resolution finite-element modeling of underbody blast is taking advantage of the high-performance computing resources so that it can provide a useful tool for designing underbody protection. Once these advances are fully implemented in MUVES, Army efforts can be completed more quickly and more efficiently. This advance, once fully implemented, will be a significant improvement, since it will increase the range of possible outcomes that can be considered in modeling and simulation studies.

The underbody blast modeling effort is a significant advancement. Underbody blasts from improvised explosive devices (IEDs) are a major concern, and the BSVL team is developing tools to predict damage and causalities due to underbody blasts. Without a validated and verified tool to predict the effects of underbody blasts, the Army has been forced to conduct very many underbody blast tests in an attempt to improve underbody blast protection and reduce casualties. This is a slow and costly process. Once the emerging tools are verified and validated, they can be put into production. The Army will then be able to more quickly, economically, and effectively address underbody blast damage.

The work on assessing technologies in development on helicopter blades and technologies that depend on global positioning system (GPS) data in a denied environment demonstrated the payoff of conducting these analyses earlier in the acquisition process—that is, “moving left.” Because these had very different impacts to Army systems, the A&A Campaign needs to work closely with the Assistant Secretary of the Army for Acquisition, Logistics, and Technology on prioritization of the programs in order to have the widest impact for the Army.

BSVL studies have relied on wooden witness plates for arena tests and crude plywood mannequins representing personnel to absorb impacts of fragments, spall, projectiles, and so on for lethality and vulnerability assessments. The damage in these wood witness damage collectors is measured by hand for location and by probes for depth, angle of entry, fragment mass, and so on. These measurements are recorded by hand on data sheets and subsequently entered into computer data bases, again by hand, for documentation of the results and further BSVL studies. The review team was shown efforts to utilize a digital “wand” to probe and automatically record data. This offers the possibility to improve the accuracy and precision of recording results, to reduce the time required and to minimize the potential for manual data handling errors. This is a worthwhile effort that ought to be implemented as quickly as possible and expanded to automate as much of the current manual effort as possible.

Opportunities and Challenges

Recognizing technologies that require new modeling methodologies and having the tools developed in time to support evaluation is an increasing problem, and the A&A Campaign is falling further behind in keeping up with the technical and environmental complexities. For example, the work on multihit and underbody protection modeling, while very worthwhile efforts, were too late to impact current and near-future designs. A validated model of underbody blast is important to have as soon as possible to avoid future designs that are very vulnerable to underbody blasts. As another example where the A&A Campaign is behind, in order to lessen combat vehicle weight, ceramic armors have been proposed as a replacement for metal-based armors, but ceramic armor is very sensitive to multihit damage. The A&A Campaign is just now initiating efforts to develop methodologies and tools to address multihit damage to ceramic armor. These armor technologies were developed for the Future Concepts Systems more than 10 years ago.

There is a need for the Analyses and Assessment Campaign to move to multithreat analyses. For example, an active protection system can have targeting and tracking sensors on a vehicle that is being

Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×

protected. These are vulnerable to electronic warfare and cyber attack. Hence, an analysis of the effectiveness of the system requires interdependent analysis of electronics, cyber, and ballistic interactions. Other technologies where electronic and ballistic interactions are not independent include robotics, manned and unmanned teaming systems, and smart munitions. It will be important to analyze these as a coupled system.

Not only are the technologies increasing in complexity, but the environment in which the Army is fighting and may fight in the future include complex urban environments with a crowded electromagnetic environment, variable atmospheric conditions, and close proximity between the enemy and noncombatants. The electromagnetic environment can be generated only in computer simulations, as there is no comparable environment where the United States can do open-air tests. These added complexities will need to drive changes in the methodology used to produce ballistic survivability and lethality information, and multithreat analyses. There is a need to include multithreats (ballistic, electromagnetic, and cyber) and complex environments in ballistic survivability and lethality analyses. Currently, such information is averaged too early to be most useful in subsequent studies. The increases in complexity will drive a need to take full advantage of the DOD high-performance computing (HPC) resources.

At present, there is a lack of active duty military personnel in the BSVL team. Military experience and expertise is vital in developing study plans and understanding results in a military combat context so that efforts better relate to the military environment; and the impact of results on the military situation can be more readily understood by the Army community. This lack of military personnel impacts the value of the BSVL team’s efforts for the Army.

The BSVL team has been reduced in personnel by one-third from the peak of a few years ago. Nevertheless, there continues to be a significant volume of study requests by the Army community to address as well as significant tool improvement and development required on current and emerging systems, technologies, threats, and so on.

While the efforts being introduced to address critical technology, environment, and threat changes in areas such as manned-unmanned teaming, active protection systems, GPS-denied environments, multidomain integration, behind-armor blunt trauma (BABT), multihit damage/effects, and so on are a positive development, most (if not all) of these concerns have been known for substantial periods of time. The BSVL team needs to stay current with ongoing changes and address emerging and new technologies and threats with new and modified tools to address emerging challenges. The efforts in the past have not always been successful in keeping up as required to support the Army community needs.

The use of HPC resources has improved the BSVL team efforts. However, there are opportunities to make even more extensive use of HPC resources to improve BSVL tools, productivity, and overall contributions to the Army community they support. Using HPC resources to a much greater extent would have a multiplying effect on the overall contributions to the Army community and on how BSVL tools are utilized.

While the efforts on data acquisition are an important first step, methods to fully automate this process and eliminate touch-labor are important and needed. There may be high-resolution imaging techniques to gain all the information required.

While the review team noted some accomplishments and advancements in computing for displaying data and increasing speed of results, no mention was made of any effort to address a basic factor in the whole BSVL approach. Shot lines are modeled as mathematical nondimensional straight line rays, while physical projectiles, fragments, jets, and so on obviously have three-dimensional (3D) shapes that interact with armor, components, fluids, and crew. While it is not fully understood what the impact this simplifying assumption might have on BSVL studies, this deviation of the model from reality warrants some thought, as it is a long-standing concern about a basic tenet of the BSVL tools.

Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×

PERSONNEL SURVIVABILITY

Accomplishments and Advancements

In the ARL A&A activities in personnel survivability, there was a strong focus on performing analyses at the earliest suitable time during the life cycle of programs, early enough that positive or adverse findings and analysis results could be used to inform and to improve decision making. There was clear understanding that this “move to the left” is a trade-off between informing programs at an early stage and multiple, potentially costly analyses for programs prior to major decision points. The implication is that “move to the left” does not necessarily imply the need for early-stage analysis to be performed for every program. The choice depends on cost and program importance.

Statistical analyses underpin most or all of ARL’s A&A missions. In light of the importance of statistics, the presentations showed a laudable emphasis on statistical concepts beyond simple means that are often used in military programs, and need to understand confidence intervals and variance in the analysis of programs. As a principal part of statistical analyses, the presentations especially emphasized proper selection of appropriate analyses. This is of significance, since ordinal data has been misused as integer-continuous data in engineering statistical analyses. The ARL team clearly recognizes this issue (Abbreviated Injury Scale [AIS] 2 is in no sense twice as severe as an AIS 1) and specifically underlined the use of the rich field of nonparametric statistics for ordinal data.

Behind-armor blunt trauma (BABT) and behind-helmet blunt trauma injury risk assessments are longstanding and urgently needed capabilities both for fundamental research and in analysis and assessment missions. This is of crucial importance for informing weight trade-offs with deforming passive momentum defeat mechanism personal protection in both the head and torso. The current focus on BABT injuries from the helmet is commendable and appears to be strongly supported. The team has early-career researchers who are engaged, and are knowledgeable about previous limited work in the field. ARL has taken a good approach for the physiological experimental work combining high-speed X-ray imaging with state-of-the-art and exploratory sensor technologies. This is an example of outstanding work in an area where there has been an urgent need for over a decade. This work ought to continue and be extended to thoracic BABT, for mid-chest (mediastinal), abdominal, and spinal impacts.

An additional area of commendable progress that supports fundamental analysis of personnel is ARL’s development of capabilities for imaging and ongoing generation of human model data, particularly a range of anthropometry data from medical imaging. Promising aspects include acquisition of imaging data from living people. In addition, presentations outlined the development of dynamic surface imaging for occupational and personnel survivability assessments inside vehicles, using commercial 3D imaging systems. Full development and use of this capability can provide additional valuable information in assessing dynamic positioning for both use assessments and realistic dynamic locations for vulnerability assessments. There is a continuing effort to strengthen and develop high-speed flash X-ray imaging for dynamic blast and ballistic assessments; this is an important tool for assessment of injury from physical models to physiological models, providing valuable feedback on the high-rate dynamic response of rapidly deforming personal protection.

The ongoing development of enabling technology for detailed and realistic finite element models of humans for personnel vulnerability assessments is important. These efforts include good use of high-performance computing support from the core supercomputing facilities of the army. These efforts include support for fundamental human tissue property characterization building on previous work both at ARL and in academia. Efforts in this direction need to be actively encouraged for both mid-term and near-term research supporting A&A efforts. Such efforts are enabling for future generations of high-fidelity models, which may support substantial improvements in both vehicle and personal protection.

For biomedical research, the personal survivability group has good, modern facilities. These include live-fire ranges for explosives, fragmenting weapons, ballistics, and available medical assessments of various types. The facilities include a mobile clinical computed tomography (CT) scanner, available computational facilities include substantial Army supercomputer resources. In recent years, ARL has

Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×

developed the ability to perform needed biological experimentation using various physiological and cadaveric models. In parallel with the near- and mid-term development of analysis tools using these models, ARL continues to refine simple physical models such as efforts to replace existing plywood dummies with more anthropomorphic and physically responding manikins. Progress in this area needs to be maintained and improved with core efforts and collaborative efforts among potential research partners to support A&A efforts.

Opportunities and Challenges

The principal opportunities and challenges arise with the essential tools of ARL’s A&A Campaign—MUVES and operational-requirements-based casualty assessment (ORCA). Both are indispensable, but both need to be further developed and validated in several areas over a near-term to long-term time frame. For MUVES, validation and extension efforts are clear and critical; no other organization will develop a robust analysis tool central to the ARL mission. For ORCA, though long-needed efforts have begun to improve basic aspects of components of the framework, principally increasing resolution for the ComputerMan component, planning and core funding for additional improvements and validation are urgently needed. It is important that the development of these tools does not occur solely on an ad hoc basis when responding to immediate A&A tasks. The scope of these tools for Army development and planning is so broad that development needs to occur on a deliberate and well-funded basis. Long-term tool development of both MUVES and ORCA have similar issues, the priorities are not clear on any time frame beyond current development. The challenge is to develop an organic plan for long-term development of both tools that incorporates current threats and addresses risk assessments for threats to personnel in the intermediate and long term. Desirable developments include increases in computational efficiency, especially for parallel use. Much of the use of ORCA and MUVES are essentially fully parallel assessments. Some of this development has started. In addition, the provision for closer and more organic integration of the individual tools within ORCA would be highly desirable. It would also be extremely valuable to add BABT capabilities for personal protective equipment (PPE), particularly helmet and thoracic body armor. It is essential that ARL provide consistent long-term planning and funding for this effort.

Another key area of challenges and opportunities for personnel vulnerability A&A is in the medical arena. Building on a substantially improved collection of battlefield data by Joint Trauma Analysis and Prevention of Injury in Combat (JTAPIC) and the Joint Trauma Program, ARL analysts have the potential for supporting substantial near-term, mid-term, and long-term improvements in survivability and injury risk assessments. However, substantial challenges impede the most efficient use of this data. Key among these challenges is the difficulty of obtaining detailed medical data (principally from JTAPIC and U.S. Army Medical Research and Materiel Command [MRMC]) beyond the coded injury descriptions provided in redacted form, apparently based on the incorrect assumption by the medical community that detailed medical information is not useful for engineering survivability risk assessments. This is an essential limitation (or domain conflict), since effective personnel risk assessments cannot be performed without granular knowledge of injuries well beyond AIS coding categories. Indeed, such detailed medical information offers the opportunity to inform and develop more effective risk models based on battlefield functional capacity, rather than simple risk of injury or fatality reflected in the AIS scores alone.

A related challenge for ARL in developing appropriately granular personal vulnerability A&A for military programs is the continued use of injury coding schemes such as AIS to classify injuries. In this system, injuries are ranked using an ordinal scale from 1 (minor injuries) to 6 (maximum). This scaling, developed principally as a threat to life scale in the automobile biomedical community, has been extended to military injuries. However, this extension cannot fully remediate the problem with its use in the military domain; threat to life is one potentially important aspect in program A&A, but is not necessarily paramount. For example, a survivable mid-shaft femur fracture (AIS 3) may be as militarily significant as a likely fatal injury from an aortic laceration (AIS 5), depending on the needed functional capacity for the

Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×

mission. For future efforts, it is important to both collect and assess injury data with higher granularity and with an eye toward the use of military injury data as an assessment of the mission-oriented functional capacity on the battlefield. The benefits of using additional available battlefield injury data for various threats spans both the development of research tools for A&A and their use in various operational domains.

One long-standing challenge is in the development of models for BABT, especially for the head and torso. This development includes the refinement or replacement of existing physical models for ballistic BABT with various fragments and ballistic threats. This subject has been addressed by several previous National Academies studies, and the essential recommendations have not been addressed even after the passage of a number of years. These previous National Academies studies emphasize the limitations of both the physical models and the underlying injury biomechanics. For example, the expedient “Prather model” for ballistic BABT injury applied to soft VIP body armor for handgun threats in the 1970s was never intended to apply to rifle-round ballistic threats behind hard body armor. As noted earlier, laudable investigations of helmet BABT in physiological models has begun, but the development of such models and physical surrogates has been needed from the beginning of the development of aramid composites for ballistic helmets in the 1980s and the development of hard body armor in the 1990s. Owing to the importance of such models to the Army, addressing this challenge needs to take a central position in the development of personal vulnerability models. It is important for these models to include a plausible set of injury assessments with various battlefield functional capacities and universal joint tasks, not just lethality and severe incapacitation models.

Principal issues in the development of such models include “closing the loop” to validate structure/physiology results derived in physiological models in humans. This effort needs to be two-pronged; both computational models of humans and surrogates and field epidemiology play a role. For computational models, the challenge of improving finite element models can be addressed, in part, by attempting to add models that contractors develop for military programs, paid for by the Army, to the mix of available tools for the broad development of useful BABT injury risk models. For field epidemiology, collaborations with Program Executive Office Soldier, MRMC, the Armed Forces Medical Examiner, the intelligence community, operational commands, and others need to be formalized to obtain as much information from existing personal protection, including vehicles and armor to assist in developing and validating risk models. For example, personal protective equipment (PPE) needs to be sent back to ARL from theatre both in instances where it has worked successfully and in instances where the soldier has sustained injury in spite of (or because of) the PPE.

The ARL A&A Campaign has focused on using and developing tools based on appropriate statistical analyses, including the assessment of variance and distributions in calculations rather than the use of simple means. Essentially every aspect of the A&A mission involves statistical distribution rather than a fixed single exemplar. For instance, threats have a distribution of energies and effects. Human anthropometry arises in military operations as a distribution. Response of vehicles and personal protective equipment can be characterized by a distribution. Even material properties of constituents of vehicles, protective equipment, and people are a distribution. The challenge of using appropriate statistics in assessments is twofold. Often the distribution needed for a given analysis is unknown and is not feasible to measure or obtain. So, approximations need to be made to make the assessment tractable. The other major challenge is determining the appropriate analysis for significance. In the medical and biomedical communities, it has recently become even clearer that the often-used statistical significance value (the “p value”) may not be a desirable or effective measure of the import of a statistical difference. That is, the simple statistical significance does not tell one whether the difference is “important” or not. If the variance of a particular measurement is small, even unimportant differences can be statistically significant. When assessing the significance of an effect, it is important to consider not only the statistical significance but also to consider the size of the effect and whether a difference is a clinically or programmatically meaningful difference. Evolution of this philosophy continues in the biomedical field to emphasize important differences in treatments, not simply differences with a statistically discernable mean.

Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×

There appears to be insufficient core funding to advance key areas in BABT risk modeling, and the existing efforts appear to be currently funded on a somewhat ad hoc basis.

HUMAN SYSTEMS INTEGRATION

Accomplishments and Advancements

The ARL HSI team has beneficially applied the Improved Performance Research Integration Tool (IMPRINT) and digital human modeling to Army system concepts prior to contracting with industry to fully develop new weapon systems. This has led to early design improvements and has avoided costly redesigns later in the development cycle. Following pre-Milestone B applications, these models have been provided to system development contractors to continue iterative HSI analysis as system concepts and designs continued to evolve. The HSI team also developed a much-needed Manpower Requirements Criteria (MARC) toolset that enables designers to trade off candidate designs to cost-effectively optimize soldier accommodation.

The ARL Human Research and Engineering Directorate (HRED) has recruited and hired early-career scientists and engineers and is mentoring them to fill in for more experienced personnel who will be retiring in the near future. This contributes positively to ARL’s ability to provide continuous support to the development and maintenance of their HSI tools, models, techniques, and methods.

There has been much work, over many years, to improve and extend the capabilities of the digital human models and the accompanying soldier clothing and equipment models. This has allowed comprehensive static evaluation analysis and assessment of male and female soldier accommodation in ground combat vehicles and other systems. ARL supports contractors in the use of these models during design development to ensure their proper employment. The extension of modeling analysis techniques to include dynamic conditions is applauded, as it could significantly improve the ability to analyze vehicle safety.

The ARL HSI team developed, improved, and used the IMPRINT model to predict soldier performance and workload in Army systems for nearly two decades. IMPRINT has been employed by hundreds of contractors to assess system concepts, preliminary designs, baseline designs and to propose design changes. This has resulted in significant cost avoidance over many years.

The Job Assessment Software System (JASS) tool is being developed to compare the skills and aptitudes required for Army jobs against existing military occupational specialty (MOS) requirements. This tool has potential to assist contractors in assessing soldier interface characteristics and job demands against the capabilities of the personnel identified in the target audience description (TAD) for operational, maintenance, support, and training positions. This tool, if appropriately verified and validated, could help to identify areas where jobs have been designed that do not match the capabilities of the soldiers identified for those jobs. It then may also allow contractors to correct user interfaces, modify job or training requirements, or propose changes to personnel requirements prior to establishing the system baseline at the stage of preliminary design review (PDR).

Opportunities and Challenges

Opportunities to involve military personnel as subject matter experts (SMEs), subjects, and assistance with HSI tool/model validation ought to be identified. More direct interaction with warfighters is essential to provide high confidence that real Army problems are being successfully addressed in a proactive, timely, and efficient manner. The mechanism for acquiring soldier assistance needs to be more formalized and not inconsistent and ad hoc as it currently appears to be. Without good ARL situation awareness (SA) with respect to the real needs of the soldier and a continuous focus and feedback loop to ensure that the solutions are usable, there is a high risk that the mission will not be properly supported; there will be

Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×

much resulting loss of life and material, and a failure to achieve objectives. As an absolute minimum, ARL HSI principal investigators (PIs) need to take advantage of opportunities (and be encouraged) to spend time with soldiers in the field to gain an appreciation for their tasking, operational environments, risks, hardships, and the trade-offs that need to be made on a daily basis.

An overarching framework for HSI analysis and assessments ought to be defined and implemented. Needs from potential users (including the Army, contractors, Federally Funded Research and Development Centers [FFRDCs], academia, etc.) ought to be solicited to identify gaps, prioritize them, and use the results to guide future analysis and assessment investments. This would provide a clear rationale for each A&A tool, technique, and method in terms of Army customer needs and provide the information necessary for a coherent evidence-based prioritization for application of resources. It is not clear why A&A tools that have been emphasized and used on past Army acquisition programs (such as goal-directed task analysis [GDTA, IMPRINT, and SA analysis) are not being aggressively maintained and applied. Other tools (e.g., Preventative Maintenance Checks and Services [PMCS+]), while reducing task completion time and errors, do not appear to add new analysis or assessment capabilities. An overarching framework, that includes all HSI domains and relates the A&A techniques to Army needs, is needed to identify areas where stakeholders are being under-served.

It is very important for the operational requirements-based casualty assessment (ORCA) model to accommodate various sizes and shapes of female and male soldiers to provide more useful results to guide program-level decisions. The current version of ORCA uses a 50th percentile male digital human model for all calculations. Creation of several different female and male models of varying anthropometry would significantly enhance the ability of ORCA to provide accurate analytical results.

Rigorous verification and validation (V&V) ought to be viewed as an essential step in HSI tool and model development. Warfighter participation is needed to validate the effectiveness of HSI A&A tools, models, and techniques because they are intimately familiar with the combat environment in which soldiers must perform their mission tasks. Failure to rigorously validate A&A tools in a timely manner throughout the entire cycle increases the likelihood of inefficient use and loss of personnel and material resources to successfully accomplish mission objectives.

More emphasis needs to be given to the transition of tools, operator manuals, training, accompanying approaches and methods to industry. Many of the models, tools, techniques and methods developed by HSI could be cost-effectively applied by system development contractors as they iteratively define, refine, and baseline system designs. To facilitate this, ARL could provide contractors with HSI models and tools, operating manuals, and training. ARL HSI scientists need to also establish a professional relationship with contractor personnel operating these tools, models, and so on to gather lessons learned and ideas for tool and model enhancement.

ARL ought to focus more on using the results of analyses and assessments to improve human-system integration requirements for future acquisition programs. This could take the form of improved contract requirements or upgraded HSI domain standards. The current approach reflected in the human modeling area, is to evaluate contractor designs post-Milestone B. Delaying evaluation until post-Milestone B may be too late in the acquisition process to make cost-effective design changes. It is suggested that improved accommodation or soldier “space claim” requirements could be developed for future contracts, based on ARL’s 10+ years of Jack modeling experience. Improved requirements could lead to earlier problem detection and resolution.

It is important for ARL to develop a capability for analyzing and assessing HSI (e.g., human factors engineering [HFE], soldier interface and training) technologies for their potential in improving human-system integration and performance. One of the areas of emphasis for the A&A Campaign is titled “A&A on Technologies.” Currently, this area is limited to “SLV A&A of Technologies” and “Technology Trade Space A&A.” “Human Systems Integration Technologies” need to be added as a third subarea. Emerging soldier interface technologies need to be analyzed to determine if they hold potential for application to future Army systems. Additionally, formal usability assessment tools ought to be developed for analysis of soldier interface technologies.

Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×

JASS may hold potential for broader applications. Contractor HFE and user interface designers usually do not have knowledge of the soldier occupational specialties and grades that will be operating their systems to accomplish their required mission tasks. Without knowledge of the skills, aptitudes and knowledge of the users, contractors depend upon retired military or surrogate soldiers to analyze and assess their designs. With modification, JASS might be used as a tool to evaluate soldier jobs and user interfaces against the military occupational specialty (MOS) requirements for that position as established by the target audience description (TAD). This could result in earlier detection of mismatches between soldier capabilities and job requirements.

It is important that anthropometric models that deal with more realistic scenarios and dynamic conditions for soldier protection be developed soon. ARL presented plans to develop digital human models capable of analyzing varied scenarios and conditions that cannot be analyzed today. In addition, plans were put forward to extend accommodation analysis to include the dynamic conditions that vehicle occupants may experience—for example, rough terrain and improvised explosive device (IED) detonations. Including realistic, dynamic conditions in physical accommodation analysis could dramatically improve the validity of results and allow a higher probability of overall mission success; it could also materially improve soldier safety in the operational environment.

Some popular and important tools and models (e.g., IMPRINT) are not being supported as strongly as in the past. Improvement plans for other A&A capabilities (e.g., digital human modeling, dynamic accommodation) seem to extend over an excessive time span. A capability to perform dynamic soldier accommodation modeling, for example, is long overdue and compromises the ability to appropriately design vehicles and execute missions in an optimal manner.

The number and scope of analyses and assessments need to be expanded to address all domains of HSI. The analyses and assessments presented did not address all of the HSI domains. ARL ought to proactively collaborate between domains to ensure that all aspects of soldier integration are addressed. Currently, the ARL A&A scope emphasizes physical and cognitive soldier accommodation. To extend this, ARL/HRED can “reach out” to other Army organizations (e.g., Survivability and Lethality Analysis Directorate [SLAD] for force protection and survivability and ARL Orlando for training) and agencies responsible for the other HSI domains (i.e., habitability, safety and occupational health) and include their A&A tools, techniques and methods in the overall HSI toolset.

OVERALL QUALITY OF THE WORK

The ARLTAB assessment of this campaign is different from that of most, if not all, other campaign assessments, since the Analysis and Assessment Campaign is intended to be more of an analytically focused, crosscutting activity rather than being research focused. As a result, the criteria are different from those of research-focused campaigns. Nevertheless, the work needs to have technical depth.

The quality of the technical staff and the quality of the work reviewed was generally outstanding. Nevertheless, current A&A efforts are falling further behind in incorporating the complexities of the technologies and environments needed for A&A. This could be due to lack of resources—either financial, personnel, or both. In several areas, the team is only one deep, so there is a lack of personnel, likely driven by funding. Resources need to be made available to address the requirements to analyze and assess complex technologies in complex environments.

The modeling work is generally of high quality, but rigorous verification and validation is not always included in model and tool development; this is especially the case in HSI.

Of the core technical BSVL efforts reviewed, the projects on underbody blast modeling and the collection of data that shows the impact of multiple hits on armor panels were of very high technical quality, and the teams were highly educated and skilled to conduct these efforts. However, limited resources, either funding or personnel, resulted in important effects being modeled well after this information was needed. A lack of validated models has led to the need to carry out a very costly experimental program illustrating that it is important to stay ahead of the need.

Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×

Three of the Key Campaign Initiatives (KCIs), “Framework for Complex Multidomain Analysis,” “Analysis & Assessment of Congested & Contested Operational Environments,” and “CEMA Analysis and Assessment Methodology for Congested and Contested Environments,” have significant overlap, are well outside the current mission space of ARL, and are so broad that the outcomes cannot be clearly seen in the 15-year time frame. More definition and work is needed on these initiatives. The fourth KCI, “Virtual Interactive Simulation Analysis and Assessment” is a long-term follow-on effort from the immersive demonstration that served as a proof-of-principle for these visualization techniques. These KCI efforts would all benefit with near- and long-term deliverables defined.

CONCLUSIONS AND RECOMMENDATIONS

The area of analysis and assessment is a very important area to the Army. The analyses and assessments prepared by ARL support Army decisions at all levels of the Army. The products of these assessments and analyses are used by the Army Evaluation Command in preparing recommendations to Army leadership up to and at the Secretariat level. A&A is an important activity that is very underresourced and falling further behind in meeting mission goals, which puts ARL at risk to losing this activity to another Army organization. It is important to keep this activity in ARL to link the 6.2 tool development with 6.6 tool application.

Military experience and expertise is vital in developing study plans and understanding results in a military combat context so that efforts can relate better to the military environment and the impact of results on the military situation can be more readily understood by the Army community.

There is a potential problem when a contractor or 5-year term employee is the only employee possessing a critical skill for a large project of major importance for the Army.

In some cases, government employees need to use contractors and contractor facilities to get work done that they cannot do because of the work environment and lack of adequate computational support.

The following recommendations are offered.

Recommendation: ARL should prioritize tool development to reflect current and future Army acquisition needs and provide longer-term projects with predictable funding.

  • ARL should develop and articulate clear long-term priorities.
  • ARL should provide long-term sustainment and high-priority improvements to core analysis tools (MUVES, ORCA).
  • MUVES and ORCA should be examined to see if they are structured adequately to handle multihit, multithreat, and complex environment interactions in the future.
  • ARL should include plans for verification and validation as an integral activity throughout the tool development process; this is especially the case for HSI.
  • ARL should consider more sophisticated frameworks (sort of a grand challenge), with a focus on creating the right team, with industrial and academic partners.
  • ARL should consider a red team/blue team approach for analyses/assessments.

Recommendation: ARL should increase engagement with military personnel and program managers (PMs).

  • ARL should increase involvement of military personnel in the prioritization process and in tool and model verification and validation.
  • ARL should involve PMs at an early stage of the prioritization and development process and in the contracting language for A&A support.
Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×
  • Engagement with other services, NATO, TTCP, and other military partners should be further improved with benefits to the Army.

Recommendation: ARL should review the use of contractors and 5-year term employees.

Recommendation: ARL should improve the work environment for A&A staff.

  • ARL should speed up the approval process for conferences and equipment purchases and have these take place a lower level.
  • Information technology (IT) support for A&A staff should be improved.

Recommendation: ARL A&A Campaign should develop and lead only centers well related to the scope of A&A activities and responsibilities within ARL and for which the ARL A&A Campaign leads. For activities that ARL A&A Campaign does not lead, ARL A&A should consider joining existing centers.

Recommendation: ARL should develop increased interactions between the A&A Campaign and various relevant 6.1 and 6.2 efforts within ARL.

Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×
Page 123
Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×
Page 124
Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×
Page 125
Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×
Page 126
Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×
Page 127
Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×
Page 128
Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×
Page 129
Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×
Page 130
Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×
Page 131
Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×
Page 132
Suggested Citation:"8 Analysis and Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018 Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/25011.
×
Page 133
Next: 9 Crosscutting Conclusions and Recommendations and Exceptional Accomplishments »
2017-2018 Assessment of the Army Research Laboratory: Interim Report Get This Book
×
Buy Paperback | $50.00 Buy Ebook | $40.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The National Academies of Sciences, Engineering, and Medicine's Army Research Laboratory Technical Assessment Board (ARLTAB) provides biennial assessments of the scientific and technical quality of the research, development, and analysis programs at the Army Research Laboratory (ARL), focusing on ballistics sciences, human sciences, information sciences, materials sciences, and mechanical sciences. This interim report summarizes the findings of the ARLTAB for the first year of this biennial assessment; the current report addresses approximately half the portfolio for each campaign; the remainder will be assessed in 2018.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!