Improved Operational Testing and Evaluation Better Measurement and Test Design for the Interim Brigade Combat Team with Stryker Vehicles: Phase I Report (2003) / Chapter Skim
Currently Skimming:
Executive Summary
Executive Summary
Pages 1-10
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 1... ...
, the Panel on Operational Test Design and Evaluation of the Interim Armored Vehicle considers the Stryker IOT an excellent opportunity to examine how the defense community might effectively use test resources and analyze test data. The panel's judgments are based on information gathered during a series of open forums and meetings involving ATEC personnel and experts in the test and evaluation of systems.
|
From page 2... ...
OVERALL TEST PLANNING Two specific purposes of the IOT are to determine whether the IBCT/ Stryker performs more effectively than the baseline force, and whether the Stryker family of vehicles meets its capability and performance requirements. Our primary recommendation is to supplement these purposes: when evaluating a large, complex, and critical weapon system such as the Stryker, operational tests should be designed, carried out, and evaluated with a view toward improving the capabilities and performance of the system.
|
From page 3... ...
Failure modes should be considered separately, rather than assigning a single failure rate for a vehicle using simple exponential models. EXPERIMENTAL DESIGN With respect to the experimental design itself, we are very concerned that observed differences will be confounded by important sources of uncontrolled variation.
|
From page 4... ...
For example, it would be informative to run pilot tests with the Stryker situation awareness capabilities intentionally degraded or turned off, to determine the value they provide in particular missions or scenarios. We make technical suggestions in several areas, including statistical power calculations, identifying the appropriate test unit of analysis, combining SME ratings, aggregation, and graphical methods.
|
From page 5... ...
If the confounding issues were reduced or eliminated, the remainder of the test design, aside from the power calculations, has been competently developed from a statistical point of view. Furthermore, this report provides a number of evaluations and resulting conclusions and recommendations for improvement of the design, the selection and validation of MOEs, the evaluation process, and the conduct of future tests of highly complex systems.
|
From page 6... ...
separating the operational test into at least two stages, learning and confirmatory. · ATEC shoulcl consider applying to future operational testing in general a two-phase test design that involves, first, learning phase studies that examine the test object under different conclitions, thereby helping testers design further tests to elucidate areas of greatest uncertainty and importance, ancl, seconcl, a phase involving confirmatory tests to address hypotheses concerning performance vis-a-vis a baseline system or in comparison with requirements.
|
From page 7... ...
· When specific performance or capability problems come up in the early part of operational testing, small-scale pilot tests, focused on the analysis of these problems, should be seriously considered. For example, ATEC should consider test conditions that involve using Stryker with situation awareness degraded or turned off to determine the value that it provides in .
|
From page 8... ...
the problems raised by the simultaneous evaluation of the Stryker vehicle and the IBCT system that incorporates it, (c) whether the operational test can definitively answer specific tactical questions, such as the degree to which the increased vulnerability of Stryker is offset by the availability of greater situational awareness, and (~)
|
From page 9... ...
The third issue is whether the current operational test design can provide adequate information on how to tactically employ the IBCT/Stryker system. For example, how should the greater situational awareness be taken advantage of, and how should the greater situational awareness be balanced against greater vulnerability for various types of environments and against various threats?
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.