Improved Operational Testing and Evaluation Better Measurement and Test Design for the Interim Brigade Combat Team with Stryker Vehicles: Phase I Report (2003) / Chapter Skim
Currently Skimming:
3. Test Measures
3. Test Measures
Pages 31-58
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 31... ...
and to be significantly more strategically deployable than a heavy Mechanized Infantry Brigade (MIB)
|
From page 32... ...
For example, in a perimeter defense mission, these may include the probability that no penetration occurs, the expected value of the time until a penetration occurs, and the expected value of the number of friendly casualties, all of which are of interest to the analyst. For the IBCT JOT, mission-level MOEs can provide useful information to: evaluate how well a particular mission or operation was (or will be)
|
From page 33... ...
Although numerical values of mission-level MOEs provide quantitative information about the degree of mission success, the analysis of operational test results should also be a diagnostic process, involving the use of various MOEs, MOPs, and other information to determine why certain mission results occurred. Using only summary MOE values as a rationale for decision recommendations (e.g., select A over B because MOEA = 3.2 > MOEB = 2.9)
|
From page 34... ...
may also be applied across these mission types, although objective casualty-related MOEs are especially useful for evaluating SSC engagements, in which both the IBCT and the OPFOR casualties are indicators of mission success. Casualty-related measures are less commonly applied to SOSE missions, in which enemy losses may have little to do with mission success.
|
From page 35... ...
However, to do so, care must be taken to ensure that the SME ratings have the usual properties of subjective data used in other scientific studies, that is, that they can be calibrated, are repeatable, and have been validated. One good way to support the use of SME ratings in an IOT is to present a careful analysis of SME training data, with particular attention paid to demonstrating small inter-SME variance.
|
From page 36... ...
Casualty-Related Measures In this section we discuss some of the casualty-related MOEs for evaluating IBCT mission success, appropriate for both combat and SOSE missions, but particularly appropriate for SS C-like engagements in which both sides can inflict significant casualties on each other. Specifically, we discuss the motivation and utility of three casualty ratio MOEs presented by ATEC in its operational test plan.
|
From page 37... ...
against friendly forces M _ initial number of friendly forces in an engagement FRo - N/M= initial force ratio nits _ number of surviving enemy forces at time tin the engagement myth _ number of surviving friendly forces at time t in the engagement FR6t9_ n~t9/m~t) - force ratio at time t Chit)
|
From page 38... ...
Force Exchange Ratios The LER indicates the degree of mission success in tactical-level engagements and allows an examination of the impact of different weapon systems, weapon mixes, tactics, etc. At this level, each alternative in a study traditionally has the same initial U.S.
|
From page 39... ...
This suggests considering a measure that "normalizes" casualties with respect to initial force size, which gives rise to the force exchange ratio (FER)
|
From page 40... ...
~ . Initial force ratios.
|
From page 41... ...
There are equally good statistical approaches that can be used with the FER and the LER Fratricide and Civilian Casualties ATEC has correctly raised the importance of developing suitable MOEs for fratricide (friendly casualties caused by friendly forces) and civilian casualties caused by friendly fires.
|
From page 42... ...
These casualties occur when rounds fired at the enemy go astray (for a variety of possible reasons, including erroneous intelligence information, false detections, target location errors, aiming errors, weapons malfunction, etch. Accordingly, we recommend that ATEC report, as one MOE, the number of such casualties for IBCT/Stryker and the baseline force and also compute a fratricide frequency (FF)
|
From page 43... ...
question that needs to be answered during the Stryker operational test is: Is a Stryker-equipped force more effective than the current baseline force? The TEMP states that: The Stryker has utility in all operational environments against all projected future threats; however, it is designed and optimized for contingency employment in urban or complex terrain while confronting low- and midrange threats that may display both conventional and asymmetric warfare capabilities.
|
From page 44... ...
The reasons for performance differences can be divided into two categories: Stryker capabilities and test factors. Stryker capabilities include situation awareness (which contributes to survival by avoidance)
|
From page 45... ...
MOE 4-3-5 Capability to integrate MEP and FBCB2 data The measures associated with Criterion 4-3 are primarily technical and address the ability of the existing hardware to be integrated onto the Stryker platforms. As with many of the other capabilities, any difference in performance that might be attributed to hardware integration will be assessed using the military judgment of the evaluator supported by technical and developmental testing.
|
From page 46... ...
provided by ABCS to commander and staffs? MOE 4-2-4 How well did the ABCS allow the commander and staff to gain and maintain situation awareness/understanding?
|
From page 47... ...
These kinds of measures readily allow for direct comparison to the baseline, and definitions can be written so that they are measurable. System Degradation Measures: Situation Awareness as an Experimental Factor The other type of measure that would be useful in attributing differences to a specific capability results from degrading this capability in a controlled manner.
|
From page 48... ...
· IBCT systems should be able to rapidly "hand off" targets to enhance unit kill rates at all ranges. · Using combinations of the above situation awareness advantages, IBCT units should be capable of changing traditional attacker-defender battle dynamics favoring the defender at long ranges and the attacker at shorter ranges.
|
From page 49... ...
There must be a scripted turning off of the situation awareness hardware. This kind of controlled test condition leads to results that can be directly attributed to the situation awareness capability.
|
From page 50... ...
OBJECTIVE MEASURES OF SUITABILITY The overall goal of the IOT is to assess baseline force versus IBCT/ Stryker force effectiveness. Because inadequate levels of reliability and maintainability (R&M)
|
From page 51... ...
; · infant mortality, durability/wearout, and random failures (types and consequences of these three types of failure modes) ; · durability accelerated testing and add-on armor; and · random failures, GFE integration, and scoring criteria.
|
From page 52... ...
Other uncontrollable covariates, such as weather conditions, could certainly have an impact, but it is not clear whether these effects can be sorted out cleanly. For 70f course, these widening intervals are not likely to be true in the immediate period of transferring from developmental test to operational test, given the distinct nature of these .
|
From page 53... ...
Infant Mortality Operational tests, to some extent, serve the purpose of helping to uncover and identify unknown system design flaws and manufacturing problems and defects. Such "infant mortality" problems are normally corrected by making design or manufacturing changes or through the use of sufficient burn-in so that the discovered infant mortality failure modes will no longer be present in the mature system.
|
From page 54... ...
RandLom Failures, OFF, and Scoring Criteria Random failures are those failures that are not characterized as either infant mortality or durability/wearout failures. These should be tracked by vehicle type and failure mode.
|
From page 55... ...
Availability and Maintainability MOPs for availability/maintainability, described in the SEP, include mean time to repair; the chargeable maintenance ratio (the ratio of chargeable maintenance time to the total amount of operating time) ; and preventive maintenance, checks, and services time required.
|
From page 56... ...
2. Although ratio MOE values may be useful in assessing degrees of mission success, both the numerator and the denominator should be reported.
|
From page 57... ...
16. ATEC shoulcl use the "instantaneous LER" measure to determine changes in traditional attacker/clefencler engagement dynamics clue to improvec ~ situation awareness.
|
From page 58... ...
20. Accelerated testing of specific system components prior to operational testing should be considered in future contracts to enable testing in shorter and more realistic time frames.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.