Reliability Growth Enhancing Defense System Reliability (2015) / Chapter Skim
Currently Skimming:
8 Operational Test and Evaluation
8 Operational Test and Evaluation
Pages 105-116
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 105... ...
The environment of operational testing is as close to field deployment as can be achieved, although the functioning of offensive weapons is simulated, and there are additional constraints related to safety, environmental, and related concerns. In this chapter we first consider the timing and role of operational testing and then discuss test design and test data analysis.
|
From page 106... ...
, which currently complicates the judgment as to whether a system's reliability as assessed in developmental testing is likely to be found acceptable in operational testing. In addition to using developmental tests that include stresses more typical of real use (see Chapter 7)
|
From page 107... ...
Second, the prototypes used in operational testing will be used only for short durations, not even close to the intended lifetimes of service or availability expected of the system. Thus, until developmental testing makes greater use of accelerated life testing in various respects (see Chapter 6)
|
From page 108... ...
Also, one or more focused follow-on tests could be conducted after the initial operational testing, allowing previously observed deficiencies and newly implemented redesigns or fixes to be examined. However, achieving a test of suitable length for reliability remains a challenge; thus, it is far better for the emphasis to be on developmental testing to produce a system whose reliability is already at the required level.2 2 Some appeals to use sequential methods have been made to make testing more efficient.
|
From page 109... ...
However, such designs could often be usefully modified to provide information on problems raised in developmental testing, especially when such problems either have not been addressed by design modifications, or when such design modifications were implemented late in the process and so have not been comprehensively tested. TEST DATA ANALYSIS To greatly enhance the information available from the analysis of operational testing, it is important to collect and store reliability data on a per test article basis to enable the construction of complete histories (on operating times, operational mode summary/mission profile phase or event, failure times, etc.)
|
From page 110... ...
In other words, the analysis of an operational test should not be focused solely on the comparison of the estimated key performance parameters to the system's requirements. Although such aggregate assessments are important because of their roles in decisions on whether the system has "passed" its operational test, there is other information that operational test data can provide that can be extremely important.
|
From page 111... ...
THE DT/OT GAP It is well known that reliability estimates using data from developmental testing are often optimistic estimates of reliability under operationally relevant conditions. Figure 8-1 shows the ratio of developmental test and operational test reliability for Army acquisition category I systems from 1995-2004.
|
From page 112... ...
This DT/OT gap is likely due to several aspects of developmental testing: 1. the use of highly trained users rather than users typical of opera tional use, 2.
|
From page 113... ...
Finally, in terms of the representation of a system's functions that have not been completed, this is one reason for having a full-system, operationally relevant test prior to delivery to DoD for developmental testing. In addition to design changes, the DT/OT gap could be analytically accounted for by estimating the size of the gap and adjusting developmental testing estimates accordingly.
|
From page 114... ...
To reduce the size of this DT/OT reliability gap, Ellner proposed ways of either modifying the design of developmental test events, or modifying the test analysis. For the design of developmental testing, it could be based more closely on the operational use profile through the use of what Ellner called "balanced" testing, which is that the cumulative stress per time interval of test should closely match that of the operational mission profile.
|
From page 115... ...
Also, later developmental testing often involves full-system tests, while early developmental testing often involves component-level and subsystem tests. Restricting attention to the comparison of late-stage developmental to operational testing, Ellner pointed out that there may be "normalizations" that can be used to reweight scenarios of use in developmental tests to match the scenarios faced in operational tests (or field use)
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.