Statistical Issues in Defense Analysis and Testing Summary of a Workshop (1994) / Chapter Skim
Currently Skimming:
LINKING INFORMATION ACROSS THE ACQUISITION PROCESS
LINKING INFORMATION ACROSS THE ACQUISITION PROCESS
Pages 23-32
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 23... ...
The system then undergoes developmental testing. Developmental tests frequently identify problems with the initial design, such as a failure to achieve needed performance standards, and consequently the design is improved and then tested again.
|
From page 24... ...
In practice, some problems may involve fairly straightforward applications of existing techniques while other problems may involve difficult modeling issues and require the development of new methods. In the Bayesian approach, one's uncertainty about any phenomenon may be represented by a probability distribution for the quantity of interest under consideration.
|
From page 25... ...
The posterior distribution is also less dispersed than the prior distribution because the data have reduced our uncertainty about the hit probability. In general, for small sample sizes the posterior distribution will be similar to the prior distribution; for large sample sizes it will be tightly concentrated around the observed hit frequency; and for intermediate sample sizes it will be moder
|
From page 26... ...
.6 .8 1 ately concentrated about some value between .85 and the observed frequency. It is also instructive to compare the posterior distribution in Figure 2 with results that would be obtained without using the engineering judgment described above.
|
From page 27... ...
The result of this analysis is a posterior distribution that reflects uncertainty about the hit probability after the test is run. When a point estimate is required, it is common to specify a loss function that reflects the relative costs of underestimating or overestimating the hit probability by different amounts.
|
From page 28... ...
Combining Information We have described how one might combine engineering judgment, prior knowledge about similar systems, and developmental test results to arrive at an overall assessment of knowledge about a system's performance with respect to some measure of effectiveness. In the same way, one could use the posterior distribution from this analysis as the prior distribution for the analysis of data from operational testing.
|
From page 29... ...
For example, operational testing conditions are rarely precisely reflective of combat conditions, so the projection of test results to combat performance is necessarily subjective. There are a number of points in any statistical modeling effort at which engineering judgment is used to select assumptions and to assess the direction and severity of the impact if the assumptions are violated.
|
From page 30... ...
As a practical matter, however, hierarchical Bayes analyses are frequently indistinguishable from their empirical Bayes counterparts, which retain the hierarchical structure but estimate the prior distribution directly from the data rather than subjectively determining it. Classical Approaches Other non-Bayesian (frequentist)
|
From page 31... ...
with high unit costs and for highly reliable electronic subsystems that would require operational tests with a fixed configuration to be carried out over a period of several thousand hours. Beyond current applications in defense testing, further employment of reliability growth methods during developmental and operational testing might involve setting performance targets for systems during an initial reliability growth period.
|
From page 32... ...
~. Linking information collected in various stages of the acquisition process together has benefits besides assisting in the estimation of costs and performance.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.