Statistical Methods for Testing and Evaluating Defense Systems Interim Report (1995) / Chapter Skim
Currently Skimming:
1 Introduction
1 Introduction
Pages 7-14
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 7... ...
In addition, between milestones 0 and I, various planning documents are prepared. In particular, the Operational Requirements Document details the link between the mission need and specific performance parameters, and the Test and Evaluation Master Plan provides the 7
|
From page 8... ...
has implications for how statistical methods would be applied. Similarly, the operational test design or evaluation of system performance might conceivably make use either of optimal experimental design methods which depend on parameters that must be estimated or of statistical techniques that "borrow strength" from data earlier in the process.
|
From page 9... ...
The prospectus for the panel's study anticipated the need for breadth in the scope of the study: "In addition to making recommendations on how to improve operational testing under current requirements and criteria, the panel would also take a longer term perspective and consider whether and to what extent technical, organizational, and legal requirements and criteria constrain optimal decision making." Furthermore, the prospectus mentions a major point expressed in the workshop that gave rise to the panel study: "An idea that was expressed often Eat the workshop] is to encourage moving away from the present advocacy environment surrounding quality assurance, in which one party or the other is characterized as being at fault.
|
From page 10... ...
Operational Testing of Complex Systems Overview Modern methods of manufacturing and product development recognize that operational test and evaluation is a necessary part of placing any product or system into widespread public use. Evaluation of the performance of products and systems in operational use against prespecified performance standards, such as effectiveness, suitability, efficacy, or safety criteria, is usually the last experimental testing stage in an evolutionary sequence of product development.
|
From page 11... ...
In most cases, for example, operational testing involves human interaction as part of product or system usage, and the isolation and control of human factors usually cannot be achieved to the same extent as is the case, say, with certain environmental or physical factors. 2Public Law 99-661 states, "In the case of a major defense acquisition program, no person employed by the contractor for the system being tested may be involved in the conduct of the operational test and evaluation required under subsection (a)
|
From page 12... ...
Statistical methods such as experimental design and reliability engineering are now used up front to compare designs and vendors, as well as to optimize product and process designs. There is also less reliance on end-product testing and inspection to assure quality, especially in settings in which operational testing of the manufactured product cannot feasibly be carried out under all likely scenarios of use.
|
From page 13... ...
Efforts toward a taxonomy structure for DoD systems for operational test (Chapter 6) In addition, five appendices are provided: Appendix A describes in detail the organizational structure of defense acquisition; Appendix B presents a short history of experimental design, with commentary for operational testing; Appendix C addresses the optimal selection of a small number of operational test environments; Appendix D lists the individuals consulted by the panel; and Appendix E provides charts showing the organization of test and evaluation within DoD overall and within the Army.
|
From page 14... ...
We will also look at the current role in the acquisition process of various oversight and consulting groups, such as the Institute for Defense Analyses and the Office of the Director of Operational Test and Evaluation. Finally, we will address issues involving the extent to which additional statisticians, statistical training, or access to expert statisticians would improve operational testing.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.