Reliability Growth Enhancing Defense System Reliability (2015) / Chapter Skim
Currently Skimming:
Summary
Summary
Pages 1-18
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 1... ...
Between 2006 and 2011, one-half of the 52 major defense systems reported on by the DoD Office of the Director, Operational Test and Evaluation (DOT&E) to Congress failed to meet their prescribed reliability thresholds, yet all of the systems proceeded to fullrate production status.
|
From page 2... ...
To evaluate the efficacy of that effort and, more generally, to assess how current DoD principles and practices could be strengthened to increase the likelihood of defense systems satisfying their reliability requirements, DOT&E and USD AT&L requested that the National Research Council conduct a study through its Committee on National Statistics (CNSTAT)
|
From page 3... ...
In contrast, DoD has generally relied on extensive system-level testing, which is both time and cost intensive, to raise initial reliabilities ultimately to the vicinity of prescribed final reliability requirements. To monitor this growth in reliability, reliability targets are established at various intermediate stages of system developmental testing (DT)
|
From page 4... ...
• Fundamental elements of reliability improvement should continue to be emphasized, covering: -- -- operationally meaningful and attainable requirements; -- -- requests for proposal and contracting procedures that give promi nence to reliability concerns; -- -- design-for-reliability activities that elevate the level of initial system reliability; -- -- focused test and evaluation events that grow system reliability and provide comprehensive examinations of operational reliability; -- -- appropriate applications of reliability growth methodologies (i.e., compatible with underlying assumptions) for determining the extent of system-level reliability testing and the validity of assessment results; -- -- empowered hardware and software reliability management teams that direct contractor design and test activities; -- -- feedback mechanisms, spanning reliability design, testing, enhancement initiatives, and postdeployment performance, that inform current and future developmental programs; and -- -- DoD review and oversight processes.
|
From page 5... ...
This designation would emphasize the importance of reliability in the acquisition process and enhance the prospects of achieving suitable system reliability. During developmental testing, opportunities to relax the reliability requirement should be limited: it should be permitted only after high-level review and approval (at the level of a component acquisition authority or higher)
|
From page 6... ...
Design for Reliability High reliability early in system design is better than extensive and expensive system-level developmental testing to correct low initial reliability levels. The former has been the common successful strategy in non-DoD commercial acquisition; the latter has been the predominantly unsuccessful strategy in DoD acquisition.
|
From page 7... ...
While the reliability design and testing topics addressed in these documents are extensive, the presented expositions are not in-depth and applications to specific acquisition programs have to draw upon seasoned expertise in a number of reliability domains -- reliability engineering, software reliability engineering, reliability modeling, accelerated testing, and the reliability of electronic components. In each of these domains, DoD needs to add appropriate proficiencies through combinations of inhouse hiring, consulting or contractual agreements, and training of current personnel.
|
From page 8... ...
[Recommendations 4, 7, and 11] The objectives for early reliability developmental testing and evaluation, focused at the component and subsystem levels, should be to surface failure mechanisms, inform design enhancement initiatives, and support reliability assessments.
|
From page 9... ...
are produced, system-level reliability testing can begin, but that should not occur until the contractor offers a statistically supportable estimate of the current system reliability that is compatible with the starting system reliability requirement prescribed in the program's reliability demonstration plan. System-level reliability testing typically proceeds, and should proceed, in discrete phases, interspersed by corrective action periods in which observed failure modes are assessed, potential design enhancements are postulated, and specific design improvements are implemented.
|
From page 10... ...
To enhance the prospects of growing operational reliability, developmental system-level testing should incorporate elements of operational realism to the extent feasible. At a minimum, a single full-system, operationally relevant developmental test event should be scheduled near the end of developmental testing and evaluation -- with advancement to operational testing and evaluation contingent on satisfaction of the system operational reliability requirement or other justification (e.g., combination of proximate reliability estimate, well-understood failure modes, and tenable design improvements)
|
From page 11... ...
The strategy of building the reliability growth curve to bring the system operational reliability at the end of developmental test and evaluation to a reasonable point supporting the execution of a stand-alone operational test and evaluation, with acceptable statistical performance characteristics, is eminently reasonable. Some judgment will always be needed in determining the number, size, and composition of individual developmental testing events, accounting for the commonly experienced DT/OT reliability gap, and in balancing developmental and operational testing and evaluation needs with schedule and funding constraints.
|
From page 12... ...
One important exception, from a program oversight perspective, is assessing the reliability growth potential when a system clearly is experiencing reliability shortfalls during developmental testing -- far below initial target values or persistently less than a series of goals. Reliability growth methods, incorporating data on specific exhibited failure modes and the particulars of testing circumstances, can demonstrate that there is little chance for the program to succeed unless major system redesigns and institutional reliability management improvements are implemented (i.e., essentially constituting a new reliability growth program)
|
From page 13... ...
The outline reliability demonstration plan should also provide the technical basis for how test and evaluation will track in a statistically defendable way the current reliability of a system in development given the likely number of government test events as part of developmental and operational testing. Prior to being included in the request for proposal for an acquisition program, the outline reliability demonstration plan should be reviewed by an expert external panel.
|
From page 14... ...
reliability growth models should not be used to forecast substantially into the future. An exception to this is early in system development, when reliability growth models, incorpo rating relevant historical data, can be invoked to help scope the size and design of the developmental testing programs.
|
From page 15... ...
, including to the relevant operational test agencies, all data from reli ability testing and other analyses relevant to reliability (e.g., modeling and simulation) that are conducted.
|
From page 16... ...
Department of Defense for developmental testing, the contractor must provide test data supporting a statistically valid estimate of system reliability that is consistent with the operational reliability requirement. The necessity for this should be included in all requests for proposals.
|
From page 17... ...
RECOMMENDATION 24 The Under Secretary of Defense for Acquisition, Technology, and Logistics should create a database that includes three elements obtained from the program manager prior to government testing and from the operational test agencies when f ormal developmental and operational tests are conducted: (1) out puts, defined as the reliability levels attained at various stages of
|
From page 18... ...
RECOMMENDATION 25 To help provide technical oversight regard ing the reliability of defense systems in development, specifically, to help develop reliability requirements, to review acquisition proposals and con tracts regarding system reliability, and to monitor acquisition programs through development, involving the use of design-for-reliability methods and reliability testing, the U.S. Department of Defense should acquire, through in-house hiring, through consulting or contractual agreements, or by providing additional training to existing personnel, greater access to expertise in these five areas: (1)
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.