Pre-Milestone A and Early-Phase Systems Engineering A Retrospective Review and Benefits for Future Air Force Systems Acquisition (2008) / Chapter Skim
Currently Skimming:
4 Systems Engineering Functions and Guidelines
4 Systems Engineering Functions and Guidelines
Pages 75-108
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 75... ...
In preparing these guidelines, the committee identifies those items that address the causes of the large decline in development program productivity and the attendant cost and schedule growth over the past 30 years, as discussed at the beginning of Chapter 1. These guidelines cover the following topics: • The "six seeds of failure": a discussion of six areas of risk that the com mittee believes are worthy of particular attention in the pre-Milestone B phases and that are the basis for the recommendations that the committee makes later; • Other potential causes of development productivity decline: a discussion of several often-mentioned causes of development productivity decline, including talent diminishment in government and industry, and excessive oversight; 75
|
From page 76... ...
These functions are reiterative and thus occur in parallel, to produce the key outputs from the preMilestone A systems engineering process: KPPs, CONOPS, supporting models, and program implementation strategy. • Concept Creation • Preliminary CONOPS • Performance Assessment • Key Performance Parameters • Architecture Development • Program Strategy • Risk Assessment • CONOPS • Cost Scoping • Supporting Models FIGURE 4-1 Critical pre-Milestone A systems engineering functions and outputs.
|
From page 77... ...
The world changes too fast to be friendly to long development cycles. Further, extended delivery times run the risk of the system becoming obsolete before deployment and can be an indication that the concept is excessively complex or excessively dependent on immature technology in its first delivery.
|
From page 78... ...
Often system performance models developed in this phase become the basis for supporting the government's management of the development phase. The following guidelines are important to consider: • The models must typically also provide the basis for setting segment performance requirements and assessing the impact of deviations from expected performance after Milestone A
|
From page 79... ...
A pre-Milestone A goal is to identify the key risk drivers to set the stage for effective risk management in the procurement and development phases. The six key risk drivers identified in the next section are areas to focus on in risk assessment.
|
From page 80... ...
• Handle the expected throughput, and • Meet response time requirements. Performance Performance analysis should be sufficient to: analysis • Predict performance against mission needs, • Assess the impact of segment-level performance on end-to-end performance, and • Include the development of system performance models to support performance assessment in the acquisition phase.
|
From page 81... ...
Table 4-1 summarizes some of the committee's observations on the key functions and outputs. Six Drivers of Cost, Development Time, and Performance Risk that are Addressable by Systems Engineering Processes The committee heard wide-ranging views from its guest speakers, members, and others about the twofold-or-greater growth in program cost and completion time over the past 30 years.
|
From page 82... ...
While this was at best an anecdotal analysis, the committee examined the myriad factors that may have contributed to serious development issues and identified six factors that are pervasive sources of poor performance and that are addressable through sound systems engineering processes. The committee calls these factors, illustrated in Figure 4-2, the six seeds of failure.
|
From page 83... ...
In addition, systems dependent on highly complex external interfaces can be far more difficult to operate after deployment. Pre-Milestone A Mitigation Systems engineering should treat the minimization of external interface complexity as a key driver in selecting concepts and architecture.
|
From page 84... ...
Pre-Milestone A Mitigation Systems engineering should treat complexity minimization as a key driver in selecting concepts and architecture. Architecture selection can have a powerful effect on controlling complexity.
|
From page 85... ...
, Booz Allen Hamilton, "Defense Acquisition Performance Assessment: Report Summary Briefing," presentation to the committee, February 28, 2007. National Research Council, 1993, An Examination of the Air Force's Pre-Milestone One Planning/ Decision Process, Washington, D.C.: National Academy Press.
|
From page 86... ...
In 2000, the Defense Science Board stressed the following guidelines for software program structure that the committee believes are worthy of attention: • Aggressively limit development time to no more than 18 months; • Minimize complexity; • Highly incentivize development; • Allow program management to trade functionality for time and stability; • Have good processes, but value past performance over process; • Use an iterative, not a waterfall, development process; and • Develop an executable architecture first. Contrary to intuition, the integration of commercial off-the-shelf (COTS)
|
From page 87... ...
TABLE 4-2 Other Cited Sources of Development Productivity Decline Source of Decline Committee Observation Diminished talent in It is not clear that the talent in the government is less capable government to manage than it was 30 years ago. The "good old days" may not have acquisitions been as good as we think they were.
|
From page 88... ...
This section addresses some of the most important general guidelines and practices for personnel and organizations responsible for systems engineering policy, methods, and tools that apply to the entire program life cycle. The prime contractor assumes much of the responsibility for systems engineering after Milestone B, with the government involved in oversight, review, and approval.
|
From page 89... ...
Modeling, simulation, and analysis will play a major role in identifying key design parameters -- the "long poles in the tent" -- which are associated with potential risk, sensitivity, or uncertainty and on which the ability of the design to meet important requirements will depend. Whether the parameter is processing throughput or response time or the acquisition cost per removal-free operating hour, early identification provides guidance to the designers and focus to riskmitigation planning activities such as early benchmarking or development of alternatives.
|
From page 90... ...
While the design of systems of systems may impose somewhat different engineering processes, the committee believes that, in general, most of the elements of good systems engineering apply to systems of systems as well, and the committee has not attempted to differentiate between them in this report. However, these types of system aggregations increase the complexity of external interfaces that are difficult for a development program office to define and control.
|
From page 91... ...
The resulting system dynamics model could be used to help establish appropriate policies and guide program management to insightful decisions that affect program success. While the committee did not hear any accounts of the successful use of system dynamics modeling on government development programs, the committee believes that this discipline deserves further evaluation as a systems engineering and program management support tool. While system dynamics modeling is relevant to both the system being developed and the development process itself, there are many equally good (some would argue substantially better)
|
From page 92... ...
those that result from competition-driven biases in the initial cost and schedule estimates. By "execution performance," the committee refers to how closely ultimate cost and schedule performance mirror the cost and schedule performance obtainable by an optimally managed program.
|
From page 93... ...
Requirements Creep and Requirements Traceability Matrices After Milestone A and to a greater degree after Milestone B, the contractor, with help from government overseers, will be developing detailed requirements at the segment, subsystem, and component levels. Systems engineering in govern Ronald Kadish, Gerald Abbott, Frank Cappuccio, Richard Hawley, Paul Kern, and Donald K ozlowski, 2006, Defense Acquisition Performance Assessment.
|
From page 94... ...
It plays a critical role in guarding against the top-down and bottom-up requirements creep and requirement instability, discussed above with respect to the six seeds of failure. During the pre-Milestone A period, the KPPs of the system (or system of systems)
|
From page 95... ...
NASA's companion to the DOD acquisition process has elements similar to those described in Chapter 1. A good systems engineering process that includes up-front development planning is just as beneficial to NASA as it is to DOD.
|
From page 96... ...
Defense Science Board and Air Force Scientific Advisory Board Report (Young Panel Report) In August 2002, the Under Secretary of Defense for Acquisition, Technology and Logistics; the Secretary of the Air Force; and the Under Secretary of the Air Force/Director of the National Reconnaissance Office chartered the Defense Science Board/Air Force Scientific Advisory Board Joint Task Force on Acquisition 10 National Research Council, 1993, An Examination of the Air Force's Pre-Milestone One Planning/ Decision Process, Washington, D.C.: National Academy Press.
|
From page 97... ...
Key Performance Parameters and CONOPS 5. At Milestone A, have the KPPs been identified in clear, comprehensive, con cise terms that are understandable to the users of the system?
|
From page 98... ...
Are the major known cost and schedule drivers and risks explicitly identified, and is there a plan to track and reduce uncertainty? Identifying the major cost and schedule risk areas, with particular attention to this checklist and the six seeds of failure -- inexperienced leadership, ex ternal interface complexity, system complexity, incomplete requirements at Milestone B, immature technology, and high reliance on new software -- can help focus management on these issues early.
|
From page 99... ...
The committee advocates freezing new requirements and new technology insertion after Milestone B but also notes that making provisions in the initial requirements to facilitate later upgrades could have great long-term value. Architecture Development 12.
|
From page 100... ...
A combination of assignment extension and time-certain milestones will help align incentives. NOTE: KPP, key performance parameter; CONOPS, concept of operations; IOC, initial opera tional capability; FFRDC, federally funded research and development center; SETA, systems engineering and technical assistance.
|
From page 101... ...
11 Defense Science Board/Air Force Scientific Advisory Board Joint Task Force, 2003, Acquisition of National Security Space Programs, Washington, D.C.: OUSD (AT&L)
|
From page 102... ...
Government Accountability Office Reports Pertinent to this committee's discussion of pre-Milestone A SE, the GAO, in its 2003 report Defense Acquisitions: Improvements Needed in Space Systems Acquisition Management Policy,13 stated that the most leveraged decision point in a program is matching the customer's needs with the developer's resources. This initial decision sets the stage for the eventual outcome.
|
From page 103... ...
14 Ibid. 15 Government Accountability Office, 2005, Space Acquisitions: Stronger Development Practices and Investment Planning Needed to Address Continuing Problems, July 12.
|
From page 104... ...
Attention to a few critical systems engineering processes and functions particularly during preparation for Milestones A and B is essential to ensuring that Air Force acquisition programs deliver products on time and on budget. Today's weapons systems provide unprecedented capabilities but also involve complex interfaces with external command, control, and communications systems and rely on a greater volume of software than ever before.
|
From page 105... ...
While the committee considers that each item on the checklist is important, it calls attention to several items that warrant further discussion: • Checklist item 2 recognizes that the world changes too fast to be friendly to long development cycles. The committee believes that the Air Force should strive to structure major development programs so that initial deployment is achieved within, say, 3 to 7 years.
|
From page 106... ...
The Air Force used to have a development planning organization that applied pre-Milestone A systems engineering processes to a number of successful programs, but that organization was allowed to lapse. The role of the Air Force development planning organization, which was within the Air Force Systems Command, was to provide standard evaluation tools and perform pre-Milestone A systems engineering functions across acquisition programs.
|
From page 107... ...
The roles and functions of the various organizations involved in acquiring major weapons systems need to be clearly defined. The responsibility for executing systems engineering and program management in the pre-Milestone A and B phases should be vested in the military departments that do the actual development planning functions.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.