Critical Code Software Producibility for Defense (2010) / Chapter Skim
Currently Skimming:
2 Accept Uncertainty: Attack Risks and Exploit Opportunities
2 Accept Uncertainty: Attack Risks and Exploit Opportunities
Pages 45-67
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 45... ...
This happens when similar decision points have been experienced in other settings, experience was gained, and it has been possible to transfer that experience to new projects that are sufficiently similar. For precedented development efforts, managers can project plans further into the future of a development process with higher accuracy.
|
From page 46... ...
, while engineering practices and technology choices for unprecedented systems and compo nents are guided by processes for mitigating engineering risk. In fact, there is a constant pace of innova tion even for seemingly established functional capabilities, such as back-office systems, with some areas of innovation and other areas that are more guided by convention.
|
From page 47... ...
This is because these process patterns do not account for the engineering risks and uncertainties inherent in developing innovative software, where there are no laws of physics and materials to constrain solutions to particular structural patterns. In precedented software, the structural patterns derive from established software ecosystems and from the body of precedent.
|
From page 48... ...
This applies particularly to engineering risks related to architecture and ecosystems choices, quality attributes, and overall resourcing. With innovative projects, programmatic risk can be reduced through use of iteration, incremental engineering, and modeling and simulation (as used in many engineering disciplines)
|
From page 49... ...
Second, there is need for modeling, simulation, prototyping, and other "early validation" techniques for many different kinds of software engineering decisions, for example, those related to architecture, requirements, ecosystem choice, tooling and language choice, and many others. Third, system engineering models must be developed through which appropriate "credit" can be given in earned value models for activities that mitigate identified engineering risks.
|
From page 50... ...
Available online at http://www.telegraph.co.uk/motoring/news/7189917/Toyota-offers-UK-Prius-owners-brake software-upgrade.html. Last accessed August 20, 2010.
|
From page 51... ...
However, in the history of software engineering, the principal "measurables" have been time, effort, lines of code produced, and defects found and fixed. These are only approximate surrogates for the attributes of progress that matter in complex development projects, such as identification and resolu tion of engineering risks, assurance with respect to quality attributes, manifestation of critical functional features, ability to support future evolution, and so on.
|
From page 52... ...
However, in the past two decades, considerable progress has been made in assessing organizational capability to enact process models to manage complex software developments. This understanding has been packaged in a variety of ways, such as the capability maturity model, ISO 9000, and the spiral development model.
|
From page 53... ...
Time-Certain Development and Feature Prioritization The fact that (particularly SIDRE) software development effort and duration cannot be estimated precisely means that it is unwise to try to lock a software project into simultaneously fixed budget, schedule, and feature content (as has been found in many fixed-price, fixed-requirements software development contracts)
|
From page 54... ...
Finding 2-2: The prescription in DoD Instruction 5000.02 for the use of evolutionary development needs to be supplemented by the development of related guidance on the use of such prac tices as time-certain development, requirements prioritization, evidence-based milestones, and risk management. Finding 2-3: Extensions to earned value management models to include evidence of feasibility and to accommodate practices such as time-certain development are necessary conditions to enable suc cessful application of incremental development practices for innovative systems.
|
From page 55... ...
, and measurement of progress toward thorough evidence generation.9 Some initial steps in this direction are provided in a report by Boehm and Lane.10 Recommendation 2-1: The DoD should take aggressive actions to identify and remove barriers to the broader adoption of incremental development methods, including iterative approaches, staged acquisition, evidence-based systems and software engineering, and related methods that involve explicit acknowledgment and mitigation of engineering risk. There are different kinds of barriers that can be addressed through combinations of established best practice and emergent improved practice derived from technology and other improvements.
|
From page 56... ...
In the committee's experience, successful software projects are managed in a way that, implicitly, treats scope as a variable, 13 This was a goal of the Amazon.com architecture reengineering project, as documented in NRC, 2007, Summary of a Workshop on Software-Intensie Systems and Uncertainty at Scale, Washington, DC: National Academies Press. Available online at http://www.
|
From page 57... ...
This analysis must avoid conflating engineering risk with programmatic risk and instead account for process plans (and earned value credit models) that acknowledge the reality of the engineering risk and indicate how it can be mitigated (as outlined above)
|
From page 58... ...
An iteratie process for software development requires somewhat of an iterative or, more precisely, an incremental contract with the customer, very much following the concept of a spiral model of software engineering.17 For a company to respond to a request for proposals (RFP) with some accuracy, it generally must have experience with multiple similar projects on the basis of which it can estimate with confidence the resources and risks associated with building and testing a particular system.
|
From page 59... ...
This also affords opportunity for risk mitigation regarding validation for critical requirements, enabling operational acceptance and pro viding evaluators an opportunity to mitigate the engineering risks they face regarding various kinds of evaluation criteria. Regarding budget, it enables the customer to achieve a budget target with essen tial features of the system completed and to establish options regarding additional features or quality improvements earlier in the lifecycle, thus facilitating negotiations regarding lower-priority features or bug-fixing time later on in the project.
|
From page 60... ...
This will be discussed next. REALIzING DOD SOFTWARE BENEFITS VIA DOD INSTRUCTION 5000.02 AND EVOLUTIONARY ACqUISITION As discussed above, recent DoD policy in DoDI 5000.02 has established the concept that "evolu tionary acquisition" is the recommended way to acquire DoD systems, but the policy does not provide detail about how successful evolutionary acquisition can be achieved, particularly in the software arena, and in a way that is compatible with the concepts of incremental iterative development.
|
From page 61... ...
23 See related discussion in Chapter 4. 24 "The quantity and quality of software engineering expertise is insufficient to meet the demands of government and the defense industry." Excerpted from presentation by Kristin Baldwin, 2008, "DoD Software Engineering and System Assurance," January 15, 2008, p.
|
From page 62... ...
But the short response time may preclude use of the commodity infrastructure and, in the absence of validated al ternatives, create uncertainties in the engineering phases of the project regarding architectural choices. The resulting uncertainties and consequences created within the engineering process are engineering risks.
|
From page 63... ...
For innovative projects, efforts to resolve engineering risks can be a significant component of overall project progress, and therefore in an earned value measurement regime there need to be ways to "give credit" for identification and discharge of critical engineering risks. This can be a challenge: How, for example, can the value to Apollo of the experience of Mercury and Gemini be weighed?
|
From page 64... ...
It is particularly critical to have program managers who understand modern software development and systems. Commercial industry also continues to have a strong need for the same types of basic software expertise that the DoD needs and in many areas is competing with the DoD for the same pool of talent.
|
From page 65... ...
Options may include focusing on trying to get engineers in mid-career in addition to young software engineers and improving the career environment so that, irrespective of age, a DoD software engineer can develop and maintain her skills by actually producing software. By bringing some software engineering work in-house, the DoD may be able to stimulate interest in DoD careers and opportunities.
|
From page 66... ...
Summary Because the DoD does not currently have the requisite expertise and talent it needs for effective software producibility and because the rapid pace of software development demands ongoing interac tion with the field, the DoD must engage experts outside the DoD and its primes. This engagement, to be effective, should be accompanied by internal processes to apply and incorporate contributions and feedback to software projects throughout the systems engineering lifecycle.
|
From page 67... ...
internal DoD education and development of software expertise is needed to bridge the gap.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.