Summary of a Workshop on Software Certification and Dependability (2004) / Chapter Skim
Currently Skimming:
2 Summary of Panel Sessions and Presentations
2 Summary of Panel Sessions and Presentations
Pages 5-20
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 5... ...
Some of the themes that emerged during this discussion include these: · While following particular processes cannot alone guarantee certifiably dependable software, comprehensive engineering processes are nevertheless important to achieving this goal. · In evaluating a system, appropriate metrics should be used; it may be necessary to measure secondary artifacts (e.g., process quality)
|
From page 6... ...
It is important to measure properties that actually matter directly, and economic theory suggests that measurement skews incentives: "If you want A and measure B, you will get B." All of this suggests the importance of good empirical research that relates the attributes under consideration and makes it possible to discern what the dependent and independent variables are. There was some discussion of the phenomenon that software errors are not evenly distributed throughout a system -- they tend to cluster in the more complex areas, creating black holes of software defects.
|
From page 7... ...
In these cases, secondary evidence must be used -- perhaps including measurements of process quality and staff competence -- although it is important not to lose sight of the limitations of such evidence.
|
From page 8... ...
· One increasingly sophisticated set of tools that can help in the software development process with respect to dependability is the set of tools related to programming languages, such as type checkers, static analyzers, and model checkers. · Systems integration is a growing and challenging problem.
|
From page 9... ...
The development of large systems of systems often has the unfortunate characteristic that early phases experience relatively smooth subsystem development while later phases encounter serious integration problems. For example, one participant remarked that during systems integration and testing, complex avionics tend to fail repeatedly when the pilot attempts to use the radar, communication, navigation, identification, and electronic warfare systems concurrently.
|
From page 10... ...
How to proceed, what aspects to certify, and to what levels of dependability are unclear. Improved notations and modeling tools are needed to address dependability concerns in all phases and in all aspects of development, but it is important that all software artifacts -- whether specifications, requirements, models, or code -- be kept in sync with each other.
|
From page 11... ...
· Market forces and the cost structure of the software industry may create incentives to release flawed software. · Validation -- determining what the software should do -- is often harder than verification, or determining whether the software does it correctly, and may be more important.
|
From page 12... ...
A major problem is the lack of information flow between groups, a problem that can prevent markets from functioning properly. Reference was made to George Akerlof's seminal paper on the used-car market, where asymmetric information flows prevented that market from working efficiently.7 One person noted that for used cars, the Internet and, in particular, services that permit the prospective buyer to purchase a vehicle history report have helped solve that problem.
|
From page 13... ...
The participants noted a tension between the narrower and crisper notion of certification, and the broader and blurrier notion of dependable usefulness. Although some certification processes take into account the ultimate context of use, many emphasize primarily the internal consistency, coherence, and completeness of the software itself.
|
From page 14... ...
It was observed that positive reinforcements are more effective in changing behavior than negative reinforcements, and that decades of human performance literature and research demonstrate this fact. It was also noted that care must be taken when the introduction of a new system is likely to increase workload burdens within an organization.
|
From page 15... ...
However, it was also noted that good simulation imposes additional costs and can be very expensive for many kinds of applications, which relates to issues raised in Panel C
|
From page 16... ...
All three panelists described promising directions that may lead to more costeffective software engineering techniques for creating dependable software. Overall, several key themes emerged from this panel, among them: · There are interesting substantive overlaps in approaches to software development that seem philosophically opposed on the surface.
|
From page 17... ...
Improvements over that time frame were attributed to the introduction of the REVEAL requirements analysis method, adoption of more powerful static analyses (in particular the use of the Spark Ada subset and its Examiner tool) , and a better engineering process, with earlier integration and more frequent builds.
|
From page 18... ...
The panelists focused on errors related to software flaws, inadequate security, distributed database problems, communications inadequacies, limited auditing capabilities, and poor user interfaces. There has apparently been no serious attempt to perform a detailed risk analysis of electronic voting systems -- a standard procedure in the development of safety-critical systems.
|
From page 19... ...
Panelists agreed that the certification process for electronic voting systems has failed demonstrably. Despite the procedures set up by local election officials and the work of third-party certifiers, voting systems have been found by outside experts to contain egregious flaws and to be susceptible to easy attacks.
|
From page 20... ...
The end users-voters and local election officials -- are not likely to be able to understand the code themselves or to appreciate the implications of its complexity for errors. Inappropriate collaboration between local politicians and the vendor, along with the ability to control the setup of the ballot, may be problematic not only because of potential malfeasance but also because neither the vendor nor the politicians are likely to be experts in designing usable ballots.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.