Human-System Integration in the System Development Process A New Look (2007) / Chapter Skim
Currently Skimming:
8 Methods for Evaluation
8 Methods for Evaluation
Pages 253-274
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 253... ...
RISK ANALYSIS Overview This section describes some commonly used tools for risk management, including failure modes and effects analysis (FMEA) and fault tree analysis (FTA)
|
From page 254... ...
produced • Fault Tree Analyses (FTA) and other technique variations • Lists of usability problems in the form of: written reports, presentations, or videos • Time and accuracy of user's performance • User satisfaction ratings FIGURE 8-1 Representative methods and sample shared representations for evaluation.
|
From page 255... ...
Use error can be addressed and minimized by the device designer and proactively identified through the use of such techniques as usability testing and hazard analysis. An important point is that, in the area of medical products, regulator and standards bodies make a clear distinction between the common terms "human error" and "user error" in comparison to "use error." The term "use error" attempts
|
From page 256... ...
. This approach is based on probabilistic risk analysis and fault tree task decomposition methods, and it has been applied extensively in nuclear power plant design and procedure assessment.
|
From page 257... ...
Others use the term in a more general way as a synonym for risk-management. Hazard analysis is often accomplished as an iterative process, with a first draft being updated and expanded as additional risk management methods (e.g., FMEA, FTA)
|
From page 258... ...
The shared representations typically resulting from these methods are reports containing graphical portrayals of the fault trees or tabular descriptions of the failure modes. The FTA representations show cumulative probabilities of logically combined fault events demonstrating the overall risk levels.
|
From page 259... ...
Shared Representations FMEA The recommended steps for conducting a use-error risk analysis are the same as for traditional risk analysis with one significant addition, namely the need to perform a task analysis. Possible use errors are then deduced from the tasks (Israelski and Muto, 2006)
|
From page 260... ...
Whereas fault trees trace the precursors or root causes of events, event trees trace the alternative consequences of events. The starting point (referred to as the initiating event)
|
From page 261... ...
A qualitative risk rating requires the development of criteria establishing risk levels based on combinations of severity and occurrence ratings.
|
From page 262... ...
. In recent years, graphical software programs have been made available for personal computers that enable users to rapidly assemble fault trees by "dragging and dropping" standard logic symbols onto a drawing area and connections are made (and maintained)
|
From page 263... ...
FIGURE 8-3 Example FTA for a hypothetical automatic external defibrillator. SOURCE: Israelski and Muto (2006)
|
From page 264... ...
The probability of the individual gates and the overall the branches leading fault tree probability are computed by using numerical to the top-level combinatorial rules for various logic gates. hazard.
|
From page 265... ...
Special software required for rapid development of fault trees. USABILITY EVALUATION METHODS Overview This section explains how usability evaluation methods can contribute to systems development by providing feedback on usability problems and validating the usability of a system.
|
From page 266... ...
The cost of rectifying any divergence between the design and user needs increases rapidly as development proceeds, which means that user feedback should be obtained as early as possible. Without proper usability evaluation, a project runs a high risk of expensive rework to adapt a system to actual user needs or of potential rejection of the product or system.
|
From page 267... ...
. Summative usability testing of an existing system can be used to provide baseline measures that can form the basis for usability requirements (i.e., objectives for human performance and user satisfaction ratings)
|
From page 268... ...
• Summative methods measure the quality in use of a product and can be used to establish and test usability requirements. Summative usability testing, normally based on the principles of ISO 9241-11 (International Organization for Standardization, 1998)
|
From page 269... ...
perform Comparison Identify Early in 5-8 Can be combined with strengths and design benchmarking. weaknesses of an existing design Summative Usability Testing Benchmarking Real users and Prior to 8-30 To provide a basis for Competitive real tasks design setting usability criteria.
|
From page 270... ...
Some models have the potential advantage that they can be used without the need for any prototype. However, setting up a model usually requires a detailed task analysis, so model-based methods are most cost-effective in situations in which other methods are impracticable, or the information provided by the model is a cost-effective means of managing particular risks.
|
From page 271... ...
Conformance to detailed user interface guidelines or style guides is an important prerequisite for usability, as it can impose consistency and conformance with good practice. But as published sources typically contain several hundred guidelines (e.g., ISO 9241 series)
|
From page 272... ...
. TABLE 8-6 Types of Expert-Based Evaluation Methods Task Scenarios Guidelines No Yes None Expert review Usability walkthrough Pluralistic walkthrough General guidelines Heuristic inspection Heuristic walkthrough Detailed usability guidelines Guidelines inspection Guidelines walkthrough Information processing view N/A Cognitive walkthrough SOURCE: Adapted from Gray and Salzman (1998)
|
From page 273... ...
Strengths, Limitations, and Gaps There is a long history of research into the usefulness of different types of usability evaluation, resulting in broad agreement on the value and importance of including it in any system development for human-intensive systems. While the costs and benefits of usability evaluation are well established (Bias and Mayhew, 2005)
|
From page 274... ...
There may also be a temptation to apply the same evaluation procedure to every project, although the most effective approach will depend on a wide range of issues, including the availability and diversity of users, the range of tasks, and the potential risks of poor usability (see Chapter 4 and the first section of this chapter)
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.