Human-System Integration in the System Development Process A New Look (2007) / Chapter Skim
Currently Skimming:
10 Conclusions and Recommendations
10 Conclusions and Recommendations
Pages 296-330
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 296... ...
. Our most fundamental conclusion is that human performance and human-system integration will never be most effective in system design unless it is seen by all stakeholders as an integral part of the entire systems engineering process, from initial exploration and concept evaluation through operational use, reengineering, and retirement.
|
From page 297... ...
As a process for integrating human considerations into the systems engineering process, the committee has built on the strengths of existing systems engineering process models (waterfall, V-model, concurrent, incremental, spiral, evolutionary, agile) to synthesize an incremental commitment model (ICM)
|
From page 298... ...
It is essential that the HSI team contribute an evaluation of HSI risks and opportunities to be considered in collaboration with the rest of the system engineering team. It is through the risk analysis that the argument may be made for assigning resources to evaluate particular risks further or to find ways to mitigate the risks that the system will fail, for example, because of safety risk, risk that it will be too costly to train the personnel in its use, or risk that it will be maladapted to the people who must use it.
|
From page 299... ...
Effort should be made to create these shared representations in a form that is readily assimilated into the engineering process, that is, expressed in terms that are compatible with other engineering outputs. This might be accomplished through the generation of scenarios of use, models and/or simulations based on the task analysis output, or analyses of the context of use.
|
From page 300... ...
. The ability to anticipate likely reverberations of technology insertions early in the design process can contribute substantively to the design of complex systems and systems of systems that are resilient in the face of a wide range of operational perturbations (Hollnagel, Woods, and Leveson, 2006)
|
From page 301... ...
These areas include • Institutionalizing the success factors associated with the incremental commitment model. • Accommodating the emergence of HSI requirements.
|
From page 302... ...
• Fostering more synergy between research and practice. Institutionalizing a System Development Process Based on the Success Factors Through our analyses of more and less successful HSI projects, our evaluation of alternative HSI process models, and our case studies, the committee makes the case that a model like the incremental commitment model better enables the kind of human-system integration that will be needed for the complex, human-intensive systems of the future.
|
From page 303... ...
Department of Defense and other govern ment and private organizations should revise current system acquisition policies and standards to enable incremental, evolutionary, capabilities based system acquisition that includes HSI requirements and uses risk driven levels of requirements detail, particularly for complex systems of systems and for collaboration-intensive systems. HSI Operational Requirements in Contracts and Acquisition Documents In discussing risk management, we have alluded to the importance of considering HSI aspects when negotiating baseline metrics for program execution.
|
From page 304... ...
Overall, the procuring agencies are able to directly influence the extent to which HSI elements are addressed in contracts by establishing wellarticulated HSI requirements reflective of end-user needs and working with the contractor to establish verification and validation methods that overcome program management concerns about the typically subjective nature of HSI elements. The contractors or suppliers should take the time to involve HSI practitioners in their business development efforts to account for HSI elements in the business offer, thereby mitigating a portion of potential HSI risks and issues that may arise during program execution.
|
From page 305... ...
and constraints across design phases. Our vision is to adapt existing tools or to develop new software tools to facilitate the traceability of HSI design objective implications and how they are being met to ensure that they are preserved across design phases.
|
From page 306... ...
Recommendation: Adapt existing or develop new methods and tools that facilitate capture and traceability of HSI design objectives, design rationale, and constraints across design phases. Specifically: 1.
|
From page 307... ...
2. Consider a specific area, such as cognitive task analysis or risk analysis: a.
|
From page 308... ...
be applied to human-system integration? Similarly, how can other HSI methods, such as cognitive task analysis and participatory design, be adapted for this complexity?
|
From page 309... ...
5. Undertake studies to develop methods and tools for analysis and design of resilient systems that foster adaptability to cope with unanticipated disturbances and change (Hollnagel, Woods, and Leveson, 2006)
|
From page 310... ...
Research is needed to develop a framework for integrating and adapting HSI methods and techniques into complex system development environments, supported by a tool implementing the framework that can be used to select the most cost-effective methods and techniques based on operational, business, organizational, and project needs. Human-system integration and systems engineering activities rely on different methods, techniques, languages, and tools.
|
From page 311... ...
Recommendation: Develop a framework for integrating and adapt ing HSI methods and techniques into complex system development environments. Recommendation: Establish a top-down framework for integrating human-system integration with contrasting development environments to provide the common ground to leverage the integration of HSI meth ods, languages, and techniques into systems development.
|
From page 312... ...
, develop a consensus-based taxonomy of skills, knowledge, and abili ties by surveying leading HSI subject matter experts in both commercial and military domains. Use the definitions and as sumptions from Booher (2003a, 2003b)
|
From page 313... ...
3. What is the rationale for selecting alternative HSI methods for dif ferent purposes?
|
From page 314... ...
While there are many examples of excellent HSI designs, their successes rely heavily on local knowledge and expertise. There is a need to develop methods and tools to more effectively leverage the knowledge and insights gained from practice and improve the cross-dialogue between research and practice.
|
From page 315... ...
This includes capture of the results of task and cognitive task analyses, field observations, participatory analysis and design activities, contextual inquiry, and work domain analyses. The research objective is to provide a suite of software tools to enable analysts to build and maintain a core corpus of work domain and context of use knowledge that can be updated easily as new information is learned, communicated to stakeholders effectively, and accessed and reused more readily across the life cycle of a development project.
|
From page 316... ...
Conduct research (lab and field studies) and develop designs and technology to support user control or influence over the online display of the user's identity, i.e., impression manage ment (Goffman, 1956)
|
From page 317... ...
and develop designs and technology to support users in developing shared repre sentations that effectively communicate the users' needs, goals, intentions, strategies, and user-generated solutions to problems (individually or collectively with other users)
|
From page 318... ...
Recommendation: Refine event data analysis methods and develop new methods in line with the following series of interrelated activities: 1. Explore the data sources described above for types of data that can be collected without interfering with the users' ongoing work (e.g., keystroke analysis, observational cameras, and transportation data)
|
From page 319... ...
In the United States, most users consider their data privacy to be their own responsibility. By contrast, countries in the European Union are more likely to have rules that govern the privacy of personal data, in which personal data can in some cases include not only private records created by an individual, but also private and public records that make reference to that individual.
|
From page 320... ...
Methods for Defining Requirements and Design The committee makes recommendations concerning the research and development needs related to human-system development and to developing prototypes of organizations and training programs. Human-System Model Development Human-system models have been shown already to be useful in the system acquisition and development process as a means to reduce uncertainty and development risk; however, they are not employed to the extent that even the current state of development would justify.
|
From page 321... ...
Recommendation: Pick, as a case study, a class of models at an in termediate level of complexity and invent a high-level human-system model development language, having as its goal to make building such models as simple as customizing an Excel spreadsheet to a specific application. Recommendation: Explore the applicability of computer learning and adaptation algorithms for growing more robust models.
|
From page 322... ...
Currently, when models are applicable, how much risk they reduce, or how valid they need to be to reduce risk is not well defined or even well explored. Models and simulations have the potential to serve as effective shared representations for communicating the state of system development across the range of stakeholders.
|
From page 323... ...
The real challenge comes when the human operator, team, or organization must be considered in a more inclusive HSI design effort. It is clear from increasingly complex system development efforts that the earlier HSI issues can be addressed, the better.
|
From page 324... ...
But their use is not employed very often or consistently. The main goal of specifying usability objectives (also known as usability requirements, usability goals, performance goals, human factors requirements)
|
From page 325... ...
• 95 percent of technicians with no prior experience with this type of network management system will achieve the target mastery level in 2 or fewer hours of use. Recommendation: For cases in which usability objectives have been shown to be useful, conduct research to develop better ways to inves tigate, set, and use them as acceptance criteria.
|
From page 326... ...
2. Improve methods for creating and setting usability objectives by surveying methods that have been used and their strengths and weaknesses; conducting experimental research on the relationship of using risk-management techniques, like failure mode and effects analysis and fault tree analysis, to set utility objectives and whether these projects are successful and meet project/mission goals; and searching the literature in other domains, such as software, electri cal engineering, and the like and how they have used quantifiable performance objectives and how they set and validate them.
|
From page 327... ...
Recommendation: Conduct research to establish how precisely the evaluation procedure needs to be specified to ensure that two organiza tions will produce acceptably similar usability measures for summative evaluation. Identify and Assess HSI Risks It is often stated in the HSI discipline that usability or human factors risks that are not addressed in the engineering design process are the basis
|
From page 328... ...
We see this research activity as going well beyond the typical cost justification exercise for human factors engineering and resulting in a systems model of HSI risks. Recommendation: Conduct research to develop a robust HSI risk tax onomy and a set of methodologies for analyzing and comparing rel evant risk representations and conflicting values.
|
From page 329... ...
Specifically: 1. Survey communication techniques in other domains, such as ad vertising, sales, and news, and categorize success factors that could apply to business and operational risk communications.
|
From page 330... ...
This has led to a newly emerging area called Resilience Engineering that attempts to advance the study and design of systems that exhibit resilience (Hollnagel, Woods, and Leveson, 2006; Woods and Hollnagel, 2006)
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.