Digital Instrumentation and Control Systems in Nuclear Power Plants: Safety and Reliability Issues (1997)

Nuclear power plants rely on instrumentation and control (I&C) systems for monitoring, control, and protection. During their extensive service history, analog I&C systems have performed their intended monitoring and control functions satisfactorily. Although there have been some design problems, such as inaccurate design specifications and susceptibility to certain environmental conditions, the primary concern with the extended use of analog systems is effects of aging, e.g., mechanical failures, environmental degradation, and obsolescence.

The industrial base has largely moved to digital-based systems¹ and vendors are gradually discontinuing support and stocking of needed analog spare parts. The reason for the transition to digital I&C systems lies in their important advantages over existing analog systems. Digital electronics are essentially free of the drift that afflicts analog electronics, so they maintain their calibration better.² They have improved system performance in terms of accuracy and computational capabilities. They have higher data handling and storage capacities, so operating conditions can be more fully measured and displayed. Properly designed, they can be easier to use and more flexible in application. Indeed, digital systems have the potential for improved capabilities (e.g., fault tolerance, self-testing, signal validation, process system diagnostics) that could form the basis for entirely new approaches to achieve the required reliabilities. Because of such potential advantages, and because of the general shift to digital systems and waning vendor support for analog systems, the U.S. nuclear power industry expects substantial replacement of existing, aging analog systems with digital I&C technology. For the same reasons, designs for new, advanced nuclear power plants rely exclusively on digital I&C systems.

Successful introduction of digital I&C systems into U.S. nuclear power plants faces several challenges:

• uncertainty inherent in introduction of new technology

• shift of existing technology base from analog experience

• technical problems identified from some applications of digital I&C in nuclear power plants

• difficult, time-consuming, and customized licensing approach

• lack of consensus (between the U.S. Nuclear Regulatory Commission [USNRC] and the regulated industry) on issues underlying evaluation and adoption of digital I&C technology and means to obtain a satisfactory resolution

In essence, the problem is to develop a systematic regulatory review and approval methodology for digital I&C systems that allows obtaining the safety and reliability benefits available from this technology while avoiding the introduction of offsetting safety problems.

The transition from analog to digital I&C systems in nuclear power plants is not straightforward; one must carefully account for the ways in which digital I&C implementations are different and frame regulations that reflect those differences.

The USNRC has reviewed a number of analog-to-digital ''retrofits" in nuclear power plant I&C systems and is in the

process of reviewing designs of advanced plants. However, the review process has largely been customized for each application because of the lack of agreed-upon applicable criteria.³ In addition, advisory committees, including the Advisory Committee on Reactor Safeguards (ACRS) and the Nuclear Safety Research Review Committee (NSRRC), have expressed concern that the USNRC may be lagging behind in its understanding of digital I&C systems and have urged the development of a framework to guide the regulation of digital I&C technology.

To address technical concerns, and in hopes of developing a wide consensus across the USNRC and the nuclear industry for a regulatory program, the USNRC held a workshop in September 1993. While a useful forum, the workshop did not lead to a consensus, and the USNRC requested the assistance of the National Research Council.

The National Research Council was asked by the USNRC to conduct a study (including a workshop) on application of digital I&C technology to commercial nuclear power plant operations. The National Research Council accordingly appointed a committee (hereafter the committee) to carry out the study, which was conducted in two phases. In Phase 1, the committee was charged to define the important safety and reliability issues that arise from the introduction of digital I&C technology in nuclear power plant operations, including operations under steady-state, transient, and accident operating conditions. In response to this charge, the committee identified eight key issues associated with the use of digital I&C systems in existing and advanced nuclear power plants.

In Phase 2 of the study, the committee was charged to identify criteria for review and acceptance of digital I&C technology in both retrofitted reactors and new reactors of advanced design; to characterize and evaluate alternative approaches to the certification or licensing of this technology; and where sufficient scientific basis exists, recommend guidelines on the basis of which the USNRC can regulate and certify (or license) digital I&C technology, including means for identifying and addressing new issues that may result from future development of this technology. In areas lacking sufficient scientific basis to make such recommendations, the committee was to suggest ways in which the USNRC could acquire the required information.

In carrying out its Phase 2 charge, the committee limited its work to those issues identified in Phase 1. The issues were chosen because they were difficult and controversial. Further, the committee recognized that by law, the responsibility for setting licensing criteria and guidelines for digital I&C applications in nuclear plants rests with the USNRC. Thus, the reader should not form too literal an expectation that the committee has provided a cogent set of principles, design guidelines, and specific requirements for ready use by the USNRC to assess, test, license, and/or certify proposed systems or upgrades. Rather, the results of the study are presented in the form of conclusions and recommendations related to each issue and primarily addressed to the USNRC for their consideration and use. In the committee's view, there is substantial further work to be accomplished. The committee expects the USNRC and the nuclear industry to extend the work of criteria development beyond where this Phase 2 report leaves it. To guide further work, the committee's report offers findings and recommendations in four broad categories: (a) current practice that is essentially satisfactory or requires some fine tuning, (b) points of weakness in the USNRC's approach, (c) issues that merit further inquiry and research before satisfactory regulatory criteria can be developed, and (d) criteria and guidelines that are unreasonable to expect in the near future.

Digital instrumentation and control systems for nuclear power plants have technological characteristics—equipment, response time, input and output range, and accuracy—very similar to those of digital instrumentation and control systems for other safety-critical applications such as chemical plants and aircraft. What distinguishes digital I&C applications in nuclear power plants from other digital I&C applications is the need to establish very high levels of reliability and safety under a wide range of conditions. Because of the potentially far greater consequences of accidents in nuclear power plants, the I&C systems must be relied upon to reduce the likelihood of even low-probability events. The USNRC has developed a regulatory process with the goal of achieving these high levels of reliability and thus assuring public safety. This process is subject to public scrutiny.

In Phase 1 of the study, the committee identified eight key issues associated with the use of digital I&C systems in existing and advanced nuclear power plants. In the committee's view, these issues need to be addressed and a working consensus needs to be established regarding these issues among designers, operators and those responsible for

maintenance of such systems, and regulators in the nuclear industry. The process the committee followed to identify these issues is discussed in the Phase 1 report and is only briefly summarized here.

In essence, the committee considered the impact of digital I&C systems against a set of standard regulatory approaches to assessing and ensuring safety (defense-in-depth, safety margins, environmental qualification, quality assurance, and failure invulnerability). From this analysis, the committee identified a number of questions and issues. After extensive deliberations, the committee selected eight key issues.

The eight issues can be separated into six technical issues and two strategic issues. The six technical issues are systems aspects of digital I&C technology, software quality assurance, common-mode software failure potential, safety and reliability assessment methods, human factors and human-machine interfaces, and dedication of commercial off-the-shelf hardware and software. The two strategic issues are the case-by-case licensing process and the adequacy of technical infrastructure (i.e., training, staffing, research plan). The committee recognizes that these are not the only issues and topics of concern and debate in this area. Nevertheless, the committee reaffirms its judgment, initially formed during Phase 1, that developing a consensus on these eight issues will be a major step forward and accelerate the appropriate use and licensing of digital I&C systems in nuclear power plants.

In conducting Phase 2 of its study the committee employed a systematic process, which is reflected in the structure of most of the chapters in this report. The committee reviewed a large number of documents made available by the USNRC and variety of other sources. The committee also interviewed selected personnel from the USNRC, from the two advisory committees discussed above (ACRS, NSRRC), from the nuclear industry,⁴ and from other industries⁵ using digital systems in safety-critical applications. The committee also sought the view of individuals from academia and research organizations. In addition, the committee visited control room simulators, a nuclear plant, and a fossil-fueled power plant with extensive digital I&C systems. The committee also had frequent and detailed internal discussions, both face-to-face and via paper and electronic communications. The committee also brought to bear a wide range of experience in and knowledge of the field.

The committee took seriously the charge that it identify criteria for review and acceptance of digital I&C technology and that it recommend guidelines for regulation and certification. In carrying out its charge, the committee recognized that:

• In order to develop useful guidance, only a limited number of issues could be dealt with in the relatively brief duration of the study.

• General, high level criteria would not be particularly useful.

• The final criteria are legally the USNRC's responsibility. Further, since the nuclear power industry is heavily regulated in the public interest, the licensing criteria should be forged in a detailed interaction among the regulators, the industry, and the public.

• The committee has a wide range of expertise and experience in digital systems and nuclear power plants but it is not a surrogate for this interaction among the stakeholders. Hence, the committee could serve by clearly delineating and defining issues and providing guidance for resolving these issues rather than developing specific licensing criteria.

Accordingly, the committee selected eight issues for study and worked on those issues. These eight issues address the two major intertwined themes associated with the use of digital instrumentation and control in nuclear power plants. These are:

1. Dealing with the specific characteristics of digital I&C technology as applied to nuclear power plants.

2. Dealing with a technology that is more advanced than the one widely in use in the existing nuclear power plants. This technology is rapidly advancing at a rate and in directions largely uncontrolled by the nuclear industry but at the same time likely to have a significant impact on the operation and regulation of the nuclear industry.

The technical issues the committee focuses on first in this report are primarily related to digital technology itself (Theme 1), while the strategic issues that follow are primarily related to the process of adopting advanced technology (Theme 2). The committee concentrated on reviewing the current approaches being taken by the nuclear industry and its regulators toward dealing with the selected key issues. The committee also tried to learn from the experience of the international nuclear industry as well as gather and evaluate information about how other safety-critical industries and their regulators dealt with these issues. Also, through the technical expertise and knowledge of its various members, the committee explored work done by the digital systems community at large, including both research activities and academic work.

As the committee worked through the issues it discovered there is a major impediment to progress. This is the communication barriers that exist among the key technical communities and individuals involved. The basic reason for the communication difficulty is apparent. Work is simultaneously going on in many areas, each with its own technology, research focus, and agenda. Unfortunately, although many of these areas use common terms, these terms often have different meanings to different groups, resulting in either a lack of communication or very difficult communication. This is particularly troublesome for the nuclear power industry and its regulators, who are not dominant in this technology and must try to synthesize information and experience from a variety of sources and apply it in power plants where safety hazards must be dealt with in a rigorous way, under public scrutiny. In Chapter 11 the committee discusses this communication problem in more detail and provides suggestions for a way forward. Making substantial progress in this area should have a multiplicative effect as it eases the resolution of many specific technical and strategic issues.

Overall, while there are important steps that remain to be taken by the USNRC and industry as addressed in this report, the committee found no insurmountable barriers to the use of digital instrumentation and control technology to nuclear power plants. The committee also believes that a forward-looking regulatory process with good and continuing regulations and industry communication and interaction will help. All participants must recognize that crisp, hard-edged criteria are particularly difficult to come by in this rapidly moving area and good practices and engineering judgment will continue to be needed and relied upon.

For the key technical issues (systems aspects of digital I&C technology; software quality assurance; common-mode software failure potential; safety and reliability assessment methods; human factors and human-machine interfaces; and dedication of commercial off-the-shelf hardware and software) the committee provides specific recommendations and conclusions which include a number of specific criteria. These are listed in each chapter (see Chapters 3 through 8). But recognizing the difficulty of defining specific criteria, and the need for the nuclear technology stakeholders, particularly the USNRC, to make the final decisions, the committee focused on (a) providing process guidance both in developing guidelines and in the short-term acceptance of the new technology; (b) identifying promising approaches to developing criteria and suggestions for avoiding dead-ends; and (c) mechanics for improving communication and strengthening technical infrastructure.

For the key strategic issues (the case-by-case licensing procedure and adequacy of the technical infrastructure) the committee:

• Emphasizes guidance to implement a generically applicable framework for regulation that follows current USNRC practice and draws a distinction between major and minor safety modifications. The committee also provides guidance for the evaluation and updating of this regulatory framework (see Chapter 9).

• Identifies a need to upgrade the current USNRC technical infrastructure and suggests specific research activities that will support the needed regulatory program and USNRC's research needs. The committee also suggests several improvements to the technical infrastructure to improve and maintain technical capabilities in this rapidly moving, technically challenging area.

The results of this process are set forth below, where the committee introduces each of the key issues—first the technical, then the strategic—with an "issue statement" developed during Phase 1 of the study. Following each issue statement are the conclusions and recommendations formulated by the committee during Phase 2 of the study.

Issue Statement. Along with important benefits, digital I&C systems introduce potential new failure modes that can affect operations and margins of safety. Therefore, digital I&C systems require rigorous treatment of the systems aspects of their design and implementation. What methods are needed to address this concern? How can the experience and best practices of the various technical communities involved in applying digital I&C technologies be best integrated and applied to nuclear power plants? What procedures can be put in place to update the methods and the experience base as new digital I&C technologies and equipment are introduced in the future?

Conclusion 1. Continued effort is warranted by the USNRC and the nuclear industry to deal with the systems aspects of digital I&C in nuclear power plants.

Conclusion 2. The lack of actual design and implementation of large I&C systems for U.S. nuclear power plants makes it difficult to use learning from experience as a basis for improving how the nuclear industry and the USNRC deal with systems aspects.

Conclusion 3. The USNRC's intent to upgrade their regulatory guidance in the systems aspects of digital I&C applications in nuclear power plants is entirely supported by the committee's observations about systems aspects.

Conclusion 4. Existing regulatory guidance lacks the specificity needed to be effective, and the revision should address this shortcoming.

Recommendation 1. The USNRC should make a trial application of the proposed regulatory guidance documents on systems aspects to foreign nuclear plant digital systems, both

existing and in progress. In particular, this review should focus on assessing whether or not the revised guidance documents have the necessary level of specificity to adequately address the systems aspects of nuclear plant digital I&C implementations.

Recommendation 2. The USNRC should identify and review systems aspects guidance documents provided in other industries, such as chemical processing and aerospace, where large-scale digital I&C systems are used. The focus of this review would be to compare these other guidance documents with those being developed by the USNRC, paying due attention to common problems and application-specific differences.

Recommendation 3. To obtain practical experience, the USNRC should loan staff personnel, perhaps on a reciprocal basis, to other agencies involved in regulating or overseeing large safety-critical digital I&C systems.

Recommendation 4. The USNRC should require continuing professional training for appropriate staff in technologies particularly germane to systems aspects, such as fault-tolerant, distributed systems.

Issue Statement. The use of software is a principal difference between digital and analog I&C systems. Quality of software is measured in terms of its ability to perform its intended functions. This, in turn, is traced to software specifications and compliance with these specifications. Neither of the classic approaches of (a) controlling the software development process or (b) verifying the end-product appears to be fully satisfactory in assuring adequate quality of software, particularly for use with safety-critical systems. How can the USNRC and the nuclear industry define a generally accepted, technically sound solution to specifying, producing, and controlling software needed in digital I&C systems?

Conclusion 1. Software quality assurance procedures typically monitor process compliance rather than product quality. In particular, there are no generally accepted evaluation criteria for safety-related software; rather, standards and guidelines help to repeat best practices. Because most software qualities related to system safety, e.g., maintainability, correctness, and security, cannot be measured directly, it must be assumed that a relationship exists between measurable variables and the qualities to be ensured. To deal with this limitation, care must be taken to validate such models, e.g., using past development activities, and to assure that the measurements being made are appropriate and accurate in assessing the desired software qualities.

Conclusion 2. Prior operating experience with particular software does not necessarily ensure reliability or safety properties in a new application. Additional reviews, analysis, or testing by a utility or third-party dedicator may be necessary to reach an adequate level of assurance.

Conclusion 3. Testing must not be the sole quality assurance technique. In general, it is not feasible to assure software correctness through exhaustive testing for most real, practical I&C systems.

Conclusion 4. USNRC staff reviews of the verification and validation process used during software development seem quite thorough.

Conclusion 5. Exposing software flaws, demonstrating reliable behavior of software, and finding unintended functionality and flaws in requirements are different concepts and should be assessed by a combination of techniques including:

• Systematic inspections of software and planned testing with representative inputs from different parts of the systems domain can help determine if flaws exist in the software.

• Functional tests can be chosen to expose errors in normal and boundary cases, and measures of test coverage can be reported for them.

• Testing based on large numbers of inputs randomly selected from the operational profiles of a program can be used to assess the likelihood that software will fail under specific operating conditions.

• Requirements inspections can be an effective method for detecting software defects, provided requirements are studied by several experienced people who did not participate in their construction. The effectiveness of these reviews also depends on the quality of the requirements.

• A system-level hazard analysis can identify states that, combined with environmental conditions, can lead to accidents. The analysis should extend into software components to ensure that software does not contribute to system hazards.

Conclusion 6. The USNRC research programs related to software quality assurance appear to be skewed toward investigating code-level issues, e.g., coding in different languages to achieve diversity and program slicing to identify threads containing common code.

Conclusion 7. Rigorous configuration management must be used to assure that changes are correctly designed and implemented and that relationships between different software artifacts are maintained.

Conclusion 8. Software is not more testable simply because the design has been implemented on a chip. Use of any technology requiring equivalent design effort to software requires commensurate quality assurance. For example, this conclusion applies to ASIC (application-specific integrated circuit), PLC (programmable logic controllers), and FPGA

(field programmable gate arrays). However, the committee notes that these technologies may be useful in addressing some configuration management problems.

Recommendation 1. Currently, the USNRC's path is to develop regulatory guides to endorse (with possible exceptions) a variety of industry standards. The USNRC should develop its own guidelines for software quality assurance that focus on acceptance criteria rather than prescriptive solutions. The draft regulatory guide, Software in Protection and Control Systems, by Canada's Atomic Energy Control Board is an example of this type of approach. The USNRC guidelines should be subjected to a broad-based, external peer review process including (a) the nuclear industry, (b) other safety-critical industries, and (c) both the commercial and academic software communities.

Recommendation 2. Systems requirements should be written in a language with a precise meaning so that general properties like consistency and completeness, as well as application-specific properties, can be analyzed. Cognizant personnel such as plant engineers, regulators, system architects, and software developers should be able to understand the language.

Recommendation 3. USNRC research in the software quality assurance area should be balanced in emphasis between early phases of the software life cycle and code-level issues. Experience shows that the early phases contribute more frequently to the generation of software errors.

Recommendation 4. The USNRC should require a commensurate quality assurance process for ASICs, PLCs, and other similar technologies.

Issue Statement. Digital technology introduces a possibility that common-mode software failures may cause redundant safety systems to fail in such a way that there is a loss of safety function. Various procedures have been developed and evolved for evaluating common-mode failure potential in analog devices. Do these same procedures apply to computers and software or are different approaches to ensuring reliability needed? What does software diversity mean? Can it be achieved and assessed and, if so, how? Do techniques exist for assessing common-cause failure and common-mode failure when computers are involved? What are the implications of common-mode software failure for the licensing process and the use of component diversity? Are redundancy and diversity the most effective way to achieve reliability for digital systems?

Conclusion 1. The USNRC position of assuming that common-mode software failure could occur is credible, conforms to engineering practice, and should be retained.

Conclusion 2. The USNRC position with respect to diversity, as stated in the draft branch technical position, Digital Instrumentation and Control Systems in Advanced Plants, and its counterpart for existing plants, is appropriate.

Conclusion 3. The USNRC guidelines on assessing whether adequate diversity exists need to be reconsidered. With regard to these guidelines: (a) The committee agrees that providing digital systems (components) that perform different functions is a potentially effective means of achieving diversity. Analysis of software functional diversity showing that independence is maintained at the system level and no new failure modes have been introduced by the use of digital technology is no different from that for upgrades or designs that include analog instrumentation. (b) The committee considers that the use of different hardware or real-time operating systems is potentially effective in achieving diversity provided functional diversity has been demonstrated. With regard to real-time operating systems, this applies only to operating systems developed by different companies or shown to be functionally diverse. (c) The committee does not agree that use of different programming languages, different design approaches meeting the same functional requirements, different design teams, or different vendors' equipment used to perform the same function is likely to be effective in achieving diversity. That is, none of these methods is a proof of independence of failures. Conversely, neither is the presence of these proof of dependence of failures.

Conclusion 4. There appears to be no generally applicable, effective way to evaluate diversity between two pieces of software performing the same function. Superficial or surface (syntactic) differences do not imply failure independence, nor does the use of different algorithms to achieve the same functions. Therefore, funding research to try to evaluate design diversity does not appear to be a reasonable use of USNRC research funds.

Conclusion 5. Although many in the software community believe that there are more cost-effective techniques for achieving high software reliability than redundancy and diversity, there is no agreement as to what these alternatives may be. The most promising of these appear to be the extension of standard safety analysis and design techniques to software and the use of formal (mathematical) analysis.

Conclusion 6. The use of self-checking to detect hardware failures and some simple software errors is effective and should be incorporated. However, care must be taken to assure that the self-checking features themselves do not introduce errors.

Recommendation 1. The USNRC should retain its position of assuming that common-mode software failure is credible.

Recommendation 2. The USNRC should maintain its basic position regarding the need for diversity in digital I&C systems as stated in the draft branch technical position, Digital