Potential Risks and Benefits of Gain-of-Function Research: Summary of a Workshop (2015)

2


Assessing Risks and Benefits

Dr. Charles Haas, Drexel University, a member of the symposium planning committee, summarized the standard risk assessment process. The major steps in risk assessment were first articulated in a National Research Council report titled Risk Assessment in the Federal Government: Managing the Process (NRC, 1983), otherwise known as the “Red Book.” This report has been updated several times (see NRC 1994, 1996, and 2009). The basic framework laid out for risk assessment consists of the steps in Box 2-1.

However, there are also other considerations besides following these technical steps, and Drs. Baruch Fischhoff (Carnegie Mellon University), Gavin Huntley-Fenner (Huntley-Fenner Advisors, Inc.), and Monica Schoch-Spana (University of Pittsburgh Medical Center [UPMC] Center for Health Security) elaborated on these and provided further details about crucial considerations that need to be taken into account in risk assessments. These comments are summarized later in this chapter.

Haas noted that the major focus of attention with regard to Gain-of-Function (GoF) research has been on hazard assessment. This encompasses occupational health risks, but needs to go beyond this to risks to the members of the public near research sites as well as global risks for pandemic organisms. A number of questions in this arena need to be addressed in a risk assessment, stated Haas. Do the safety records of high containment laboratories provide an appropriate basis for quantifying the risks of lab accidents that lead to worker or public exposures or are there more systematic approaches that need to be incorporated into a

risk assessment? Are there finer gradations of lab capabilities that must be considered that go beyond the BSL/ABSL (biosafety level/animal biosafety level) framework, for example, the competence of the laboratory staff and the steps taken by the host institution for community preparedness (see comments by Dr. Rebecca Moritz of the University of Wisconsin’s Biosecurity Task Force in Chapter 5)? And how is deliberate misuse of either the pathogens themselves or the information obtained through the research on these pathogens to be incorporated into the risk assessment?

Haas noted that the debate on GoF research has paid scant attention to either exposure assessment or dose response assessment. Both are crucial components of a risk assessment, although it is likely that at least for dose response, there is little information available, particularly for Middle East Respiratory Syndrome (MERS) virus and possibly also for Severe Acute Respiratory Syndrome (SARS) virus. As a consequence, the GoF research debate has jumped directly into the risk characterization stage without the benefit of the missing intermediate analyses and the dissection of the exposure and dose response issues that may make considerable differences in how the risk characterization is framed. It was Haas’s view that the current risk characterization picture contains too many lumped parameters, combining factors dealing with environmental effects, host properties, and infectious agents. All of these need to be taken into account when estimating outcomes and require more attention, as does the role of uncertainty. Very often we do not necessarily know that

we have incorporated all of the factors that may influence uncertainty. Similarly, the full fundamental basis of risk assessment is missing for the risk management considerations for GoF research, although it is still possible to discuss to what degree biological and methodological modifications can reduce or obviate risk. The risks of not doing the proposed work, highlighted in several talks and comments during the symposium, also should be considered and balanced against the risks of doing the research. Finally, Haas noted that a risk assessment can inform decisions, but is not determinate per se. The concept of “acceptable” risk is a trans-scientific issue that will be more appropriately addressed in the policy arena.

In Session 8 of the symposium, Baruch Fischhoff, another member of the symposium planning committee, gave an overview of what risk/benefit assessment can and cannot do, as well as what has been learned from past attempts to conduct risk/benefit assessments. He recommended a book, Risk: A Very Short Introduction (Fischhoff and Kadvany, 2011), in which the authors use simple conceptual frameworks from decision theory and behavioral research to explain the science and practice of creating measures of risk, how scientists address risks using historical records, scientific theories, probability, and expert judgment, and what cognitive scientists have learned about how people deal with risks and how these lessons apply to diverse examples and demonstrate how understanding risk can improve making choices in everyday life and public policy.

Fischhoff outlined the key considerations related to the risk assessment paradigm above. These considerations include:

1. Defining “risk” and “benefit”
2. Assessing risks and expected benefits
3. Communicating risks and expected benefits
4. Organizing to reduce risks and increase expected benefits

For the last item, he noted that for GoF research, the expected benefits are potentially reduced risks. For this reason, the same methodologies apply to assessing risks and expected benefits.

DEFINING “RISK” AND “BENEFIT”

Fischhoff stated that the terms of all analyses embody values that favor some interests above others. Thus, when transparent, the underlying assumptions can be controversial and, therefore, an analytical and deliberative process is required to create socially acceptable definitions. Such analyses utilize science to inform estimates, but they also depend on subjective value judgments about what metrics to include and how much weight to put on each. One commonly used metric is risk of death,

which can be defined as the risk that somebody dies, or in terms of the probability that someone exposed to a hazard dies prematurely, or the number of years of life that are expected to be lost with each death. A further refinement of this metric may assign higher value to deaths of particular groups, for example, young people. Other bases for evaluating the value of death as an outcome of risk include whether the deaths are equitably distributed, voluntarily assumed, well understood, controllable, or borne by future generations. Echoing Haas, Fischhoff noted that choosing among these and other alternatives require making value judgments, which is a role for the policy makers.

ASSESSING RISKS AND (EXPECTED) BENEFITS

Fischhoff noted these key needs for risk assessments:

• Socially acceptable outcomes defined
• Factors that are believed to affect outcomes identified
• Factors and interdependencies assessed based on observation and expert judgment
• Quality of the evidence assessed

Fischhoff urged policy makers to have a clear idea of what the purpose of a particular risk/benefit analysis is so that the analysis suits its purpose. He noted that risk analyses can be either for purposes of “design” or to inform decisions. Analyses for purposes of design identify better options to improve understanding of complex systems. Analyses to inform decisions focus on the acceptability of risks (given the expected benefits) by predicting outcomes. As an example of the former, Fischhoff cited a 1975 Reactor Safety Study known as “WASH-1400” (USNRC, 1975) that attempted to assess the risk of accidents at commercial nuclear power plants in the United States. The study was later critiqued by an ad hoc review group that stated the following:

This example illustrates two notable things. First, risk assessments on low probability/high consequence events are not new. Second, the roles of uncertainty as well as human factors (see more below) are crucial in risk assessment. As also pointed out by Haas, risk assessments generally are forced to deal with considerable uncertainty, which needs to be acknowledged and dealt with. As Huntley-Fenner added later during the discussion, the fact that certain types of accidents, fatalities, and injuries are rare and we do not often see them is interpreted as a sign that things are going well. But the absence of such rare events may not necessarily be a positive sign; it may be that we are just missing the right indicators. If we do not see the data relevant to what accounts for safety, then maybe we are not looking in the right places or in the right way.

HUMAN BEHAVIOR AS A SOURCE OF VULNERABILITY AND RESILIENCE

Fischhoff noted that the contribution of human factors to understanding industrial and other processes has been studied for a very long time, referencing, for example, a study by H.M. Vernon (1921), a member of the English Industrial Fatigue Research Board, on Industrial Fatigue and Efficiency. He also noted that the literature from nuclear power and other sectors makes clear that human behavior must be taken into account as both a source of vulnerability and a source of resilience. Although human error is clearly a problem, human innovation can also rescue difficult situations.

Gavin Huntley-Fenner elaborated on the topic of human factors in his presentation. He defined human factors as “the study of the interrelationships between humans, the tools they use, and the environment in which they live and work.” He provided some data on the role of human error in various accident scenarios: 80 percent of motor vehicle accidents, 80 percent of medical errors, and 60-80 percent of aviation accidents are estimated to be attributable to human factors. He stated that studies have shown that physical (e.g., working in personal protective equipment) and cognitive (e.g., working under conditions of fatigue) stresses undermine human reliability. Not only can human error not be eliminated, but it also has actually increased as a contributor to accidents in some arenas, such as traffic accidents. Analyses of human reliability and errors must identify the critical areas that are incompatible with human capabilities and the areas where a system is vulnerable to human error. He cited a 2009 Gov-

ernment Accountability Office (GAO, 2009) report that found that role of human error is unappreciated.

Huntley-Fenner provided a list of characteristics to guide hazard analysis processes (Box 2-2). He added some key questions to be asked:

• Are task demands compatible with human capabilities and characteristics?
• Has the system been designed to cope with the inevitability of human error?
• Does the system take advantage of unique human capabilities?
  
  
  

According to Huntley-Fenner, the benefits of a risk assessment guided by consideration of human factors are summarized in Box 2-3.

Huntley-Fenner cautioned the audience about our limited capacity to understand and manage risk. He noted that we tend to underestimate risk, are optimistic about our capacity to control local risk, and need to be aware of the potential to accrue benefits (science) and externalize risks (public health). He also, however, highlighted the fact that establishing simple, consistent routines can yield significant reductions in errors, referencing, for example, a paper by Haynes et al. (2009) that reported the use of a simple surgical safety checklist that resulted in a significant decline in errors related to anesthesia in surgical procedures.

Fischhoff further elaborated on the general area of limitations in risk assessment and noted that the limits include variability among observations, the quality of the studies on which the analysis is based (internal validity), whether these studies are generalizable (external validity), and how good the underlying science is (“pedigree”). “These are the standard considerations that a policy maker needs to know in order to make responsible judgments about the risks and benefits of a technology…” said Fischhoff. He stressed the importance of risk communication and taking account of behavioral research that demonstrates how humans tend to make faulty intuitive judgments. He cited two special issues of the Proceedings of the National Academy of Sciences (PNAS)¹, one in 2013 and another in 2014, devoted to “The Science of Science Communication” as well as a Food and Drug Administration Strategic Plan for Risk Communication (USFDA 2009) as good sources of additional information on risk communication. The topic was also elaborated on by Schoch-Spana in her presentation in Session 8.

PUBLIC ENGAGEMENT

Monica Schoch-Spana framed her presentation with four questions: Who is the public? What do we mean by engagement? Why is engaging the public valuable? And what are some take away considerations for the National Science Advisory Board for Biosecurity (NSABB), National Institutes of Health (NIH), and workshop attendees?

Who Is “the Public”?

Schoch-Spana defined “the public” in the broadest sense as all the people who are interested in or affected by GoF research governance decisions. However, who is in that group depends on political jurisdiction

_______________

¹PNAS 2013. vol. 110 Supplement 3 and PNAS 2014 Vol. III Supplement 4.

and many other factors that complicate definitions. Global, national, and local publics are all relevant to this particular debate. In the pandemic context the population at risk is global. Anyone in the world, at least in the abstract, can be equally in danger of infection and equally in need of medical countermeasures potentially informed by GoF research.

In a U.S. context, Schoch-Spana referenced a study by Sandra Quinn and colleagues (Quinn et al., 2011) who proposed that U.S. racial and ethnic minorities were at a threefold disadvantage during the 2009 H1N1 influenza pandemic. These subgroups faced enhanced exposure to the H1N1 virus because of social, economic, and behavioral elements. They faced greater susceptibility to influenza because of the high prevalence of chronic disease and immunosuppression, and they had impaired access to timely and trusted health information, vaccination, and treatment. There are also other “national” publics that come to mind in the United States. Ultimately, the U.S. taxpayer underwrites the cost of government-sponsored research and confers authority and operating budgets on federal bodies implicated in the biosafety systems that have been created and continue to be refined to keep researchers and the larger public safe in the context of GoF and other research of concern.

Schoch-Spana also noted that at the local level there also is another potentially relevant public—the communities that actually host the facilities in which GoF research is conducted. In the case of a laboratory release they could be on the front end of an emerging pandemic. As a result, they have a strong and direct interest in the biosecurity and biosafety systems designed to avert any release and, should prevention fail, they also have a direct interest in locally robust systems to treat the sick and interrupt transmission.

What Do We Mean by “Engagement”?

Schoch-Spana stated that “engagement” usually refers to the processes by which citizens influence the policies and programs that affect them. In a democracy people have a variety of means to make their voices heard. They can vote, write letters, lobby, demonstrate, and take other collective actions. Over the past 50 years more direct means of public participation in the decision-making process itself have developed as citizens have become less deferential toward authorities and public policy issues have become more complicated.

The theories of deliberative democracies have flourished and practical experience in participatory approaches has accumulated. Scholars and practitioners usually talk about public engagement as a flow of influence and information between authorities and constituents. Very simplistically,

there are three different modes of public engagement: communication, consultation, and collaboration.

In the communication mode, an official or an agency conveys information to members of the public in a one-way fashion, often with the intent of educating and informing the public. Public feedback is not required and not necessarily sought (Schoch-Spana, 2007). In the case of the GoF research debate, this could take the shape of press releases, educational websites, and reports, such as the proceedings of meeting such as this one.

The consultation mode is an interaction in which authorities solicit opinions through surveys, polls, and focus groups or during public comment periods. Again this communication is one-way, but it is from the citizens to the authorities. The public’s points of view, criticisms, and constructive advice can inform policy options, but this input is just one of many that decision-makers take into consideration.

The third mode, collaboration, is considered to be a two-way flow of information and influence between citizens and authorities; it is about dialogue fostering better understanding of very complex problems from all sides and perspectives. Collaboration allows an opportunity for collective learning as part of honest and respectful interaction among the authorities and diverse constituents (Schoch-Spana, 2007). Such iterative exchanges, as Fischhoff indicated earlier, are necessary to approach policy concerns that are technically and ethically complex.

Why Is Engaging the Public Valuable?

Schoch-Spana noted that there is a valuable summary in the 2008 NRC report Public Participation in Environmental Assessment and Decision Making, which identified three important justifications for deliberative processes: improving product quality, enhancing legitimacy, and building capacity.

• Improving product quality: Collaboration enhances decision quality by helping to get the science right. People who are not typically considered experts may nonetheless have relevant local knowledge that is sensitive to context. Their input has often been able to correct technical analyses that have been misapplied to local conditions. The public can also bring fresh eyes not encumbered by technical presuppositions that in the end can improve the technical competence of policy decisions.
• Enhancing legitimacy: Participation can serve as a means to inform and elicit the consent of the governed on complex issues in ways that traditional methods such as elections cannot provide. Participatory forms of engagement, when performed in good

• faith, can help build trust between officials and the public and enable officials to consider different points of view, including those of otherwise disenfranchised people. They can also provide evidence even to dissenting participants and nonparticipants that officials have indeed acted in a fair and accountable manner.

• Building capacity: Well-executed public participation builds a foundation of trust and mutual understanding as well as practical experience with dialogue, which can benefit future policy formulation implementation and evaluation. The public can derive greater facility with the science and the political process, and scientists and governing officials can develop a better understanding of public concerns. Such an exchange helps scientists, citizens, and governing officials understand the aspects of a problem that go beyond their immediate circumstances and provides the opportunity for refinement and even the changing of opinion.

Schoch-Spana reiterated a fourth cross-cutting justification for public engagement—navigating uncertainty—which Fischhoff, Huntley-Fenner, and Haas had also mentioned. Involving the public can strengthen the capacity of civil society and technical experts, industry, and government for analysis and reflection on the uncertain and ambiguous nature of many scientific and technological developments. Judgments informed using scientific fact and social values are necessary in the context of unforeseen consequences that can be good or bad or in between and can unfold over decades or more. The benefits of public participation are not merely aspirations. The 2008 NRC report that Schoch-Spana referenced provides information on a large number of studies from across the social sciences that demonstrate these benefits.

Considerations for the NIH and the NSABB

Schoch-Spana concluded with two points that she believes merit further attention for broad public engagement in the proposed GoF assessment—nested engagement and enduring structures. Broader publics at local, national, and global levels could participate in public engagement exercises that are national in scope, diversely populated, and involve technically and ethically complex health security matters, for example, how does one distribute scarce medical resources in an influenza pandemic? Policy makers could consider holding deliberations in communities hosting GoF research laboratories and populating the national conversation to address the health disparities aspects of risks and benefits. Federal agencies and partners such as the National Academies and other interested entities could also encourage their counterparts internationally

to develop comparable deliberative processes. She noted that in 2009 a citizen consultation on climate policy was conducted simultaneously in 38 countries. Transnational consideration of a transnational public health problem seems to Schoch-Spana a reasonable goal to at least consider.

On enduring structures, public engagement on GoF should not be limited to a “one and done” performance. Engagement mechanisms on this issue could serve as a foundation for the development of deliberative systems to tackle analogous dilemmas that are certain to emerge in the future. Participatory endeavors and the diffusion of well-crafted communication products emanating from them are investments in democratic governance. Such efforts would enhance the scientific literacy of citizens as well as the capacity of scientists, their sponsors, and their regulators to represent their work in broadly meaningful ways. Her final takeaway message was “How a decision is made is just as important for many people as the outcome of that decision.”

SUMMARY OF RISK AND BENEFIT OVERVIEW

Fischhoff summarized as follows the tasks to be accomplished by the risk/benefit assessment that the NIH plans to conduct:

• Define the risks and benefits;
• Assess the risks and expected benefits;
• Communicate the risks and expected benefits; and
• Organize to reduce the risks and increase the expected benefits.

The bottom line, he stressed, is that a credible risk benefit analysis must be both technically sound and socially acceptable. There should be a strategic decision on whether to focus on design or decision. There should be proper disciplinary breadth and proper treatment of uncertainty. An ongoing two-way communication with stakeholders is needed to ensure that the assessment receives the credibility it deserves. The process should be organized for transparency and learning.

For this particular endeavor, it is Fischhoff’s view that the NIH would do best to focus on design and on determining how to reduce the risks and increase the expected benefits. He pointed out that the structure of risk analysis is well known and has been used many times. But the benefits side of the equation is more difficult and poses more interesting problems that require an investment in formalizing the benefit arguments as well as formalizing the arguments for those who see alternative paths. It is necessary to know which numbers are really important and whether they are even relevant to an analysis. He noted that the NIH could consult with people who have some experience with these issues, for example,

Michael Gorham and Kevin Dunbar at the University of Virginia; both study scientific discovery processes, at the individual and laboratory levels, and know something about this. It is possible to take advantage of such people who have already studied the world of scientific innovation in quantitative or qualitative terms.

Fischhoff reiterated that the assessment should seek to inform decisions, not presume to make them. “Anybody who thinks that putting out a contract for a risk/benefit analysis will tell the country what to do on this topic is just deluding themselves.” The subjectivities that inevitably exist in setting the terms of any analysis also need to be acknowledged. When taking a multi-attribute approach to things, somebody needs to decide which attributes are the outstanding ones and whether mortality will be measured in terms of probability of premature death or in terms of expected lives saved. In addition, how should the externalized costs and benefits to the rest of the world, i.e., those that can take advantage of breakthroughs in this country if our sociopolitical and economic systems allow, be weighed? There is a need for some socially acceptable way to resolve the subjectivity in scientific judgment, which must be explicitly acknowledged. As scientists we know that all analyses are incomplete. We can do a better job of quantifying things that are often left out, such as human factors, but there are certain things, such as the quality of the underlying research, that will remain matters of judgment. Similarly, the associated uncertainties in the analysis must be elicited and expressed.

Fischhoff also reiterated the importance of considering and evaluating human factors in scientifically sound ways and that public engagement should be treated as an opportunity to increase the public’s literacy and to build trust in a community. This means reaching out, pulling the community into the process, and taking its opinion seriously. It means accommodating the concerns, which are easier to deal with at the beginning than at the end. A good design process is one that does not need some kind of patchwork at the end. Finally, the appropriate level of aggregation for making decisions needs to be determined while considering the variability in the research and the resolution of the decision-making processes. If it is appropriate to evaluate research proposals on a case-by-case basis, then bodies that are properly staffed and resourced and that have credibility with the public to make those case-by-case decisions will be needed.

In the discussion following the end of the Session 8 presentations, Harvey Fineberg noted that Haas included on one of his slides the question “When should the precautionary principle be invoked?” This slide referenced a 1997 report by the Presidential/Congressional Commission on Risk Assessment and Risk Management. In this report, the following comments were provided on the question of use of the “precautionary principle:”

Several participants, including Haas himself, noted that there may be a dearth of information for quantifying many aspects of the GoF research risk assessment. Dr. Michael Imperiale of the University of Michigan commented that there is a lot of debate about how one quantifies the risks and benefits and that there are different ways to look at this question. People can come up with different numbers depending on what is fed into the equation. In addition, benefits, as many people noted, may be intangible or difficult to predict. Some outcomes may not be evident until 20 years in the future. He stated that we should not kid ourselves into thinking we can come up with some formula to plug in all the variables and produce something that shows that the risks outweigh the benefits or vice versa. It needs to be acknowledged that it will be difficult to quantify the equation and, in addition, if we were able to determine exact numbers, then different individuals would place different values on different variables. Some may believe that the advancement of knowledge is much more important than whether risky research is going to inform vaccine preparedness. He believes that one of the best things to come out of the risk assessment would be to convince ourselves and the public that we considered the issues in depth and that whatever decision we made was not pulled out of thin air, but rather the result of a careful deliberative process.

Fineberg also asked Fischhoff whether there are special issues that should be considered in a situation such as that posed by GoF research where there is a very small likelihood of a catastrophic possible outcome. Fischhoff replied that there are people who say that the public is incapable of understanding small risks or making difficult decisions, but that he does not believe that the evidence supports that. He said he would not give up on the public on the basis of a glib meme about public incompetence. People respond in ways seemingly contrary to the evidence because the evidence has not been presented in a credible way. The precautionary principle is brought into play when the people who are uncomfortable with technologies are analytically outgunned by the officials in charge of those technologies. Proposers of projects, such as new power plants, are often well financed and reluctant to modify their proposals, which are

often assembled without listening to other people or incorporating other concerns. This can produce an either/or situation, and the people who object are simply outgunned. The precautionary principle may be the only arrow in their quiver, but it may make objectors appear to be demanding zero risk and unwilling to accept any kind of trade-off.