The Elements of Scientific Advice
Henry Vaux, Jr.
University of California, Berkeley
INTRODUCTION
With the passage of time, the substantive and contextual bases in which public policy must be made grow ever more complex. This is especially true of natural resources and natural resource policy. The explanation lies with the fact that as populations and economies have grown, the competitive pressures on natural resources have also grown. This has led, in turn, to levels of exploitation which either cannot be sustained or can be sustained only by using management systems which are based upon a clear and unequivocal understanding of how the underlying natural resource systems behave. The fact that there are limits to the resiliency of natural resource systems means that the management of such systems without adequate scientific underpinnings is inherently a high stakes gamble in which the entire system may be lost or its biological and economic productivity severely impaired (Houck, 2003).
The success of modern management systems for natural resources is almost always determined by the adequacy of the scientific understanding of those systems except in instances in which the policy maker simply rejects the underlying science in the interests of securing other objectives. To be effective, management strategies must be based upon and incorporate accurate information about the state of the system being managed and about the way that system will change over time both in the presence and absence of managerial manipulation. Several different types of scientific information are required. The first is a description of the current state of the system. The second would include a description of how the system varies over time and, more specifically, a characterization of the relationships between the descriptive parameters and all of the variables that cause those descriptive parameters to change. A third category of information derives from the second and includes information on how the system will respond and react to all manner of managerial interventions.
Good managers of natural resource systems are good appliers of science and scientific information. Thus, the successful manager of water systems has access to pertinent scientific information which characterizes the system to be managed and applies that information in making policy and managerial decisions related to how the water is managed. A water manager cannot usually be either a good manager or an effective manager if he or she does not have access to the pertinent scientific information. One of the obligations of scientists and the scientific community is to provide the needed scientific information to natural resource managers. This can be done either directly or through the development of scientific methodology which resource managers can then employ in developing their own scientific information. The latter strategy may be particularly attractive since it allows managers to tailor information development activities to the specifics of the particular system which they manage and the acquisition of the particular managerial information which they need.
As the purveyors of critical information which is needed to manage the world’s resources sustainably to meet multiple objectives, scientists need to be clear on what constitutes adequate
scientific advice. What are the elements of such advice and information? What principles should guide scientists in developing and rendering scientific advice? The remainder of this paper is devoted to a characterization and discussion of the elements of good scientific advice, particularly as it is applied to the management of water systems.
THE ELEMENTS OF GOOD SCIENTIFIC ADVICE: FIRST PRINCIPLES
Principle # 1: Frequently, scientists compromise their effectiveness and credibility by failing to distinguish among scientific information, scientific interpretation and policy value judgments. There is an understandable resentment among policy makers about scientists who behave as if their scientific backgrounds make them especially qualified to make policy value judgments. Policy value judgments are inherently non-scientific and scientists are no more qualified to make them than anyone else. In addition, scientists frequently compromise their effectiveness by failing to be clear about what is scientific fact and what is an interpretation of that fact. The first fundamental principle that should govern the development of good scientific advice can be stated as follows: It is crucial to distinguish between fact and what follows logically from fact, on the one hand, and interpretation of fact and value judgment on the other. This is not to say that scientists should be restrained from rendering interpretations of scientific fact and value judgments about the formulation policy. Rather it is to emphasize that scientists must make clear when their advice contains elements of interpretation or policy value judgments.
Principle # 2: There are at least three distinct dimensions of scientific advice which can be offered either independently or in conjunction with each other. These are: 1) existing scientific knowledge; 2) interpretations of existing scientific knowledge; and 3) methods for acquiring scientific knowledge. Existing scientific knowledge is comprised of information that is known with certainty, information that is known probabilistically and information that is uncertain or unknown1. It is rare that scientific information is known with complete certainty and there are circumstances in which information is unknowable with scientific certainty. In rendering scientific advice, it is thus important to inform the decision maker of the relative degree of risk1 or uncertainty associated with specific pieces of knowledge so that risk and uncertainty can be accounted for in designing policies and management strategies. Similarly, interpretations of scientific information rest in part on what is known with certainty; what is characterized by risk and what is inherently unknowable and uncertain. Again, in making scientific interpretations, it is important that the scientists be very clear in describing the extent to which a given interpretation is based on hard knowledge and the extent to which it is based on probabilistic knowledge and/or judgments even where they are employed to reduce uncertainty.
There are other circumstances where scientific advice will not consist of scientific information at all but rather in the characterization and design of processes or methods for acquiring scientific information either on a one-time or on a continuing basis. Here again, reliability and accuracy are important characteristics of any system that generates scientific information. The task of the scientist in these situations is to provide knowledge not just about methods and processes for acquiring scientific information and their design but also to characterize ex ante the reliability of
the system and the accuracy of the data which the system produces. This latter characterization is particularly important since frequently there are circumstance where the existing scientific state-of-the-art does not allow for the gathering of data and knowledge with complete certainty. A fundamental element for virtually all good scientific advice is that it characterizes accurately the extent to which the scientific knowledge in question is known with certainty; is known only probabilistically or is completely unknown (National Research Council, 1993).
These two principles are the fundamental principles that govern whether scientific advice is good or not. Advice based on the solidest and most comprehensive bodies of scientific knowledge will not be good if advisors do not clearly distinguish between facts and opinions. Similarly, good scientific advice must characterize scientific knowledge explicitly in terms of what is known with certainty; what is known with some attendant risk and what is not known or uncertain. Scientific advice will be severely compromised and even misleading where these two principles are not followed.
THE ELEMENTS OF GOOD SCIENTIFIC INFORMATION FOR WATER MANAGEMENT
In an ideal world, a water manager would wish to have comprehensive information. It is important to recognize that information is costly and the benefits of additional information will not be uniform. In the normal course of events, the marginal benefits of additional information will decline. Where this is so, the economically optimal amount of information will not normally be a completely robust and comprehensive set of information. In such instances, optimizing the amount of information will be economically efficient.
Optimizing Scientific Information: The totality of all of the scientific information that would be useful in managing water is substantial. While it would be helpful to have access to all such information, it is important to recognize that information is not costless. Consider, for example, the problem of protecting ground water from a possible leak in a toxic waste storage pit equipped with a clay liner. As shown in Table 1, the costs of a monitoring network rise exponentially as the probability of detection rises. The analysis based on this example shows also that the costs of a monitoring network and the probability of detecting a spill may depend critically on the shape of the spill or plume profile. The costs associated with uncertainty can be illustrated further by emphasizing that an optimally designed monitoring grid that will detect a radial spill with a probability of 0.9 will detect with a probability of only 0.23 if the spill turns out to be elliptical. These calculations were made assuming that the soil profile was homogeneous. Most substrates through which water and contaminates migrate are not homogeneous and this injects further uncertainty and raises still further the costs of acquiring adequate information (Vaux and Jury, 1985).
TABLE 1 Number of sensors required for different probabilities of detection with different spill profiles.
Probability Radial spill |
# of sensors Elliptical spill |
# of sensors |
0.1 |
3 |
27 |
0.5 |
21 |
180 |
0.9 |
70 |
597 |
As a general rule, it will not be economically efficient to develop and gather a total or complete set of scientific information. In economics jargon, the marginal costs of gathering or developing the last bits of scientific information will outweigh the marginal benefits. (One exception will be instances in which toxic wastes threaten an aquifer and there is no alternative source of water supply.) In offering scientific advice it is important for scientists and managers alike to be clear on the fact that there is an optimal amount of scientific information (where the net benefits of the information are maximized) which will be less than a comprehensive set of information in most instances. Thus, one of the problems of formulating good scientific advice is to determine which pieces of scientific information are really important and beneficial and which are less important and less beneficial. Emphasis should always be placed to developing and communicating the most important and beneficial information first.
Dynamic versus Static Information: Water managers focus on both water quality and water quantity. Competent water management requires knowledge of how past and current actions will affect the future qualitative and quantitative conditions of the water system in question. The manager needs to be in a position to anticipate and react to future circumstances. This means that good scientific information on water will almost always be dynamic or time dependent. Such information is generated invariably with the aid of models. Models of varying types, include different sets of parameters, are accurate over specific ranges of conditions and vary in the degree of robustness with respect to different circumstances and parameter values.
Good scientific advice surrounding the adequacy of different water models will always include information on the appropriateness of the model for the circumstances in question; the estimates of the degree of accuracy, usually stated in terms of error bars; and a characterization of both the strengths and weaknesses of the model. In circumstances in which it is necessary to build new models data requirements may be extensive and the data expensive to acquire. Nevertheless, it is important to reiterate that the quality and accuracy of the model need to be made transparently clear.
Uncertainty and Adaptive Management: In many circumstances good scientific information upon which to base water management policies and schemes is simply not available. Yet there may be considerable urgency and need for management in order to protect the resource and to generate additional supplies of water in circumstances of scarcity. The prescription for such situations is adaptive management which entails flexible management regimes that can be altered
and adapted as more experience with the system yields more information (Walters and Holling, 1992). The importance of adaptive management cannot be overemphasized. Data on water are lacking in many regions of the world even as the need for water management intensifies. Moreover, projected levels of population growth suggest that water management will need to become more pervasive if sufficient quantities are to be available to meet the drinking water and sanitation demands and grow the additional food needed to support more people.
The challenge here is to design a management regime which serves to protect the water and generate sustainable levels of exploitation in ways that also aid in determining experimentally the properties of the water system and its response to the manipulation of different management variables. It will rarely be true that the ideal experimental regime will be the same as a regime designed to accomplish the management objectives for the aquifer in question–even under uncertainty. The trick, then, is to design a management regime which balances the need for immediate management intervention and the need for scientific information. In most such circumstances, the scientist and the ground water manager will be required to exercise judgment jointly to design such a system. Here, good scientific advice will consist of knowing how to design an optimal experiment as well as knowing how to depart from that optimal experiment in ways that will allow management objectives to be achieved while at the same time ensuring that useful scientific data will be generated. The need to design water management regimes which are adaptive and yield useful scientific information relatively quickly represents a new and important area of endeavor for the water science community (Walters, 1986).
CONCLUSIONS
Successful water managers must be masters of many trades. The must be skilled policy analysts and imaginative devisers of policy. They need excellent communication and political skills. And, they must be good applied scientists. As water demands grow in response to population and economic growth throughout the world, water will need to be managed more intensively if new increments of demand are to be served. If water managers are to succeed in their ever more complex and ever more demanding endeavors they will need the best possible scientific knowledge and information.
In developing and communicating this scientific information research scientists must be mindful of two fundamental principles that govern good scientific advice irrespective of the kind of science involved. First, it is essential for scientists to be clear always about the distinction between scientific fact and value judgments. Scientific information consists of scientific fact and what logically follows from that fact. Interpretations of fact and value judgments should not be confused with scientific fact and scientists should be clear in labeling interpretation and value judgments for what they are. Second, there is hardly any certainty about anything. There is always a need for more scientific information and some phenomena are not completely knowable given the limitations of the scientific state of the art. In providing scientific information, scientists need to be clear to distinguish between what is known with certainty, what is known probabilistically and what is completely uncertain. The water manager deserves no less than to be advised when he or she is proceeding in realms where the science is inadequate or unavailable.
The particular elements of good scientific advice for the specific case of water management are three in number. First, it is important to recognize that scientific information is always costly. Rarely will it be economically justifiable to insist on complete information. Good scientific advice will focus on the most significant information and de-emphasize information which is less important or would be merely nice to have. Second, ground water is a dynamic resource whose condition changes with time depending upon environmental and managerial variables. Good scientific advice should be couched, where possible, in a dynamic framework. Third, and finally, too often there is little or no scientific information available. Here adaptive management in which the manager learns by doing will require solid scientific input and a careful balancing between the experimental needs and the objectives of the management regime.
References
Houck, O. 2003. Tales From A Troubled Marriage: Science and Law in Environmental Policy. Science. Vol. 302. Pp. 1926-1929.
National Research Council. 1993. Ground Water Vulnerability Assessment: Predicting Relative Contamination Potential Under Conditions of Uncertainty. (Washington, D.C.: National Academy Press).
Vaux, H. J., Jr. and W A. Jury. 1985. Some Economic Problems of Ground Water Contamination from Hazardous Waste Disposal Sites. Proceedings of the Fifteenth Biennial Conference on Ground Water. University of California Water Resources Center. Riverside, California pp. 103-110.
Walters, C. J. 1986. Adaptive Management of Natural Resources. (New York, N.Y.: McGraw Hill.)
Walters, C. J. and C. S. Holling. 1990. Large-scale Management Experiments and Learning By Doing. Ecology. Vol. 71. Pp. 2060-2068.