Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
2 DESIGN OF THE PRIORITY-SETTING SYSTEM Developers of priority-setting systems must decide the following: the goal of the system; the chemicals to be considered; and the structure of the system, i.e., the information to be gathered in each step of the system, the type of analysis to be performed in each step, and the decision rules that determine which chemicals are to be tested and which are to be considered for other action (removed from consideration, placed on a holding list for future consideration, etc.~. pose tasks can be accomplished by experts, using their skills and common sense, but some issues are not easily addressed unless the designers of the system make special provisions for addressing them. For example, what is the most effective sequence in which to gather different kinds of information? How effective is the system? What is the impact of changing some component of the system? To address the latter two issues, it is helpful to use elements of decision theory and systems analysis. Such an approach to toxicity testing requires that the system priority-setting criteria and toxicity tests be described in measurable terms and that goals be defined to enable the system's effectiveness to be determined in relation to its goals. This description requires the system's designers to be explicit about the assumptions on which the design is based. Components of the system must also be described explicitly in terms of the number of toxic chemicals among the chemicals considered, the effectiveness of each procedure or toxicity test, and the resources required for each such procedure or test. The value-of-information concept is used to address the issue of which information is best to collect in each stage. This concept underlies a strategy in which decisions are based on the relationship between the importance of a piece of information and the degree of uncertainty about it (Raiffa, 1968~. Each additional piece of information is sought to enable decisions about the potential health hazard of a chemical to be made with less risk of error. The value of the information is defined as the difference between the expected costs of error in classifying the chemical with and without the additional information. The value-of-information concept is a simplification that may be difficult or even impossible to apply in complex decisions about testing priorities. To apply it rigorously, strong assumptions are needed to relate the multiple dimensions of chemical hazards to a single objective that is quantifiable. Although the committee recognizes that full implementation of a system based solely on these principles may in fact be impossible, it also holds that value of information is a useful and important concept. By designing an illustrative priority-setting system using the concept, the committee sought to analyze the factors that 207
are important for priority-setting and to explore how they interact, and to do so in a systematic and integrated manner. The discipline of a relatively formal analysis has produced insights that should be useful in a more flexible and operational priority-setting procedure. GOAL OF PRI ORI TY-SETTING When formulating a goal for use in designing a priority-setting system, one should consider that the goal of the system should reflect the mission of the user, provide guidance in designing the priority-setting system, and provide a mechanism to measure the performance of the system. The Committee on Priority Mechanisms considers that a goal to meet the need of NTP's priority-setting system could be the ability to assess accurately the potential public-health impacts of chemicals to which humans are exposed. As demonstrated below, that goal influences all elements of the priority-setting process, including selection of the chemicals to be considered for priority-setting, measurement of the effectiveness of the system, and the means for judging the information gathered and how it is analyzed. It also leads to viewing the priority-setting process and testing program as related parts of a larger effort to assess the potential public-health impact of exposure to toxic substances. The goal thus calls for the system to produce the accurate assessment of all chemicals, not just the proper selection of chemicals for testing. The system should, in general, dispose of chemicals of low concern, rather than selecting them for testing, and should not suggest additional testing of a chemical if adequate information is already available for an accurate assessment of its potential public-health impact. The above goal also implies that toxicity testing and the gathering of information on human exposure should be considered jointly in assessing the potential public-health impact of a chemical. Because concern about a substance depends not only on its intrinsic toxicity, but also on the extent of human exposure to it, the gathering of information on the numbers of people exposed to various concentrations of the chemical may be just as important in setting its priority for testing as is determining its potential for toxicity. CHEMICALS CONSIDERED Defining the universe of chemicals to be considered is an important design decision and is influenced by the goal chosen for the system. As indicated above, all chemicals to which there is potential human exposure should in principle be considered by the system. Although such a goal might cause the number of chemicals to be too large to manage or too difficult to estimate, several approaches can be used to solve the problem. One approach would attempt to define additional categories so that all chemicals to which there is potential human exposure would be 208
included in a category and then estimate the number of chemicals in each category; another would remove some categories of chemicals. A third approach could be to redefine the goal. The universe developed by the Committee on Sampling Strategies-- which consists of food chemicals, pesticide chemicals, cosmetic ingredients, drug ingredients, and industrial chemicals--totals over 50,000 discrete (although not always well-characterized) substances. This universe could be expanded by adding categories--such as combustion products, pollutants, and some categories of naturally occurring substances--until most chemicals with a substantial potential for human exposure were included. Adding these categories could expand the universe of chemicals considered by several thousand chemicals. If a smaller universe were desired, some categories of chemicals could be deferred or removed. In either case, the designers could make an explicit decision concerning the system. Testing priorities may be set for pure, well-defined compounds, commercial grades of such compounds, elements and all their compounds, categories of compounds (e.g., cyanides), mixtures of known or unknown composition, radicals, or other classes of chemical entities. The terms "substance" and "chemical" are used interchangeably in this report to include all these classes. When designing an exposure assessment, a toxicity assessment, and their interface, however, it is important to define as precisely as possible the identity of the substances being considered. The most commonly accepted (and usually unambiguous) identifier for a substance is its Chemical Abstracts System (CAS) Registry number. Establishing the universe of substances usually involves some initial screening or the exclusion of some candidate chemicals. Some of the schemes reviewed by the Committee on Priority Mechanisms have been applied only to specific classes of chemicals (such as food additives or drugs); others have been applied only to chemicals on existing priority lists or to chemicals nominated by panels of experts (see Appendix A). In the latter case, the chemicals have already been screened through a process that involves scientific judgment, so chemicals on which there is little information are very likely to have been excluded without adequate review. Thus, the establishment of the initial universe in itself constitutes a major step in the priority-setting process. In several of the schemes reviewed by the committee, the universe is immediately narrowed by the deletion of substances that are judged to be either irrelevant to the exercise or difficult to review. Classes of substances deleted in this way include the following: 0 Chemicals already regulated, such as pesticides, drugs, and food additives. · Substances not subject to regulation by a member agency of NTP, such as natural products. 209
· Chemicals only nominally subject to regulation and adequately tested under existing regulations. · Substances without CAS numbers, including complex and ill-defined mixtures. · Other substances difficult to characterize, including combustion products, pyrolysis products, and environmental breakdown products. · Environmental mixtures, such as extracts of air pollutants and water pollutants. Although the omission of such substances and mixtures can usually be understood on the grounds of convenience and practicality, it should be recognized that the classes of substances that are omitted include many that are both poorly characterized and potentially harmful. STRUCTURE OF THE SYSTEM The entire process of priority-setting and toxicity testing consists of a series of interconnected steps, each containing an information- gathering component and a decision-making component. An example of one such step, or stage, is diagramed in Figure 1. It begins with an information-gathering activity, which includes a search for and interpretation of specific data, or "data elements." Combined with information already on hand, the new information enables one to achieve a better understanding of the public-health concern of a chemical (middle box on right side). If the information is useful, the understanding will be more certain than it was in the initial state of knowledge (top box on right side). The figure represents concern as a position on a two-dimensional map of exposure and toxicity. In this example, the information-gathering activity is a toxicity test that narrows the uncertainty about toxicity without providing information on exposure. Other representations could include an estimated probability distribution for the concern (e.g., the probability that a given number of people would incur a specific effect during the next year) or a discrete probability distribution concerning the degree of hazard (e.g., the probability that the substance involves or does not involve a "significant" concern worthy of control activity). After the new state of knowledge is determined, a decision must be made (lowest box on right) to determine what additional pieces of information would be most valuable or that no additional information is required to make a reliable classification of public-health concern. The latter decision is usually made when it appears that economic, health, or other costs of misclassification are unlikely to be reduced enough to justify additional information-gathering activities. Thus, exit from a step is followed either by no further consideration of the substance or by a new information-gathering step. 210
Initial state of knowledge: great uncertainty about a substance's hazards (both exposure and toxicity). (ENTER) I nformation~athering component: generating or assembling and interpreting data element. (EX IT , - ._ c7 ._ x o l ~Exposure- ~ . r ! Later state of knowledge: less uncertainty about hazards because of new information (thus less uncertainty) about toxicity. ._ x Exposure- ~ ~1 Decision-making component: determining what information to gather or what action to take. FIGURE 1 Example of one stage or building block in process of investigation and control of toxic substances. Information-gathering activity here is toxicity test that produces useful data. 211
Any priority-setting and testing scheme can be represented, at least conceptually, by a network of such information-gathering and decision-making steps. Traditional methods depend on evaluation by an expert committee that reviews a dossier on a substance (information- gathering component) and then makes recommendations about its disposition (decision-making component). Systems for screening chemicals for priority-setting may have one stage or multiple stages. In one-stage systems, the same screening criteria and procedures are applied to all the chemicals under consideration. In the simplest type of multistage system, chemicals are screened out of the system at each successive stage; the only chemicals considered in the last stage are those which have survived all the earlier stages. A more complex type of multistage system is the decision tree, in which the screening criteria applied at each stage depend on the outcome of the previous stage. In most multistage systems, the first stage is a simple screen based on chemical class, use, or production volume; the second stage is based on criteria that reflect exposure; and the third and later stages are based on criteria that reflect toxicity or potential risks. Although this sequence is a feature in six of the different systems reviewed by the committee (see Appendix A), the reasons for its choice were not made explicit in them; it was adopted probably because crude indexes of use--production and exposure--are relatively easy to obtain for large numbers of chemicals, whereas indexes of toxicity are more difficult to obtain and require more scientific review and judgment. In more elaborate systems--e.g., the decision tree of Cramer et al. and the six-stage linear screen described by Nisbet--the late stages require fairly extensive compilations of toxicity and exposure data and fairly detailed scientific review. Multistage systems are advantageous, in that the screening process can use simple, readily retrieved data, so many chemicals of low priority can be eliminated from consideration quickly and scientific attention can be focused on the chemicals of greatest interest. Systems of this kind appear to offer a practical way to deal with very large numbers of chemicals. An offsetting disadvantage is that the criteria used in the early stages are necessarily crude, so some chemicals may be eliminated erroneously in an early stage. Another disadvantage is that exposure information is usually considered in less detail than toxicity information; hence, chemicals with unusual pathways of exposure may not be identified. These problems may be alleviated by providing for reconsideration of chemicals eliminated in early stages or reconsideration of exposure in later stages. These features are included in the Interagency Testing Committee's system. Designing a multistage screening system, under the constraint of an overall budget, involves balancing of the costs of generating information on a large number of chemicals in early stages against the 212
costs of generating more detailed information on fewer chemicals in late stages. The efficiency of such a system depends on the number of stages, the amount of information considered in each stage, and the number of chemicals correctly classified in each stage. In the systems reviewed by the committee (Appendix A), these characteristics appear to have been chosen rather subjectively, and it is not clear that maximal efficiency was achieved. Decision-tree systems are in principle more flexible than linear multistage systems, because they can use more appropriate criteria for screening at various stages. However, the systems that have been proposed to date require rather precise information and would be difficult to apply to several classes of chemicals, especially those with little or no toxicity testing. ANALYSIS OF INFORMATION Data on a chemical must be analyzed so that decisions can be made about its disposition. Most priority-setting systems assess the public- health concern about chemicals with respect to exposure, toxicity, and social considerations. The different degrees of concern can be thought of as ranges on a meter, with high readings generally warranting serious social concern and low ones suggesting that little or no concern is called for. The meter would register higher readings to the extent that there was an increase in any of the following: · Number of people exposed. Per capita exposure in that population. Frequency of exposure. Probability of a toxic response at that exposure. · Severity of a toxic response at that exposure. · Cost to society or to an individual related to the manifestation and treatment of health effects. because: smoked. On such a hazard scale, cigarette smoke would rate high, · Millions of people smoke cigarettes, and additional millions are exposed to others' smoke. · Exposures are high: grams of material are inhaled for every pack 213
· Frequencies of exposure are high--one or more packs per day for many smokers. · Cigarette smoke is known to increase significantly the probabilities of lung cancer, other cancers, heart disease, and other adverse health effects e · Lung cancer, heart disease, and some of the other effects are frequently fatal or incapacitating for long periods. · Costs of treatment and the impl early death are great. icit cost of pain, suffering, and Other chemicals would rate low, for various reasons. In between would be yet other chemical hazards. For example, one assessment estimated that vinyl chloride might cause about 10 deaths per year (Kuzmack and McGaughy, 1975), compared with 1.3 x 105 cancer deaths per year from cigarette-smoking (U.S. Department of Health and Human Services, 1981~. This example also demonstrates another feature of such assessments: their uncertainty. The figure cited for deaths associated with cigarette smoke is probably an accurate statement of the "true" value, to within a factor of 3; for vinyl chloride, the uncertainty is at least a factor of 10, and probably greater. Although there might be other reasons to test cigarette smoke further, an analysis based on the value of information would imply that the classification of high public- health concern was good enough, and that further testing would not markedly improve it. Another study of the carcinogenicity of vinyl chloride at very low exposures might or might not be warranted, but it would probably have a lower priority than a lifetime bioassay on a chemical that was similar to vinyl chloride in structure, uses, and exposures and that was mutagenic in vitro, but had never been tested in a rodent bioassay. These examples are based on the argument that hazard depends on both the human exposure to a chemical and toxic responses in humans. Exposure and responses interact in a complex way. For example, some carcinogens may be presumed to exhibit a nearly linear relationship between per capita exposure and the probability of causing cancer. In this case, the hazard would be similar, regardless of whether 100 people were each exposed to 1 g/yr or 100,000 people were each exposed to 1 mg/yr. But, if the chemical exhibited a threshold for effects or had a distinctly nonlinear dose-response relationship, the hazard would depend markedly on the distribution of doses; 1 g/yr might be fatal to all of 100 people exposed, but 1 mg/yr might have little effect on 100,000 people. Or the outcome could depend on the fractionation of the dose over the year, on the route of exposure (oral, respiratory, or dermal), on synergistic exposures, or on the age, sex, race, and health status of the people exposed. 214
An ideal priority-setting system would be able to capture the importance of all information regarding these subtleties of exposure and toxicity and to modify accordingly the probabilities with which a chemical should be assigned to the various degrees of hazard. However, such information is generally scarce and expensive to retrieve and interpret, and the system would become excessively complicated if it attempted to treat the information in such detail. Consequently, the committee assumes that hazard is defined by the combination of exposure and toxicity, each capable of being defined and estimated independently of the other. For example, if the exposure range and the toxicity range were each divided into three segments (high, medium, and low), there would be nine categories of hazard, each corresponding to a different pairing of exposure and toxicity. (Some might turn out to be equivalent to one another, e.g., high toxicity and low exposure, low toxicity and high exposure, and medium toxicity and medium exposure.) Data and tests are selected to help decrease the uncertainty in estimating the probabilities with which exposure or toxicity will truly fall into defined categories. For example, production volume is not a direct measure of human exposure (as are number of people, per capita exposure, and frequency of exposure); but, in the absence of more direct information on exposure, knowing the production volume does reasonably influence our perception of the probability that exposure is high. The results of an Ames Salmonella/microsome test do not prove or disprove toxicity in humans, but a positive result would increase, rather than decrease, our concern that the substance might be carcinogenic. The following subsections contain more detailed discussions of the concepts of exposure and toxicity. ESTIMATING EXPOSURE In an ideal exposure assessment, the investigator would attempt to answer each aspect of each of the following questions: o Who is exposed? - age - sex race ~ health status · How many are exposed? · How are they exposed? - occupationally - in the community - as consumers - environmentally 215
· By what route? - oral dermal inhalation single route of exposure multiple routes of exposure (e.g., orally and dermally, as workers and consumers) · In what pattern? - g/yr - g/incident mg/m3 or mg/L · How frequently? - continuously - regularly, periodically (e.g., 8 in/d, 4 pills/d, once every month) - irregularly, but repeatedly in single incidents · Trough what chain of events? - production or extraction manufacturing process transportation storage use environmental transformation bioaccumulation industrial discharge waste disposal environmental transport If all such information could be acquired, one could imagine dividing each dimension of exposure (route, amount, frequency, etc.) into ranges and then classifying all exposed people into groups defined by every combination of these ranges. Clearly, however, it would not be technically feasible to collect such details on most chemicals. Moreover, it would not be economically possible for a priority-setting process to acquire and use this much information, nor would it be easily understood. The committee therefore proposes to use available exposure-related information (in a given stage of the priority-setting process) merely to help estimate the probability that human exposure to 216
the chemical will be relatively high, intermediate, or low. Thus, if all other factors were equal, exposure would be considered higher than average if the data element suggested that: 0 Large numbers of people are or may be exposed. · Concentrations in air, water, soil, food, or other environmental sources are higher than average. · Exposure is more frequent or lasts longer than averages ~ Exposure occurs by a route suspected of eliciting the most toxic response. ~ Exposure occurs among groups that are especially susceptible because of age, race, sex, health status, or other conditions. when a surrogate for exposure data is used, the objective is to choose a variable that reflects exposure as accurately as possible, in view of the extent of the toxicity information available at that stage of the priority-setting process. If, for example, reliable information shows an abrupt rise in the dose-response curve (e.g., from essentially zero effect to 100% occurrence of a serious effect), then the definitions of high, low, and medium exposure would clearly correspond to the per capita doses above, below, and near the point where the response curve rises rapidly. At the other extreme, when there is no toxicity information whatsoever, it is appropriate to assume a linear dose-response relationship of unknown slope, in which the extent of hazard varies continuously and proportionally with the extent of exposure. Here, the definitions of high, medium, and low exposure are largely arbitrary. Of the entire universe of chemicals, probably only a few substances are usually considered to be in the high exposure region, more are in the medium region, and most are in the low region. This distribution conforms with common perceptions about the number of toxic chemicals to which humans are exposed at high concentrations. That is, if toxicity is rarely produced at a dose less than 1 mg/kg of body weight for single acute doses, then exposure would be considered high for a 50-kg adult only if typical doses were higher than approximately 50 mg in a single exposure incident. In the absence of a quantitative model of the entire exposure-toxicity-hazard phenomenon, it will ordinarily be necessary to work with information that is much more ambiguous than the above guidelines suggest. For example, recourse only to some crude production and use information is unlikely to produce a convincing prediction of the number of people who will breathe a substance in concentrations greater than 30 mg/m3, even if it is assumed that all other properties 217
of the substance are "typical." In such cases, therefore, classification of exposure could not be based on the concentration at which humans are actually exposed, but must instead be expressed in terms of the pieces of information {data elements) that are available. To continue the example, exposure is likely to be considered high if production is greater than 106 kg/yr and the substance is used predominantly in consumer products with high potential for exposure, e.g., foods, drugs, and cosmetics. This logic is not extended to estimate that exposure is greater than 106 kg/225 x 106 people) per year, i.e., 4 g/yr per person on the average; that figure might not be at all relevant to the assessment of toxicity. ESTIMATING TOXIC EFFECTS The process of predicting or estimating the biologic activity and toxic potential of a chemical is extremely complex. It requires the consideration of several types of information, coordination of a number of scientific disciplines, judgment and intuition, and an appropriate schedule of testing, if warranted. There are no firm rules for toxicologic prediction, nor are there guidelines that will ensure reliable Answers. Attention is generally focused on human health effects for which there are attributable chemical causes. Surrogates (i.e., laboratory animals) are commonly used in testing for human health effects. In the best circumstances, toxicologists may be able to predict a particular activity of a chemical, ascribe a potential effect, identify a useful end point in an established animal model, elucidate the mechanism of the effect, and arrive at the conclusion that the chemical either will have no toxic effect in humans or will cause a measurable effect. The toxic response to a chemical is multidimensional because of the wide variety of possible effects on human health, many of which are dose-dependent. In some cases, one human health effect may be clearly more serious (and thus have a higher priority for testing resources) than is another. For example, many more testing resources are devoted to cancer bioassay than to skin irritation tests, and severity is a major reason for the difference. In contrast, the relative severity of other pairs of effects may be difficult to evaluate and may differ according to individual perceptions. Nevertheless, choices between tests for different effects bring with them value judgments about relative severity, and a systematic priority-setting procedure will reflect some set of judgments. For the illustrative system in Chapters 3 and 4, the committee evaluated only one health effect--cancer--and avoided comparing the severity of effects. A complete and general system would need to consider explicitly the values that society assigns to different effects. Chemically induced health effects in humans can be cataloged and 218
organized in a number of ways. One of the most convenient is to classify them by target organ. The range of such effects and their corresponding organs or tissues can be seen in the list of examples presented in Table 1. Toxicity data derived from human exposure are necessarily limited-- and are nonexistent for most substances. Therefore, it is customary to depend on data from toxicity tests in animals, from which extrapolations are made to predict health effects in humans. The most useful data are those derived from an accepted animal model for a given chemically induced condition in humans, but adverse effects in animals not related to a known disease in humans are also of value. Results of toxicity tests in animals have been recorded in machine-readable files for approximately 15,000 chemicals, whereas chemicals to which there is potential human exposure number several tens of thousands. Therefore, information on many chemicals is limited merely to molecular structure and physical constants. some idea of the relative importance of toxic effects can be gained from examining their consequences to the injured person and to societY. For example, is an impairment structural or functional? Is it reversible or irreversible? Is it progressive? These questions are implicitly considered in most priority-setting schemes, including those that depend exclusively on human judgment. They imply value judgments about relative severity and involve not only judgments about biologic consequences, but also individual perceptions of harm. To examine consequences for the individual, a number of factors should be considered: · An impairment may be caused by structural or functional damage or both. Irritation and corrosion may be inconvenient and painful, but are primarily structural. Lead-induced behavioral changes may be regarded as functional. Hexane-induced azonal neuropathy is both structural and functional. · An impairment will generally be considered to be of greater concern if it is irreversible than if it is reversible. Many biologic effects are expressed only in the presence of the causative chemical and cease with or soon after its disappearance. Others, such as death, are obviously irreversible. Irritation, depression, and methemoglobinemia are examples of reversible effects; sensitization, retinal damage, pulmonary fibrosis, and carcinogenesis may be regarded as irreversible effects. ~ An impairment may result immediately after acute exposure, may be delayed until some time after acute exposure, or may require repeated and prolonged exposure to become manifest. Corrosion is the most obvious immediately apparent impairment. Delayed hypersensitivity frequently results from an initially innocuous exposure. Alcohol-related cirrhosis of the liver is an example of an impairment manifested only after repeated and prolonged insult. 219
u) o · - Al ~ In · - 0 a, ~ ~A a x ~o ~ ~o O O ~r1 ~· - ~· - ' ~ ~ ~ · - ~as o v ~ ~ u] In · - al · - ~ . - . - . - · - ~ us ~ ~ ~ ~ 3 ~O ~V .~1 ~, - ~ ~o 0 ~. - O ~ ~ ~ X ~ O ~ ~ ~"-it/ ~· - · - ~ O ~ ~1 ~ ~ V O · - ~ ~ be ~ U] ~ ~ ~ ~O ~ ~ · - O en 1 A ~ ~ 0 ~ ~ ~v .o O O ~ O -I O S O O .- al ~ ~ ~ ~ ~ O Q. I: ~ ~ a: ~ ~ ~ ~ ~S X O O ~ ~ V ~ ~ ~ Q ~ ~ ~ ~ ~O ~ O u, 0 ~ s a) ~ ~ ~ ~ ~ ~ a) ~ 0 ~ ~n ~ a) O O ~ (~) Z ~ O Z ~ 3 ~ ~ H ~ ~ ~ i ~H $~ ~ Z U .- u' ~a, tR ~n ~ ·,' ~V O E ~u, ~Q. U] O O ~n ~ · - U] i4 Y O ~n v O :^ L' a, 0 ~ a v a) ~O eq ~O ~: 3 ~m :^ ~ :' :~: 0 .- . - ~n tQ u, a v ~ ~ 0 ,1 o.~. J: ~ Q V ~ ~ ~= O ~ O H 54 ~. - ~ 3 ~ ~X ^ ~- V S ~ O ~ O U ~O ~ ~U _t Q. ·-l ~ ~·- O ~ O ·- ~ ~ ~ ~ G1 0 GS ~O O ~ ~O ~O ~ ~ ~ ~ ~ ~ ~O ~.~1 N O . - O .~1 0 . - N ~ 0 0 - - ~ ~ ~ ~ O ~· - ·,1 0 'T~ J~ '-~ ·- ~ ~ ~ ~ ·- ~ U ·- ~ ~ ·- O ~ ~ ~ ~ ·- 0 ~ ~ ~ ~ 0 ~ ~ ~ 0 0 ~ ~ X · - S ~ ~ O a) u ~ ~ ~ 0 ~ ~ ~ 0 ~ o ~ ·- ~ 0 ~ 0 ~ ~ ~ a~ ~0 t) a,~ ·,~ u~ ~ ·-l i4 ·~1 ~ CQ ~) :~ )~ Q~ ~ S ~ ~ I O C) ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ Q O ~ ~S ~ ~ O V -1 ~a) 0 ~0 ~0 ~0 ~0 ~ s 1 ~U) t) ~O i] O 1~1 0 H U] 04 P ~4 Z ~a] P {R ~ ~0 ~n o ~ q) ~Q E~ ~a, ~n O a: ~n ,. 0 Q. o .,, ~q .~ tn ~q ~ :, O ~O · - a, v Y ~:, U] ~Z 220
· Some impairments may be regarded as threats to life. Although many structural and functional impairments may be undesirable, painful, and debilitating, they are nevertheless clearly less objectionable than life- threatening impairments. These factors exemplify the criteria that may be used for assessing relative importance of health effects, such as are presented in Table 1. They are obviously incomplete in a number of respects e For example, no account is taken of acute lethality, individual susceptibility, dose-response relationships, and age and sex relationships. ASSESSMENT AND DECISION-MAKING Pr for ity-setting sys tems include some procedure to reach a j udgment about chemicals with the information collected. On the basis of this judgment, chemicals are considered further, recommended for testing, or removed f rom cons iteration. Several assessment procedures may be used: scoring, modeling, sorting, ordinal ranking, and expert judgment. The choice of procedures depends on the number of chemicals considered, information available, cost, and type of personnel required. In addition, designers could consider the likelihood that the procedure might introduce bias, the ability to function even though some data are missing, and the ability to respond to uncertain or inaccurate data. · In scoring systems, the data used as ranking criteria are assigned numerical scores (usually integers), and the scores are combined by a rule (often a weighted addition) to yield a single score that represents relative toxicity, relative exposure, or relative overall hazard. Scoring systems have the advantages of being easy to use and providing consistent treatment of all chemicals considered. However, it is difficult to combine scores in a valid way. Missing data often require substitution of default values. Uncertainty in the data is not reflected in the scores. o Modeling-based systems use the data elements directly (kilograms of chemical produced, LDso in milligrams per kilogram, etc.) and combine them into an index that represents the degree of human exposure, the degree of toxicity, or the overall health hazard. Modeling-based systems require more analysis than other procedures and require moderately skilled personnel. Uncertainty in the data can be dealt with more easily in models than in other procedures. · Sorting (or screening) procedures answer questions regarding aspects of exposure and toxicity and sort chemicals into categories in accordance with the answers. The categories and sorted chemicals in each category are then ranked according to judgments as to which ones represent greater or more important hazards. Sorting procedures are easy to use and may be applied to large numbers of chemicals. 221
· In ordinal ranking, chemicals are ranked on each of various elements of exposure and toxicity, and the ranks are combined, according to a rule, to derive an overall ranking. Ranking procedures are easy to use and may be computerized. When rankings are combined, it is difficult to understand the meaning of the combined ranking. Missing data require substitution of default values. Uncertainty in the data is not indicated in the ranking. · Expert judgment requires one or more highly skilled persons to review data, make an assessment, and recommend further action, such as testing. Methods of improving the elicitation of expert judgment are described in Appendix C. Although the use of experts is probably the most accurate assessment method, it is also the most expensive and requires highly skilled personnel. DESIGN FACTORS DESCRIBING THE PRIORITY-SETTING SYSTEM While choosing the elements of the priority-setting system, the designer should consider that the system must respond to the following external parameters: the nature of the chosen universe of chemicals, accuracy of selection stages or other components and accuracy of the toxicity testis) for which the system is selecting chemicals, and costs of the system components and toxicity tests. Each of these factors may be described in as much detail as desired by the designers of the system or as needed for the analytic technique being applied. The minimal description is a word description. Use of a mathematical model designed to optimize system performance requires a quantitative description. Even a minimal description may be useful, because it requires system designers to address issues explicitly. For example, if one attempts to estimate the proportion of chemicals that cause a specific type of toxicity, several elements must be defined (e.g., what types of toxicity and which health effects are specified?. And one must specify the population of chemicals to which the estimate applies. ACCURACY OF STAGES OR TESTS Priority-setting systems may be regarded as classification procedures. Each stage may assess the degree of public-health concern separated into categories of toxicity and exposure for different types of health effects. Degree of public-health concern may be based on any number of categories of exposure or toxicity. The main limitation in defining categories of exposure and toxicity is the amount of information that is available. It does little good to define a large number of narrow categories if almost no information is available for assigning chemicals to them. Because the information is far from perfect, there will be errors. If there were only two categories of toxicity (or exposure)--such as toxic 222
and nontoxic--there would be four possible classifications: true-positive, false-positive, true-negative, and false-negative. If the exposure and toxicity categories were divided into high, medium, and low toxicity and high, medium, and low exposure, there would be nine possible combinations, each with some degree of error (see Table 2~. Figure 2 illustrates misclassification in a complex situation. Public-health concern is a function of exposure and toxicity. Categories of concern are labeled S1 to S6, in order of increasing severity. For example, S1 might refer to a situation of low exposure and low toxicity, and S6 to one of high exposure and high toxicity. Each cell in the diagram stands for a classification. For example, if a chemical is truly in category S2, but is classified in category S5, it would be in cell C5,2. Chemicals that are correctly classified lie along the diagonal--Cl,!, C2,2, etc. These diagonal cells represent zero misclassification. As one moves from the diagonal to either the upper right or lower left corner, the extent of misclassification becomes greater. The cell in the top right corner (C1,6) represents the most serious false-negative classification--a chemical with high public-health concern (high exposure and high toxicity) classified as one with low public-health concern (low exposure and low toxicity). The bottom left corner (C6,1) represents the most serious false-positive classification. MEASURING PERFORMANCE In a perfect priority-setting and testing program, all chemicals would be classified correctly, so they would appear in cells along the diagonal (Cl,l to C6,6) in Figure 2. Even with unlimited funds, this would not be possible in practice, because even the best test batteries may sometimes yield false-positive and false-negative results. For any priority-setting system, there will be a spread of chemicals away from the diagonal. A testing and priority-setting program should make this spread as narrow as possible; more precisely, it should minimize the misclassifications for a given investment of resources. In designing a priority-setting process, value judgments are unavoidable. Two such judgments are especially important. First, it is necessary to make some judgment about the relative importance of a false-negative and a false-positive for a given effect. This judgment is built into the design by the relative weights attached to the upper right and lower left corners of Figure 2. An illustrative method of deriving the relative weights is given in Appendix E. Second, it is necessary to make some judgment about the relative importance of health effects. This is built into the design by relating the cost of a false-positive for one effect to the cost of a false-positive for another effect (or comparing the relative costs of false-negatives). In principle, one type of true classification could be treated as more important than another type of true classification. In general, however, available information appears insufficient to make such a judgment (see Appendix B). 223
TABLE 2 Possible Results of Test Having Three Results: High, Medium, and Low Result of Test True Toxicity Low Medium High Low Correct False-negative Greatest false-negative Medium False-positive Correct Greater false-negative High Greatest Greater Correct false-positive false-positive 224
TRUE CATEGORIES S1 s2 S3 S4 S5 s6 1 1 1 1 1 1 ~ S1 in a: S. o 2 CD of: C: C) UJ in ¢~ in cl,1 C1 6 C2,2 S3 C3,3 S4 C4,4 S5 c5~2 C5,5 1 s6 c6~1 C6,6 FIGURE 2 Illustration of misclassif ication. concern (S) is based on exposure and toxicity and ranges from S1 (lowest) to S6 (highest). C and its subscripts indicate cell into which chemical is categorized and false and true classifications. Cells C1, 6, C5,2, and C6,1 are misclassifications. Cells on diagonal (Cl,l, C2,2, etc.) are correct classifications. 225
Once the extent of misclassification is at least conceptually defined, it is possible to compare alternative designs of priority-setting processes and to select the one with the lowest cost of misclassification within a given budget for priority-setting and testing. COST The costs of components of priority-setting systems and toxicity tests (in dollars per substance) are used in designing selection processes to ensure the selection of the most productive information-gathering activities per dollar spent. Cost data for toxicity tests are described in Stage 4 in Chapter 4. 226
