Major Conclusions and Recommendations
Because the charge to the committee focused on an examination of the gray literature to determine its possible contribution to our knowledge about links between environmental exposures and chronic diseases, we summarize here in some detail our conclusions regarding the gray literature in the United States. The remainder of this chapter focuses on the other main element of the charge—improvement of methods for performing epidemiologic studies—and presents major conclusions and recommendations on methodologic and related issues.
The review of the gray literature, in which the committee examined studies that states selected for our review, probably caused us to look at the best of this genre of epidemiologic research. Nonetheless, major limitations in the utility of the data exist. A major limitation of much of the gray literature is that many of the studies reported are initiated in response to a political problem but with inadequate funding and unavoidable limitations that seriously undermine the credibility and value of the final product. Although these political pressures can never completely be avoided (and perhaps should not be), the committee believes that it would be useful to establish guidelines to determine the circumstances under which epidemiologic studies should be undertaken. In some cases, there may be appropriate responses to public concerns other than initiating an epidemiologic study, for example, using existing evidence to address the concern. Fewer, but better, studies could advance the public welfare. External scientific advisory committees can also help to promote better scientific approaches for these studies.
Many of the studies in the gray literature could have been published
in the peer-reviewed literature with additional effort. Researchers working in state agencies need both incentives and support to publish study findings. Many studies reviewed by the committee would be of great interest to researchers in other areas. The committee encourages activities, such as the Agency for Toxic Substances and Disease Registry (ATSDR) State Environmental Health Information Clearinghouse, to make these gray-literature reports available to others.
Public-health agencies need a combination of speed and low cost in their research efforts. Thus they use readily available data for populations and geographic regions, and many gray-literature studies used an ecologic design. The disadvantages of this approach are not fully and widely understood; in the literature reviewed for this report, many studies had significant shortcomings. More research on methods, improvements in the methods already available, and better training are needed for scientists using this approach. It would be beneficial to develop guidelines for typical health-department studies to assist practitioners. These might be versions of existing texts used by academic epidemiologists.
Another common limitation in these studies was their inappropriate use of exposure and health-outcome databases that do not provide adequate information for this type of health study. While some of the databases need to be and could be improved, researchers need greater sophistication about their limitations.
Yet another problem is the lack of nonexposed or less-exposed persons as controls for many studies. Often, regional rates were compared with national rates, which can give misleading results because the national rates do not take into account confounders (e.g., race or socioeconomic status) in the study population. Appropriate comparison groups must be used to correct for possible confounders.
The weakest aspects of most of these studies was their use of imprecise measures of exposure or small study populations, so type II errors may often occur (failing to reject the null hypothesis when it should be rejected). There were often unavoidable limitations in the size of the exposed population. Creative approaches (e.g., meta-analysis) may help to overcome this problem. Also, it may be possible to group related health outcomes in ways that increase statistical power. Many adverse reproductive outcomes are uncommon, so that even a large relative risk may be difficult to detect.
Conclusions and Recommendations Concerning the Gray Literature
We conclude that most studies and reports in the gray literature have serious limitations, such as lack of adequate exposure information, that
seriously undermine their credibility and value. However, the gray literature may contain studies and reports that point to directions for further research or that contain the only information on a topic. Used with caution, the gray literature may make a contribution to the study of effects of toxic substances. We recommend that the studies and reports be collected, electronically listed, and made available in a central repository at ATSDR so that investigators can have access to them. The database should include gray literature studies, whether positive or negative effects have been reported, and a list of studies in progress and completed.
Because some of the gray literature in the United States provides useful information to communities, public-health officials, and researchers about specific exposures in local areas and about general health problems that may be associated with exposures from hazardous-waste sites, these studies need to be made more available to those interested in this field.
The committee recommends that ATSDR, perhaps with the National Library of Medicine or other agency, explore the feasibility of establishing an on-line database of completed state health-department studies in environmental epidemiology.
Failure to publish nonsignificant results poses a major obstacle to meaningful meta-analysis. A possible method for minimizing this bias is the prospective registration of research. Approaches to minimizing publication bias, including registration of studies, should be actively investigated and developed.
The committee recommends that ATSDR examine the merits of developing an electronic directory of on-going environmental-epidemiology studies that includes sufficient statistical results, if available, with the publication status of the study.
Although the limitations of studies in the gray literature may not be completely avoided, the committee believes that it would be useful to establish guidelines to improve the credibility and utility of the studies. We suggest the following as a starting point.
External scientific committees, advisory to state health departments or other sponsoring agencies, should help to promote better scientific approaches for studies.
Some of the gray-literature reports that the committee reviewed could have been published in the peer-reviewed literature, and hence made more readily available elsewhere, if the authors had invested the necessary effort.
Researchers working in state agencies should be given incentives to publish studies and to invest the time and effort needed to make these reports suitable for publication.
Many of the gray-literature studies used an ecologic design. For public-health agencies, there is an advantage to using readily available data quickly and inexpensively. The limitations of this approach may not be widely understood; many of the studies we reviewed had significant
shortcomings in the definition of the question to be addressed, study design, and its implementation.
More research and method development on the use of ecologic data are needed, as well as training for scientists using this approach.
Another common limitation in these studies was the use of exposure and health-outcome databases that do not provide adequate information for this type of health study.
Databases need to be improved, and researchers need to develop a greater degree of sophistication about their limitations.
Conclusions and Recommendations on Methodologic Issues
Epidemiologic research is often expensive and time-consuming, especially where longitudinal studies of large populations are involved.
Efforts to update previous prospective cohort studies should be encouraged, as these can provide cost-effective means to evaluate chronic diseases that could be linked with exposures that were previously characterized. Similarly, new analyses of old data should be generally supported, as they may yield important clues about the etiology of chronic diseases.
Common to all epidemiologic designs is the need to consider at the outset the chance of producing a positive finding. Statistical power, or the probability of finding a real effect, should guide any decision about whether to undertake the research. The higher the expected relative risk (RR), the smaller the population that needs to be surveyed. Conversely, the larger the population studied, the smaller the RR that can be detected.
In general, larger samples are needed when exposure measures are not continuous, when the effects of confounders and errors of measurement cannot be taken into account, and when the adverse outcome is a rare event. With many environmental exposures, such as those in air and domestic water, the attributable risk can be quite large. Even a small RR can be important if it applies to hundreds of thousands of people.
Cohort studies or even case-control studies large enough to detect the effects of usually low-level environmental exposures may be prohibitively costly in many cases. Ecologic studies, although they lack the detail available in studies that define individual exposures and are not often helpful in quantifying relations between exposure and disease, provide relatively inexpensive access to large populations. New methods for estimating exposure in ecologic studies may address some of the common concerns with ecologic analysis and improve its usefulness in environmental epidemiology.
Efforts to improve on existing methods for ecologic analysis and to make greater use of existing data sets should be encouraged. Recent advances in the
use of ecologic data have greatly improved the value of such information. Efforts should continue to refine these analyses.
Special emphasis should be given to establishing training programs in environmental-exposure assessment. Whatever data gaps exist in this area can be filled only when sufficient personnel and resources are used to conduct the needed assessments. We recognize that personal monitors provide sophisticated information about exposure. However, in designing studies with fixed budgets, researchers must weigh the benefits of improved exposure measures against the costs of reduced study power. In many circumstances, exposure assessment can be adequately estimated without personal monitors. Better use of activity logs and the gathering of more information about environmental conditions, such as sources of indoor heating and cooling, can also refine exposure assessments. The development of these and other techniques for advancing exposure assessment should be supported.
Health outcomes of interest to environmental epidemiologists range from syndromes or constellations of clinical measures of physiologic or neurobehavioral function to well-characterized diseases, such as cancer. Changes in patterns of these health outcomes can be effects of environmental factors. To ascertain whether changes in chronic diseases have occurred, researchers need to determine the expected, or baseline, rates of those diseases. This is difficult, especially for diseases other than cancer, because case definitions for various diseases and syndromes are not uniform or well defined, registries and codings are not uniform and consistent and may not even exist, other demographic variables can influence the outcome, and time trends may not be reliable, because of changes in case ascertainment.
We recommend that agencies concerned with the prevention of disease and the promotion of health make a deliberate effort to identify chronic diseases and syndromes that appear to be increasing or to have an important impact on public health. Concern should not be limited to fatal diseases. All forms of morbidity, ranging from ocular effects to impaired reproductive health, should be considered in epidemiologic studies. Symptoms, such as headaches and respiratory irritation, may be disabling and are appropriate for study.
Based on those strategies, registries should be established that can be connected with existing data systems, such as hospital records or health administration information. Thus, the Health Care Financing Administration includes data on renal dialysis, hospitalization for asthma, and chelation therapy. Linking this information with better characterization of the cases might allow researchers to determine time trends in underlying diseases.
Where time trends reveal recent shifts in disease patterns, researchers should explore possible etiologic explanations. Health-information databases should contain variables that would facilitate linkage to exposure information and other
relevant factors to improve analyses of local patterns and relations among possible causal factors. Furthermore, the databases should include demographic information so that populations that are more susceptible to different environmental exposures, or more exposed, can be identified.
Notwithstanding the initial enthusiasm for their potential, biologic markers need to be validated as to their specificity for exposure and health outcome. Major gaps exist, as many of the markers that have been identified and characterized in the laboratory have not been specifically tied to chemical, biologic, or physical exposures. Validation of these will strengthen the ability of environmental epidemiology to discern relations between exposure and disease.
Almost no data systems focus exclusively on the environment-health relation. The committee identified existing data systems that could be improved in this respect. Human-health surveys—such as the National Health and Nutrition Examination (NHANES), the National Health Interview Survey (NHIS), and the National Vital Statistics Program—all have followup capabilities and should collect followup information on both exposure and health.
The federal government should coordinate the evaluation of existing data systems and data collection to develop modifications that might enhance the linkage and usefulness of data. One simple addition that could enhance the utility of NHANES and NHIS would be to obtain information on residential histories. Such an effort should involve experts from multiple agencies and from outside government. Emphasis should be placed on better documentation of the degree to which data systems represent or characterize a larger universe, such as a population or regional environment. Common definitions and specifiers of disease or health status, of geographic location, and of chemicals and other relevant data items would also enhance usability of these systems.
Entities outside government have a legitimate need for access to data collected by government agencies and for linking data from different data systems.
Use of data by outside researchers should be encouraged, under appropriate procedures to guarantee confidentiality. Because linkage of data often requires the use of confidential identifiers, agencies should where possible generate linked data tapes, stripped of identifiers.
Academic researchers often fail to recognize the multiplicity of questions and research designs that can be addressed with data systems. There is a tendency for academic researchers to discount data collected from regulatory-agency data systems. There is a tendency for policy-makers to consider the design or funding of data systems as though other data systems do not exist or as though regulation is the only purpose of a data system. The potential impact of the environment on health is too great to be approached piecemeal.
The federal government should establish mechanisms to track the health and
illness experience of populations for which data on exposures and other baseline measurements are available. For example, the National Death Index is an essential and widely used resource for the public-health community and should be maintained.
Wide application of statistical techniques to adjust for autocorrelation and covariance structures will enhance the ability of environmental epidemiology to identify exposure-related diseases. Statistical modeling (e.g., logistic regression and Poisson regression) is an effective tool, particularly for the multifactorial outcomes generally examined in environmental epidemiology. In carrying out such statistical modeling, one must recognize that relations between environmental toxicants and health outcomes may be nonlinear. Statistical models that implicitly assume linearity may miss important associations or important features of associations. Modern statistical-modeling techniques allow the investigation of nonlinear models.
Many preliminary environmental-epidemiologic studies rely on inadequate databases for exposures and health outcomes. Although some of the databases need to be improved, researchers also need to develop a greater degree of sophistication about their limitations and to refine their ability to analyze existing information.
To the extent that the estimates of gradients of exposure are improved, the ability to detect associations of exposure and response is also improved. Statistical models for improving exposure assessment can help with this process.
Although not treated in this report, meta-analysis is a potentially useful tool for the analysis of environmental-epidemiologic data. However, attention will need to be given to improving data collection so that results are amenable to meta-analysis and to the refinement of methods for performing meta-analyses.
|This page in the original is blank.|