7

Findings, Conclusions, and Recommendations

This chapter builds on the foundation laid in Chapters 1–6 to draw out the overarching themes of the report and present the committee’s findings, conclusions, and recommendations related to its statement of task. It focuses on the major issues that the Department of Veterans Affairs (VA) will need to address as it chooses how to administer and make best use of the registry in the future. Details of and the scientific backing for the findings, conclusions, and recommendations offered may be found in those earlier chapters.

OPENING OBSERVATIONS

VA was presented with a challenge when it was directed by Congress to design, test, and implement an environmental health registry for “individuals who may have been exposed to toxic airborne chemicals and fumes caused by open burn pits” in 12 months. It set a goal to create a Web-only instrument that would cover military personnel who served in the Southwest Asia theater of operations from the beginning of the Gulf War conflict in 1990 through the present (2016) and include data from other VA and from Department of Defense (DoD) databases to lessen the burden on participants.

The National Academies of Sciences, Engineering, and Medicine,¹ which was tasked with organizing an expert committee to analyze the initial months of data gathered by the registry and offering observations regarding its operation and the use of the information it generates, appreciates the difficulty of the work undertaken and the obvious effort VA put into accomplishing it. The committee’s responsibility, though, was to conduct a critical inquiry. It would like to make clear that the comments it offers are made in the spirit of helping VA to make the best use of registry they were directed to establish.

The committee wishes to acknowledge the contributions made by the participants in its 2015 workshop. Their expertise, insights, and personal stories, summarized in Chapter 1, greatly aided the committee’s understanding of airborne hazards, open burn pits, and service members’ health issues.

___________________

¹ The work was done through the operational unit formerly referred to as the Institute of Medicine (IOM); as of March 2016, the Health and Medicine Division continues the consensus studies and convening activities previously undertaken by the IOM.

THE AIRBORNE HAZARDS AND OPEN BURN PIT REGISTRY QUESTIONNAIRE

While registries that rely on voluntary participation and self-reported information are a common means of collecting data on large populations, they are an intrinsically poor source of information on exposures, health outcomes, and possible associations among these events. Even under the best of circumstances, there are substantial limits to the accuracy of the data and—when the respondents constitute only a small, unrepresentative fraction of the eligible population—the generalizability of analyses made with them as well.

These weaknesses are apparent in the Airborne Hazards and Open Burn Pit (AH&OBP) Registry questionnaire and in the data collected in the registry’s first 13 months. The weaknesses have been exacerbated by a series of flaws in the structure and operation of the questionnaire and in the questions that are asked and the way they are asked. The AH&OBP Registry questionnaire is flawed in that it

• inappropriately uses questions that were validated for and meant to be administered by other survey means such as a face-to-face or computer-assisted phone interview;
• asks questions that may be confusing for respondents because they are ambiguous or otherwise poorly written;
• elicits information on topics such as hobbies and places of childhood residence that do not yield information that could be productively used in any analysis that would be appropriate to undertake using registry data;
• fails to ask questions (regarding non-burn-pit trash burning, for example) that could yield information related to relevant exposures;
• does not take full advantage of its Web-based format to streamline and focus questions based on previous responses;
• does not permit answers to be supplemented or updated later in time; and
• requires respondents to complete a sometimes lengthy set of repetitive questions regarding deployments before addressing core issues such as health, increasing the possibility of response fatigue.

Examples of these weaknesses are offered in Chapter 2. Their cumulative effect is evidenced by the high percentage of respondents who initiated but did not complete the instrument and the number of questions that had large nonresponse rates.

The issue of how to improve the questionnaire depends critically on the registry’s intended purpose(s) going forward. VA has articulated several different purposes in various documents: to help monitor health conditions affecting eligible veterans and service members; to improve VA programs to help veterans and service members with deployment exposure concerns; to generate potential hypotheses about exposure–response relationships; to improve programs in the Veterans Health Administration; and to provide outreach to veterans who may have experienced adverse health outcomes as a result of their exposures. However, with the exception of compiling a list of persons who may benefit from future outreach efforts and the possible exception of hypothesis generation, it does not appear that the registry as currently configured is fit for the articulated purposes.

Given the inherent weaknesses of registries that rely on voluntary participation and self-reports of crucial information, the committee concludes that the best ways to make use of the registry are to

1. make it a means for the eligible population to document their concerns over health problems that may have resulted from their service, bring those concerns to the attention of VA and their health care providers, and supply VA with a list of persons who are interested in burn pit exposure issues; and
2. generate data on the prevalence of health problems in the respondents that might possibly be used to stimulate research using more sophisticated analysis means.

If VA chooses to use the registry for these purposes, then the questionnaire may be simplified as follows:

Eliminating these categories would make the questionnaire easier and faster to complete and would better focus it on the needs of the eligible population.

More generally, the AH&OBP Registry’s data collection, administration, and management efforts would be improved by taking these steps:

ANALYSIS METHODS AND DESCRIPTIVE STATISTICS FOR THE REGISTRY DATA

VA made data from the first 13 months of the operation of the AH&OBP Registry (June 2014–July 2015) available for analysis by the committee’s contractor. Not all of the fields requested were provided because none of the data that the committee were allowed to access could contain information that VA deemed to be personally identifiable. This restricted the type and level of detail of the analyses that could be conducted on the data, and it prevented the committee from carrying out some of the work specified in its statement of task. These constraints also affected the confidence with which the committee can draw conclusions regarding the process of data acquisition and the validity of the information reported on exposure and health outcomes.

Another major limitation is that questionnaire and other data were made available only for those who finished and submitted the questionnaire. A VA report (2015) indicated that nearly 40% of those who initiated an AH&OBP Registry questionnaire did not complete it; this is an outcome that should be followed up.

The committee supplemented the information made available for its analysis with reports and a peer-reviewed paper generated under the direction of VA. The resulting analyses of registry data had access to more—and more detailed—information than was available to the committee, and while they covered different time periods than the dataset used for the committee’s analyses, they provided some additional insights.

Over the registry’s first 13 months, approximately 47,000 people completed the questionnaire, representing 1.0% of eligible Gulf War veterans and 1.7% of eligible post-9/11 veterans. Approximately 7.5% of registry respondents served in the 1990–1991 Gulf War only. Compared to post-9/11 respondents, they are more racially diverse, had less education, deployed fewer times, and were older, more likely to be enlisted, to have served in the Army, and to have been active-duty. Post-9/11 respondents comprises the majority—more than 85%. Analyses were adjusted for demographic and military characteristics, but factors such as the older age of Gulf War veterans might be more salient when examining associations with respiratory and cardiovascular diseases, which are more likely to become more prevalent as the population ages.

Among the most notable of the observations that can be drawn from these data are that nearly all respondents reported one or more airborne hazards encountered in theater: 96% of all respondents reported being exposed to a burn pit on at least one deployment, and 85.6% of Gulf War era respondents reported exposure to smoke from oil-well fires, while 85.2% of all respondents reported exposure to dust storms. The lack of data on those who were deployed and do not believe they were exposed to burn pits precludes using the registry to compare exposed to unexposed individuals. Therefore, the only means available for evaluating burn pit exposure is to examine gradations of exposure among the respondents.

Several other variables have high rates of consistent responses (showing little variability), making them ultimately of little use for analyses, and a number of questions had nonresponse rates of greater than 15%. These findings lend additional supporting evidence that many of the questions are poorly worded or otherwise problematic.

Analyses of demographic data indicate that neither the Gulf War nor the post-9/11 era registry respondents can be considered representative of their respective eligible non-respondent populations. This means that findings made using the registry data—which represent the experience of a small, non-random, self-selected sample—are not generalizable to the broader, eligible population and cannot be used for making inferences concerning it.

ANALYSIS METHODS AND INTERPRETATION OF REGISTRY EXPOSURE DATA

The committee identified several problems with the way the registry’s exposure data were collected that were compounded by the inherent limitations of self-reported information. One problem, already mentioned, was found with deployment-related exposure questions, which required respondents to recall specific information for each of the locations they were assigned to. Another was that the questions do not provide information on the intensity of exposure. A high fraction of registry participants reported potential exposures to both burn pit emissions and dust, and there was a tendency for individuals reporting exposures to one type of source to report exposures to other sources as well. This raises concerns about the representativeness of the data and its usefulness in evaluating associations between exposures and health outcomes.

Given the charge—and a concern for overinterpreting the data at hand—the committee developed a reduced set of metrics to categorize exposure potential for the purpose of analysis. Because there were many sources of airborne emissions that contributed to a service member’s exposures to particulate matter and chemical exposures and insufficient data by which to determine which sources contributed the most or posed the most harm, the committee chose to weigh each potential exposure equally and focus on the totality of exposures.

On the basis of its evaluation, the committee concludes that the exposure data are of insufficient quality or reliability to make them useful in anything other than the most general assessments of exposure potential. Given this limitation, the committee believes that there may be some circumstances where supplementing these data with information from on-site environmental monitoring or with meteorological, satellite, or other relevant measurements or observations might yield results that would suggest that some individuals or groups experienced greater or lesser exposures to specific constituents that might stimulate more detailed assessments of health outcomes in particular populations.

ANALYSIS METHODS AND INTERPRETATION OF REGISTRY HEALTH OUTCOMES DATA

The committee took an approach analogous to that used for exposure data in order to characterize the health outcomes data for analysis purposes—specifically, generating variables using multiple grouped indicators of these

outcomes. While the limitations of the AH&OBP Registry questionnaire and the data collected by it are too great to allow any firm conclusions to be drawn from its analysis, the health outcomes data related to the symptoms, conditions, and diseases associated with the respiratory and the cardiovascular systems are the best candidates for study because these constitute the most plausible and well documented potential health effects of the exposures of concern.

Generally speaking, the committee found that the observed prevalences of respiratory and cardiovascular outcomes appear consistent with what would be expected in a population that is predominantly male, aged 25–60, and for whom about one-third report a current or former history of smoking. It concluded that the health data may be of sufficient quality to justify internal comparisons in which subsets of registry participants with varying levels of potential exposure are compared with one another. An examination of multiple indices of exposure to burn pit emissions and other hazards associated with deployment showed that registry participants who reported more exposures of all types also tended to report more health problems of all types.

The committee’s exposure potential variables had strong and consistent associations with self-reported asthma; any respiratory symptom; emphysema, chronic bronchitis, or chronic obstructive pulmonary disease (COPD); functional limitations due to lung or breathing problems; cardiovascular disease; and hypertension. Importantly, though, the analyses also uncovered some unexpected findings that are not consistent with currently understood scientific mechanisms of exposure and outcome, such as a statistically significant association between higher self-reported levels of asbestos exposure and a higher prevalence of neurologic, immune, or liver conditions. Such outcomes strongly suggest that the results of analyses of the registry data cannot be taken at face value and that the identified associations may be an artifact of the population’s selection and the limitations of the self-reported exposure and disease data.

Again, the bottom line is that registry analyses are not generalizable and can only describe what exposures and conditions the population of registry respondents are reporting: registry data cannot be used to determine cause or to estimate prevalence in the total eligible population of service members or veterans. The committee wishes to emphasize that it would have made this same determination had the analyses found no associations or weak associations between the exposures and health outcomes.

The strong conclusion that can be drawn is that a more rigorous and appropriate study design is needed to examine the relationship between the exposures encountered during deployment to the Southwest Asia theater of operations and health outcomes. While the registry provides a forum for collecting and recording information on those who were deployed and who are motivated to participate, it cannot answer such questions.

The 2011 Institute of Medicine (IOM) report Long-Term Health Consequences of Exposure to Burn Pits in Iraq and Afghanistan contains advice and recommendations on how such a study might be conducted.

The National Research Council (NRC) Review of the Department of Defense Enhanced Particulate Matter Surveillance Program Report recommended that a “complete inventory of all major sources of ambient pollutants and potential emissions in the theater should be constructed before assessment of health effects to ensure that all relevant pollutants are monitored” (NRC, 2010, p. 73), and recent advances in low-cost air pollution monitoring devices (Manikonda et al., 2016; Wang et al., 2015) make such data collection efforts more feasible than ever before.

The committee believes that, while medically verified health outcomes information exists only for the subset of the population that uses VA health care,² there is potential value in linking the registry data to health care use and conducting analyses on registry participants. Comparisons between self-reported information collected by the questionnaire and diagnoses in VA medical records for respondents who use VA health care would provide further information concerning the level of validity of self-reported health outcomes in the population of respondents.

___________________

² These data were not available to the committee but are contained in the Veterans Health Administration records.

Given this and the committee’s other findings regarding the registry:

The AH&OBP Registry has many flaws, but even a well-designed and executed registry would have little value as a scientific tool for health-effects research compared to a well-designed epidemiologic study. Addressing the issues identified by the committee would, though, improve the registry’s utility as a means of

• generating a roster of concerned individuals that VA can use for targeted outreach, surveillance, and health-risk communication;
• creating, via the completed questionnaire, a record of potential exposures and health concerns that is recorded in the participant’s VA electronic health record; and
• allowing VA users and nonusers who take part in the optional clinical exam to articulate concerns they may have to a health care provider and, if warranted, undergo appropriate diagnostic testing or referral, and begin treatment to improve symptoms.

OTHER FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS

The committee was also asked to offer observations on some additional issues surrounding the registry and the actions being taken by DoD and VA to address airborne hazards and open burn pit questions. Specifically, the legislation that directed VA to establish the registry called for

• an assessment of the effectiveness of actions taken by the [Department of Veterans Affairs and the Department of Defense] to collect and maintain information on the health effects of exposure to toxic airborne chemicals and fumes caused by open burn pits;
• recommendations to improve the collection and maintenance of such information; and
• using established and previously published epidemiological studies, recommendations regarding the most effective and prudent means of addressing the medical needs of eligible individuals with respect to conditions that are likely to result from exposure to open burn pits (Public Law 112-260 § 201(b)(1)(A)(i–iii)).

To date, other than the AH&OBP Registry and the airborne exposures and health information collected as part of such efforts as the Gulf War Registry and Millennium Cohort Study, there are no systematic data collection or maintenance efforts focused on the effects of burn pit emissions.³ Very limited in-theater air pollution data gathering efforts have generated information that would aid in studies of those who served in the same place and at the same time as measurements were made. Two previous reports have offered recommendations on how more rigorous and useful data could be collected: Review of the Department of Defense Enhanced Particulate Matter Surveillance Program Report (NRC, 2010) and Long-Term Health Consequences of Exposure to Burn Pits in Iraq and Afghanistan (IOM, 2011). This committee concludes that the recommendations these reports offer regarding, respectively, environmental sampling in a combat theater and the conduct of a prospective study of the long-term health effects of exposure to burn-pit emissions are still salient and, if implemented, would materially improve the knowledge base on the health effects of past, present, and future in-theater exposures.

The committee’s assessment of “established and previously published epidemiological studies” requested by

___________________

³ DoD and VA collect and analyze data on all medical conditions in the populations that participate in their health care programs, but these are not specific to airborne hazards or burn pit emissions.

the registry’s enabling legislation⁴ found very few that addressed service members and veterans exposed to open burn pits. The results of these studies do not suggest any general course of action for addressing the medical needs of this population beyond the steps that health care providers should already be carrying out: taking a thorough history, including all occupational exposures, and listening carefully to each patient to determine whether personalized diagnostic testing or treatment is indicated. The health care provider instructions for AH&OBP Registry clinical examinations published by VA (2016) are sound guidance on this.

The information developed by the registry has limited value for improving individual patient care. However, while these data may be inappropriate for evaluating the association between exposures and health outcomes, there are other ways in which they may be useful. As has already been mentioned, the committee believes that the registry’s primary utility is that it provides a means for veterans and service members to document their concerns about wartime exposures and the health problems that might have resulted from them and to bring these to the attention of both VA and its health care providers. The self-reported signs, symptoms, and diseases identified by registrants constitute a record that can alert providers to concerns and problems that may be forgotten or missed during clinical encounters.

The registry questionnaire collects a number of pieces of information that would facilitate conversations between a patient and a health care provider, without regard to whether the information might be relevant to AH&OBP exposures. For example, someone who reported difficulty walking long distances or climbing stairs might be experiencing joint pain, respiratory problems, atherosclerotic vascular disease, congestive heart failure, obesity, or even anxiety. Similarly, a complaint of chest pain can have multiple causes in addition to angina and coronary artery disease, including gastroesophageal reflux disease, chest wall pain or costochondritis, and anxiety. And often these symptoms can be multifactorial in origin. Registry questionnaire responses are already accessible to VA health care providers as part of a veteran’s electronic health record, and a complete set of responses may be downloaded and printed for a respondent to take to a clinical visit with a provider outside of the VA system.

Providers often have little time to get histories and patients do not always do a good job of raising concerns so a succinct summary would greatly benefit both.

The data the registry generates on the number of respondents who report particular health problems may also be useful to VA. For example, several thousand individuals have indicated that they have diagnosed or self-reported cardiopulmonary symptoms. If these persons subsequently present for evaluation or treatment at rates that would not otherwise have been anticipated by VA, it would indicate that the registry could be used as tool for anticipating future demand for particular provider services. However, it remains to be seen whether this would be the case, and the number of individuals who have thus far completed the questionnaire is only a tiny fraction of the overall population eligible for VA care.

Given the demonstrated concerns of respondents regarding the health effects of exposure to airborne hazards and open burn pit emissions, it is unclear why so few have yet to arrange for the optional in-person clinical evaluation by a VA provider that is made available as part of the registry.

___________________

⁴ Presented in Chapter 1 of the report.

Adding a means of scheduling an exam as part of the questionnaire—a capability that the committee understands is being implemented—is a useful first step.

CLOSING OBSERVATIONS

The committee recognizes the great interest that active duty military personnel and veterans who served in Iraq, Afghanistan, and the greater Southwest Asia theater of operations have in understanding potential threats to their health from airborne hazards and open burn pit exposures. As its analysis has made clear, though, there are inherent features of registries that rely on voluntary participation and self-reported information that make them fundamentally unsuitable for addressing the question of whether these exposures have, in fact, caused health problems. All parties—service members, veterans, and their families; VA; Congress; and other concerned people—would benefit from having a realistic understanding of the strengths and limitations of registry data so that they can make best use of them and, if desired, conduct the kind of investigations that might yield salient health information and enhance health care for those affected.

REFERENCES