One of the goals set forth by Congress and the Department of Veterans Affairs (VA) for the Airborne Hazards and Open Burn Pit (AH&OBP) Registry is to support research about the health effects caused by deployment-related airborne hazards, including burn pits. In PL 112-260 § 201, Congress specifies that the registry should include information “necessary to ascertain and monitor the health effects of the exposure of members of the Armed Forces to toxic airborne chemicals and fumes caused by open burn pits”1 and that the assessment of the AH&OBP Registry include the “effectiveness of actions taken by the Secretaries to collect and maintain information on the health effects of exposures to toxic airborne chemicals and fumes caused by open burn pits.”2 VA encourages veterans’ participation, saying “help put data to work for Veterans through research about potential health effects of airborne hazard exposures” and “[b]y joining the registry, you can provide information to help us better understand whether long-term health conditions may be related to these exposures” (VA, 2021a). Additionally, the reassessment committee’s Statement of Task requires that it offer “recommendations on how to address gaps identified in the initial report and on burn pit and airborne hazards research, diagnosis, and treatment in general.” The committee considered that the best method by which to address the Congressional mandate to assess whether exposures to airborne hazards and burn pits cause health outcomes is etiologic research, which it defined as the use of hypothesis testing to answer questions about whether a particular condition or factor, such as exposure to an environmental agent, causes adverse health outcomes.
This chapter explores the degree to which the AH&OBP Registry can be used for etiologic research that would support findings of causation between airborne hazards and health effects. The first section of this chapter briefly discusses how causation is determined. The second section describes current efforts, particularly by VA’s Airborne Hazards and Burn Pits Center of Excellence (AHBPCE), to use AH&OBP Registry data for etiologic and other research. The third section discusses six important considerations that determine a registry’s effectiveness for use in etiologic research in order to assess the research capability of the AH&OBP Registry. The committee then considers data sources that could serve as alternates for etiologic research on open burn pits and airborne hazards. Finally, the committee revisits the initial assessment recommendations relevant to etiologic research, VA’s response to those recommendations, and the reassessment committee’s findings on the extent to which VA’s response addressed those recommendations.
1 PL 112-260 Dignified Burial and Other Veterans’ Benefits Improvement Act of 2012 § 201 (a)(1)(B).
2 PL 112-260 Dignified Burial and Other Veterans’ Benefits Improvement Act of 2012 § 201 (b)(1)(A).
Causality is more than an association between an exposure and a health outcome; it is a demonstration that an exposure(s) is responsible for, or causes, the specific health outcome(s). For every exposure–outcome relationship, there are gradations of evidence and certainty. Different study designs (e.g., observational studies and randomized controlled trials) and statistical methods can address questions of causality to varying degrees, depending on the hypothesis to be examined. Causality analyses are dependent on high-quality data on exposure, the presence or absence of selected health outcomes, and measures of effect modifiers and confounding factors that are known or suspected to be related to the exposure–health outcome pathway. Challenges to determining causality include the need to identify and measure the specific environmental hazards, the difficulty of linking an exposure to an airborne hazard with a health outcome, and objectively assessing relevant health outcomes.
Factors to guide assessments of causality have been proposed, such as those by Bradford Hill (Hill, 1965). These factors include temporality (i.e., exposure must be present before the health outcome), strength of association, dose–response relationships, consistency of evidence through replication of findings and other knowledge, specificity of the association, biologic plausibility, coherence of evidence, and consideration of alternate explanations. These factors are guidelines, not a checklist that requires each to be satisfied. Evidence of an association may be based on cross-sectional study designs or on other designs that do not account specifically for temporality between the exposure and outcome; however, to assess causality with certainty, temporality must be established.
The initial assessment committee identified several challenges to using the AH&OBP Registry data to examine associations between airborne hazards and health outcomes, including participation biases, recall biases, and data quality. In particular, that committee concluded that the exposure data are of insufficient quality or reliability to draw conclusions about the association between exposures and health outcomes, but it did find that the exposure data were appropriate for the most general of assessments (NASEM, 2017).
However, that committee was hopeful that supplementing AH&OBP Registry data with information from on-site environmental monitoring or with meteorological, satellite, or other relevant measurements or observations would make it possible to show that some individuals or groups experienced higher or lower exposures to specific airborne hazards, which might stimulate more research on health outcomes in particular populations. The initial assessment committee’s analyses used registry health data and found many suspect associations, including some unexpected findings that are not consistent with currently understood scientific relationships between exposures and health outcomes. For example, that committee found a statistically significant association between higher self-reported levels of asbestos exposure and a higher prevalence of neurologic, immune, or liver conditions (NASEM, 2017).
There are many characteristics of the registry that could contribute to false associations, such as selection bias, exposure misclassification, uncontrolled confounding, and underlying data problems. Therefore, the initial assessment committee found that the analytic results of the registry data were unusable and that the identified associations or lack thereof may be an artifact of the limitations of the underlying data. That committee stated that “the bottom line is that registry analyses are not generalizable and can only describe what exposures and conditions the population of registry respondents are reporting; registry data cannot be used to determine cause or to estimate prevalence in the total eligible population of service members or veterans” (NASEM, 2017, p. 6). It urged VA to use more rigorous and appropriate study designs to examine the relationship between the exposures to airborne hazards encountered during deployment to the Southwest Asia theater and health outcomes; similar recommendations have been made in other National Academies reports (IOM, 2011; NRC, 2010). While the registry collects and records information, it cannot be used to establish causality (NASEM, 2017).
Despite the initial assessment committee’s conclusions and the fact that the registry questionnaire has changed only slightly since the 2017 report (see Chapter 3), VA continues to use registry data in attempts to inform its etiologic research efforts. During a VA webinar to educate a general audience of service members and veterans about airborne hazards, presenters stated that the AH&OBP Registry offers clinical and research opportunities,
including enhancing exposure metrics, merging information with health records, and creating case–control research designs (VA, 2022a). VA researchers use primarily descriptive statistics to estimate associations between specific deployment-related exposures and health outcomes. VA also uses the registry to support research by identifying people who meet specific inclusion criteria to create clinical cohorts for more in-depth medical examinations through its AHBPCE.
Publications and Abstracts
The committee reviewed publications and abstracts that used the AH&OBP Registry data without any limits on publication date or study type. Since the registry’s inception, few studies have been published in the peer-reviewed literature that used registry data for research purposes. As of April 2022, only three peer-reviewed publications used AH&OBP Registry data or its participants to examine relationships between exposure and disease (Jani et al., 2017b; Liu et al., 2016; Powell et al., 2020). Another two studies used questions from or modified from the AH&OBP Registry questionnaire (Bith-Melander et al., 2021; Poisson et al., 2020). Although analyses of the registry data have also described medical needs and risk factors among participants and have been presented at multiple conferences (Jani et al., 2017a, 2019a,b, 2020a,b, 2021; Molina et al., 2019), none of these abstracts have resulted in peer-reviewed publications.3 One recent analysis used the registry to describe factors relating to COVID-19 (Jani et al., 2021). These studies and abstracts are summarized next.
Liu et al. (2016) used a subpopulation of registry participants (n = 4,343) with deployment dates and locations of military bases with and without documented burn pits to examine the associations between geographically inferred exposure and self-reported burn pits emissions exposure and self-reported respiratory and cardiovascular conditions. This study was also summarized in the initial assessment (NASEM, 2017). Self-reported diagnoses were compared with VA medical record information for 2,857 respondents who used VA health care at least once between January 2007 and November 2015 and who were not diagnosed with a condition before deployment. A strong dose–response association was found between cumulative days deployed within a 2-mile radius of a burn pit and self-reported emphysema, chronic bronchitis, or chronic obstructive pulmonary disease (COPD), but the association was not statistically significant when VA medical record diagnoses were used in place of self-report. In both the self-report and VA medical record analyses, no dose–response associations were found between cumulative days of deployment near a documented burn pit and the incidence of asthma, hypertension, or cardiovascular disease. There was a statistically significant association between the self-reported hours per day of burn pit smoke exposure and self-reported hypertension and the combined respiratory outcome of emphysema, chronic bronchitis, or COPD, although there was no association between burn pit emissions and any of the respiratory or cardiovascular diagnoses in VA health records. The low correlation between self-reported diagnoses and the diagnoses in VA medical records may indicate a misidentification of self-reported health conditions, which in turn implies that analyses using self-reported diagnoses may be biased (Liu et al., 2016).
Jani et al. (2017b) examined the association between self-reported blast exposures and dyspnea (shortness of breath) or decreased ability to exercise among 42,558 registry participants. Blast exposure was reported by 79.0% of those with current symptoms and by 65.5% of those without current symptoms. The multivariable model, which controlled for age, sex, branch of service, body mass index, smoking status, exposure to burn pit smoke, nonmilitary occupational exposures, and time since deployment, found a positive association between dyspnea or decreased ability to exercise and blast exposure (odds ratio = 1.66, 95% confidence interval 1.5–1.7). The authors emphasized that these results were an indication that associations between burn pits and cardiopulmonary effects may be incomplete without considering other contributing exposures, such as blasts.
A series of abstracts have been presented at national conferences. They include the following:
- a nested case-control study of self-reported health care-professional–diagnosed constrictive bronchiolitis and quartiles of self-reported burn pit exposure among registry participants (Jani et al., 2017a);
3 Personal communication, Michael Falvo, co-director, VA Airborne Hazards and Burn Pits Center of Excellence, War Related Illness and Injury Study Center. January 18, 2021.
- high rates of insomnia, sleep disordered breathing (snoring), and sleep apnea among male and female registry participants (Jani et al., 2020a,b);
- health care needs and use among registry participants performed by comparing participants who received the optional AH&OBP Registry health evaluation and who had chest x-rays and/or pulmonary function testing within ±5 years of the evaluation (Jani et al., 2019b);
- the degrees of agreement among self-reported symptoms and conditions in the AH&OBP Registry and as documented by International Classification of Diseases (ICD)-9 and ICD-10 codes in the participants’ Veterans Health Administration (VHA) medical records, and diagnoses recorded on the standard note template for the optional AH&OBP Registry health evaluation (Jani et al., 2019a; Molina et al., 2019); and
- COVID-19 among registry participants who also use VA health care for COVID-19 testing amid concerns that deployment-related exposures could affect COVID-19 morbidity and mortality (Jani et al., 2021).
Although these primarily descriptive analyses explore relationships between possible exposure and health outcomes and discuss the types of questions to pursue, they are not strict hypothesis testing research, nor are they focused on establishing causality. Rather they only provide descriptive statistics of characteristics and risk factors of disease among participants. In addition, the committee would not consider AH&OBP Registry data presented as case studies or case series to be etiologic research.
Because the registry comprises a concerned, engaged population who report high rates of exposures (such as emissions from burn pits and dust), it could be useful for examining seemingly unrelated issues, such as COVID-19, although there would still be underlying data limitations. The ability to quickly reach out to registry participants to collect additional information and disseminate educational materials gives VA the flexibility to address new challenges in a more agile way. For example, VA used AH&OBP Registry information to examine rates of COVID-19 among participants (Jani et al., 2021).
Powell et al. (2020) examined 145 veterans with exertional dyspnea who were part of the STAMPEDE III cohort between June 2012 and June 2015 and who also completed the AH&OBP Registry questionnaire. STAMPEDE III is an ongoing prospective study of 380 military personnel with chronic dyspnea following deployment to Iraq and Afghanistan (described in Chapter 4). Of the 145 eligible veterans, 100 participants who had completed all pulmonary function testing and, if warranted, overnight polysomnography were divided into a burn pit exposure group (n = 45) and a non-burn pit exposure group (n = 55). The two groups were similar on sex (mostly male), age, body mass index, and lung function as determined by spirometry. The prevalence of obstructive sleep apnea was similarly high in both groups (69% vs 71%, p = 0.83). The mean apnea-hypopnea index was lower in the exposed group (12.8/hour vs 19.7/hour, p = 0.04), suggesting that self-reported burn pit exposure is not related to obstructive sleep apnea.
One group of investigators had two publications (Bith-Melander et al., 2021; Poisson et al., 2020). The first was a pilot study (Poisson et al., 2020) of 109 service members and veterans who had served in Operation Iraqi Freedom (OIF) between 2003 and 2011 and who were recruited through veteran-centric websites, newsletters, chatrooms, Twitter, and Instagram from July to October 2018. Selected exposure questions from the AH&OBP Registry questionnaire were part of a 66-item Internet-based questionnaire on basic demographics, health status, and exposures. This study aimed to inform strategies for an improved medical screening process for veterans who have been deployed to Iraq. The descriptive results did not include modeling or adjustment. While the study did not directly use AH&OBP Registry data, questions on exposures and “epidemiologic variables” were adapted from the AH&OBP Registry questionnaire, as the authors stated that those registry questions had been reviewed for psychometric properties such as consistency and accuracy. However, the committee has no evidence that the AH&OBP Registry questionnaire had been reviewed for these properties (see Chapter 3). The second publication by this group (Bith-Melander et al., 2021) is a qualitative analysis that used findings from their pilot study and secondary data (source unknown) to develop a conceptual framework about the challenges that veterans encounter when seeking medical care, screening assessments, and subsequent treatments. Some of the same questions from the AH&OBP Registry questionnaire were used.
Airborne Hazards and Burn Pits Center of Excellence
In 2019, PL 115-244 (House Report 115-929) recognized AHBPCE, a component center within the War Related Illness and Injury Study Center (WRIISC), as a VA Center of Excellence. The mission of AHBPCE is to conduct clinical and translational research for health concerns related to airborne hazards and burn pit exposures, particularly self-reported respiratory symptoms and conditions, and to develop education content and identify best practices for health care providers, veterans, and other stakeholders. Its goals are to identify long-term health conditions associated with those exposures, identify research needed to better understand the potential health consequences of veterans’ exposure to these hazards, and to identify possible treatments for those health conditions. To accomplish those goals, AHBPCE uses the registry in three ways:
- To identify trends and health outcomes of concern among registrants for further study;
- To identify registry participants with certain respiratory health conditions (i.e., constrictive bronchiolitis, obstructive lung disease, pulmonary fibrosis or other lung disease, or shortness of breath of unknown origin) for further study; and
- To monitor registry data to identify trends in health outcomes so they can be addressed.
VA has stated that AHBPCE is studying specific categories of health concerns drawn from registry participants to generalize or extend the findings from an individual veteran to a broader group (or cohort) of veterans. Studies and clinical examinations conducted by AHBPCE involve many of the same tests used in specialty care, such as assessments of cardiopulmonary function and exercise ability. These assessments help AHBPCE make recommendations regarding the management of symptoms and follow-up care to improve the quality of life among veterans exposed to airborne hazards, including burn pits (VA, 2022b).
In addition to maintaining and using the AH&OBP Registry, AHBPCE supports a number of research initiatives. As of June 2021, AHBPCE had created two collaborative programs to address these goals:
- The AIMES (AHBPCE IQuEST [Center for Innovations in Quality, Effectiveness and Safety] Military Exposure Surveillance) Collaboration to “accelerate knowledge harvesting from the AHOBP [Registry] and related data to improve the care and health of Veterans with airborne hazards concerns” (VA-AHBPCE, 2021, slide 24); and
- The Post-Deployment Cardiopulmonary Exposure Network (PDCEN) which invites selected registry participants to undergo in-depth clinical evaluations.
AIMES is intended to improve the veteran experience with the AH&OBP Registry health evaluation, to improve the consistency and reliability of these evaluations across VHA, and to enhance scientific understanding of airborne hazards, in part, through curation of a merged database of AH&OBP Registry and clinical data from the VA Corporate Data Warehouse (VA, 2021b; VA AHBPCE, 2021).
The second collaboration, PDCEN, is a nationwide program involving six VA physicians with expertise in pulmonary and occupational medicine who provide in-person or virtual telemedicine evaluations of veterans with cardiopulmonary conditions and concerns about airborne hazards or burn pit exposures. The goal of PDCEN is to better understand veteran health concerns in order to facilitate multisite research on treatments. PDCEN uses the registry to identify veterans who report certain conditions such as dyspnea, diagnoses such as COPD, or cardiopulmonary function abnormalities, or who have unknown conditions and who need specialized consultation and evaluation. To be eligible for the more comprehensive evaluation, veterans must be registry participants, receive VA health care services, and self-report having one of the qualifying respiratory conditions (VA, 2021c). Eligible and interested veterans are invited to receive a more in-depth core clinical evaluation. The PDCEN examination is standardized across the six PDCEN sites and includes comprehensive traditional cardiopulmonary testing as well as more specialized tests such as forced oscillometry. This evaluation is distinct from the AH&OBP Registry’s optional health evaluation, which is available to all registry participants and is conducted at VA medical centers or Department of Defense (DoD) medical facilities (VA AHBPCE, 2021). Efforts are made to capture temporal
information about exposures from the veterans. The goal of this network is to work collaboratively with AHBPCE to develop research questions, gather data, and conduct research to support the development of best practices and new clinical guidelines on the care of veterans who have been exposed to airborne hazards and burn pits. The committee notes that although this work could contribute to health care, it does not appear to meet the characteristics of etiologic research (see below).
Research on respiratory health issues conducted by WRIISC and AHBPCE investigators has resulted in several publications (Butzko et al., 2019; Falvo et al., 2016a,b; Garshick, 2019; Jani et al., 2017a; Lindheimer et al., 2019). AHBPCE researchers are attempting to examine in more depth the impact of deployment exposures on veteran health outcomes reported in the registry, such as blast injuries and cardiopulmonary symptoms (Jani et al., 2017b), as described in the previous section.
The goals of future AHBPCE research efforts are to improve the AH&OBP Registry to better serve the needs of both veterans and health care providers by revising reports in the clinical portal, allowing new deployment segments to be added, and implementing more accurate address and location matching. Some of this work may be conducted in collaboration with VA staff in Health Outcomes Military Exposures (VA AHBPCE, 2021).
Chapter 2 describes the more general use of exposure registries and common types of biases and their effects; this section describes characteristics that bear on the usefulness of registries as a resource for etiologic research in general but also on airborne hazards exposures and health outcomes specifically. The committee identified six characteristics that can be used to assess the potential of any exposure registry to be used for etiologic research and then applies them to the AH&OBP Registry. These characteristics correspond to the standard methodologic concerns in observational epidemiologic studies, namely random error (i.e., chance variation), selection bias, exposure and health outcome measurement error, unexplained variation in exposure or health outcome data, and confounding, and they have also been used in prior National Academies’ reports that reviewed epidemiologic studies of exposures and health outcomes in service member and veteran populations (NASEM, 2016, 2018, 2020a,b). The committee adapted these methodologic concerns to be applicable to the AH&OBP Registry. These characteristics are not intended to serve as a checklist or scorecard, but rather considerations most applicable for etiologic research on health effects of exposures to airborne hazards. Those characteristics are:
- a sufficient sample size for precise estimation (estimates that have minimal sampling variability) of causal effects;
- a representative sample of the population of interest;
- identification of an appropriate comparison population;
- an exposure assessment of adequate quality;
- a health outcome assessment of adequate quality; and
- identification of other contributing factors (that might be related to the exposure and the outcome that would distort their relationship).
The first characteristic addresses random error or chance variation, the second concerns selection bias, the third deals with confounding and selection bias, the fourth and fifth consider measurement error and misclassification, respectively, and the sixth addresses confounding. The committee considered other characteristics, such as longitudinal data with multiple time points of information, and determined that, while they can aid etiologic research, they are not always necessary.
Sufficient Sample Size for Precise Estimates
Sample size affects the ability of a particular study to precisely estimate the causal effect of exposure on health outcomes for a population, including the absence of an effect. In particular, larger sample sizes are needed
for precise estimates of rare health outcomes (e.g., constrictive bronchiolitis) and for those outcomes that have smaller causal effects. A critical consideration for etiologic research is to distinguish between studies that have small sample sizes and thus are unable to detect differences in the outcome and studies that appear to have had sufficient sample size to detect differences in outcomes if such differences did exist. In other words, a lack of statistical significance of an estimated causal effect does not necessarily imply a lack of true causal effect, especially if the studies are small and were not designed to examine the outcomes under consideration.
Systematic biases and errors as well as random error and uncertainty in estimates are important considerations in etiologic research. As data are rarely available on all of the people and outcomes in a given population, statistical approaches are used to appropriately represent the uncertainty from systematic biases and random error.
Of the approximately 3.7 million veterans and service members who are eligible to participate in the AH&OBP Registry, the committee had registry data for 278,645 participants as of February 1, 2022 (the date of the data extract). Thus, the sample size of the registry population would likely be sufficiently large to determine precise estimates of associations derived from the registry that do not involve rare health outcomes or rare exposure categories.
A Sample Representative of the Eligible Population
A representative sample of the eligible population may help to ensure that the findings are generalizable to the entire eligible population. The goal should be to make study samples as similar as possible to the population from which they are derived. Given a representative sample, the ability of a particular study to precisely estimate a causal effect of exposure on health outcomes for the population of interest (generalizability) is partially determined by the sample size (see the first characteristic). If the sample is not representative and the sample size is large, the estimate will likely be a biased—perhaps substantially—causal effect that has very little uncertainty (i.e., it is precise around the wrong estimate or answer).
Although the number of registry participants is large (over 278,000 in the committee’s dataset as of February 1, 2022; see Chapter 2), the participants are not representative of the total eligible population, based on era of service and their demographic and military characteristics. Notably, compared with the entire eligible 1990–1991 Gulf War population, registry participants were more likely to be women, to be over 50 years of age, to have served in the Army and as National Guard or reserve, and to have had a different distribution of eligible deployment segments. Comparisons of post-9/11 participants with the entire eligible era population found that women and Black individuals were under-represented, as were those 50 years of age or older, but that both White individuals and individuals less than 30 years of age were over-represented (see chapter 2, Table 2-4). Those with service in the Army and Air Force were over-represented, as were those from the National Guard or reserve. Deployment to Iraq or Afghanistan was also over-represented relative to the eligible population, as were those with four or more deployment segments. Defining an appropriate population denominator is also difficult because there is an ever-growing eligible population as more service members are deployed or redeployed and new countries or deployment locations, such as Syria, Egypt, and Uzbekistan, are made eligible for the AH&OBP Registry (see Chapter 3).
The characteristics of registry participants who completed the questionnaire before August 1, 2015 (early; n = 46,444) were compared with those who completed it after this date (late; n = 232,201); this is presented in Table 2-5 and also described in Chapter 3. Several differences in demographic and military characteristics are observed between early and late participants, but in general they varied only slightly, mostly with respect to average age at time of questionnaire completion, service branch and component, deployment era, and year of first and last deployment. Comparisons of early and late participants with the entire eligible population also showed that they are not representative. As described in Chapter 3, the additional years of data in the AH&OBP Registry following the initial assessment resulted in a more representative set of participants to complete the registry, although this
comparison is confounded by different rates of missing data on the population and changes in the eligible registry population over time. The additional 5 years of data collection brought the participant composition closer to the eligible population with regard to race. Although the late participants were less likely to have served in the Army, the registry still highly over-represents service in the Army and under-represents the Navy and Coast Guard. Finally, the late participants did not substantially differ from the early participants on unit component (active duty or reserve/National Guard) and continue to greatly over-represent active duty. Thus, adding more years of data collection showed some minor improvements on marginal distributions relative to the population on some, but not all, characteristics, and the registry participants are still not representative of the eligible population.
The decision to participate in a voluntary registry is often the result of personal experience and priorities, which can introduce a high likelihood of selection bias. Selection bias in this case means that individuals who are concerned about their health or who have a diagnosis they believe is due to deployment-related exposures—or burn pits specifically—are more likely to participate than individuals who do not perceive themselves as having been exposed to a deployment-related hazard or who are not experiencing adverse health issues. For example, as shown in Table 3-3, only 1.5% of registry participants indicated that they had none of the 10 possible exposures on any deployment, whereas the modal number of exposures was 6. Similarly, as shown in Table 3-4, only 2.2% of participants reported that they had none of the 21 included health conditions. Although the mean and median number of health conditions for both early and late participants was the same, the proportion who reported no health outcomes was higher among later participants than early participants, indicating that early and late participants are not the same. This further supports the possibility that any association between an exposure and a health outcome based on AH&OBP Registry data might be a function of this selection bias rather than a true causal effect.
Given the lack of representativeness of the registry participants to the eligible population, the registry does not satisfy this characteristic. Recruiting more individuals to participate in the AH&OBP Registry will not completely address or resolve these issues. As discussed under the first characteristic, given the large sample size, causal estimates using the registry data would result in a precise estimate of a biased (incorrect) causal effect.
Identification of an Appropriate Comparison Group
Being able to estimate an association or causal effect depends on the availability of an appropriate comparison group for the group to be studied. When comparing two or more groups, the goal should be to make them as similar as possible so that they differ only on level or degree of exposure. For example, to study the effect of an exposure on a population, comparison groups may be an entirely unexposed population or a population with different levels of the same exposure. In order to detect an association between the exposed and unexposed groups in an etiologic study, there needs to be some amount of exposure contrast or variability in the study population. Such comparisons can show the observed frequencies of various health conditions of interest in the exposed population. However unmeasured confounders or contributing factors can influence the comparisons (see section on Identification of Other Relevant and Contributing Factors). For example, if an exposed population is healthier than the nonexposed or less exposed population and no baseline assessments were made for either population, the observed association between exposure and health outcomes may not be a valid causal effect.
Ideally, the comparison group should closely resemble the study group in terms of the observable characteristics that are related to the risk of the health outcome (and to the exposure) so that differences in health outcomes can be attributed to the exposure of interest rather than to other factors; however, statistical methods can also be used to some extent to adjust for differences. These observable characteristics can include basic socio-demographic information, such as sex, age, education level or other socioeconomic indicators, and race and ethnicity. For military populations, additional service characteristics such as branch, component, rank, deployment dates and locations, and military occupational specialty are informative. Depending on the specific investigation, the study and comparison populations might be further restricted by, for example, service branch, a narrower age range, or specific deployment locations. Unobservable charcteristics related to the health outcome and the exposure cannot be adjusted for, although sensitivity analyses can be performed to assess their potential impact (Brumback et al., 2004).
The identification of an appropriate comparison group for service members deployed to the Southwest Asia theater during and after the 1990–1991 Gulf War is complicated by the differences in demographics, deployment
characteristics, and exposures experienced by deployed versus nondeployed service members. For example, a majority of participants in the 1990–1991 Gulf War operations had a single deployment that lasted less than 1 year. In contrast, those deployed to the post-9/11 operations (including operations Enduring Freedom, Iraqi Freedom, or New Dawn) often deployed more than once and had cumulative lengths of deployment averaging around 7–12 months for the Air Force, 9–10 months for the Navy, and 13–18 months for the Army (Wenger et al., 2018). According to the Congressional Research Service, in 2019 there were 7,200 U.S. troops in Afghanistan and 15,000 for the Combined Joint Task Force Operation Inherent Resolve (combating ISIS in Iraq and Syria; CRS, 2019). Methods exist to statistically adjust for such differences, but they are of limited effectiveness when the population differences are extensive and overlap is minimal (Imbens and Rubin, 2015).
AH&OBP Registry Assessment
There is no readily apparent comparison group for the AH&OBP Registry participants, although several potential comparisons could be considered, such as lower versus higher levels of exposure, or exposed versus unexposed populations including the U.S. civilian population, the Millennium Cohort Study participants, nondeployed service members and veterans, and VA health care users.
However, the committee finds that none of these groups would provide satisfactory comparisons because none of these populations have been asked the same exposure or health outcome questions nor has information been collected on them that is equal to that in the AH&OBP Registry.
Examples of limitations for select comparison groups are summarized in Table 5-1 (see Chapter 2 in NASEM, 2017 for a full discussion of limitations of selecting a comparison group for individuals who voluntarily participate in a registry).
National health surveys—such as the Behavioral Risk Factor Surveillance System (BRFSS), the National Health Interview Survey (NHIS), and National Health and Nutrition Examination Survey (NHANES)—conducted by the Centers for Disease Control and Prevention to collect information about the health and wellness of the U.S. population are primarily used for population health surveillance (see Chapter 6). The surveys are designed to be nationally representative and to collect a wide range of information about demographics, risk factors, health, and functional status. Veterans are included in those surveys as a segment of the general population, but veterans with the exposures of interest to the AH&OBP Registry are only a small subset of the entire veteran population. Each national survey asks about military service in a different way. The number of service members and veterans included in the surveys relative to those who have never served in the U.S. Armed Forces is relatively small and nonspecific; veterans of all services, components, and eras are generally grouped together. None of the surveys
|U.S. civilian population (NHANES, BRFSS, NHIS)||Population surveys include small numbers of veterans and/or service members. People in the military are general more fit and healthier than the general population. Different survey modes were used (Internet, phone, or in-person).|
|Millennium Cohort Study||Limited to post-9/11 service members and veterans. Different eligibility criteria than AH&OBP Registry. The limited exposure information collected has little overlap with AH&OBP Registry questions.|
|Nondeployed service members and veterans||People who deploy to combat zones differ from those who do not.|
|VA health care users||VA health care users are not representative of the entire eligible population. Little to no information on exposures is contained in health records.|
NOTE: NHANES = National Health and Nutrition Examination Survey; BRFSS = Behavioral Risk Factor Surveillance System; NHIS = National Health Interview Survey.
SOURCE: NASEM, 2017, p. 36.
ask about discharge from service, deployment, military exposures, or era of service. Because none of the surveys collect data about military exposures or proxies, such as deployment location, they cannot examine exposure–health outcome relationships. An internal comparison (exposed versus unexposed registry participants) also would not be informative for etiologic research given the biased population and the flawed questionnaire.
It may be tempting to use the AH&OBP Registry as a means to determine disease incidence or prevalence among participants and to determine whether the frequency of a disease based on registry data is different from what would be expected in a population that is otherwise similar but was not exposed. However, for the reasons previously discussed, it is not possible to confidently draw conclusions regarding disease incidence or prevalence from the registry because the data are from participants who have selectively chosen to participate and are not representative of the eligible population, which results in frequencies or estimates of adverse outcomes among participants that are not generalizable to the eligible population or other comparison populations.
Exposure Assessment of Adequate Quality
Individuals are often exposed to many different environmental agents or mixtures of agents (for example, complex air pollution mixtures, occupational exposures, heat waves, water pollution), which may have synergistic or antagonistic effects, adding to the difficulty of isolating and disentangling the effects of a single exposure in the causal pathway of a health outcome. The health effects of many environmental hazards are unknown, and many hazards may cause similar health effects (e.g., many airborne hazards may cause respiratory effects). Furthermore, there may be long latencies between an exposure and a health effect, making it difficult to attribute the effect to a specific exposure. Environmental exposures result from specific events such as oil-well fires.
An exposure’s composition, magnitude, duration, frequency, and route may vary over time, and these factors affect health assessments. For example, not all burn pits emit the same airborne hazards, and the same burn pit does not emit the same hazards every day or even every hour. It is also important to distinguish between acute and chronic exposure. Some respiratory conditions (e.g., chronic obstructive pulmonary disease) are thought or known to be caused by chronic (i.e., persistent or cumulative) exposures, whereas other respiratory symptoms (e.g., coughing, wheezing) are thought or known to be caused by a peak or acute exposure; these categories are not mutually exclusive, as some health conditions may be affected by both acute and chronic exposures. Therefore, although it may be challenging, an exposure assessment should, to the extent possible, measure both an individual’s cumulative exposure over a specified time (such as entire deployment for service members) and the highest level of exposure the individual experienced, even if the highest level was for short periods of time. If accurate data on cumulative and peak exposures are not available, it should be noted as a limitation of the investigation, but such information is not a requirement for an exposure assessment in etiological research.
Numerous health studies of veterans of different eras have been conducted, but most have been hampered by relatively poor measures of exposure and other methodologic problems. This has often been the case with research on exposure to open burn pits, which has numerous limitations including the use of single-time environmental sampling rather than sampling over the entire time of exposure; the burned materials having unknown compositions, which can vary dramatically; environmental air monitors that do not cover the full range of chemicals; and other methodologic concerns, such as an inability to fully account for the contributions of other sources of airborne hazards (IOM, 2011; NRC, 2010). As is often the case when reliable and accurate exposure information is not available for military populations, deployment to a particular area—which may be as nonspecific as a particular country or group of countries (e.g., Southwest Asia theater)—is sometimes used as a proxy for exposure. Using proxies such as deployed versus nondeployed ignores the heterogeneity of the actual exposures and thus weakens the strength of any associations and causal estimates.
Whenever individuals’ exposures are measured or estimated, there is the possibility of misclassification of exposure, which can lead to incorrect effect estimates of the health outcomes resulting from those exposures. To illustrate, people who experience an adverse health outcome may be more likely to recall their current and past exposures than people who are not experiencing a health outcome. Using only self-reports of exposure may lead to an overestimation of those exposures, particularly if there is reason to believe that the exposure is associated
with certain adverse health outcomes or if there are other incentives for reporting. This is an example of differential misclassification of exposure. Nondifferential misclassification occurs when the degree of misclassification is similar for all exposure groups and outcomes. An example would be a situation in which all study participants have similar difficulties completing questionnaires or remembering past exposures. Nondifferential misclassification of exposure tends to bias the effect estimates toward the null (i.e., attenuating the strength of an association between an exposure and outcome). Various information sources may also have important differences in the completeness and accuracy of their exposure data. Obtaining exposure data from more than one source or verifying data by examining preexisting records (e.g., pre-deployment health assessments) may help reduce the misclassification of exposures. If studies of deployment exposures to airborne hazards are to make meaningful contributions to health effects research, there should be documented and measured exposures that verify that an individual was in the location of concern; failing that degree of certainty, self-reports on carefully designed questionnaires may be adequate. Even these methods are fallible, but in most cases they may be of sufficient quality to be considered as contributory evidence.
The development and quality of the exposure questions for the registry are discussed in Chapter 3: AH&OBP Registry Development and Operations, and they were detailed in the initial assessment report (NASEM, 2017). Although open burn pits are not the only exposure of interest for the registry, VA has highlighted exposure to burn pit emissions as its primary interest. It is therefore not surprising that 91% (96% in the initial assessment) of registry participants in the dataset available for committee review reported being exposed to a burn pit on at least one of their deployments. Similarly, responses to other registry questions related to airborne hazards exposures—e.g., dust storms, symptoms attributed to poor air quality, and smoke or fumes from a burn pit entering housing or work locations—also had little variability and were reported by at least 80% of participants (see Questions with Limited Variability in Responses in Chapter 3). When there is little or no variability in responses, the usefulness of these responses is limited in any analyses of exposure–health outcome relationships.
After verifying or entering eligible deployment segments, the questionnaire then asks up to nine questions about specific exposures for each eligible deployment segment (1.2 Location Specific Deployment Exposures). As shown in Table 2-4, about 36% of post-9/11 registry participants in the committee’s dataset had 5 or more deployment segments, and 10.5% had 10 or more deployment segments. Therefore, for some individuals with several deployments and eligible deployment segments, participation in the registry can be a very time-consuming process, which may contribute to them not completing all of the exposure questions for each eligible deployment segment, to biased responses (reporting the same high levels of exposure for all deployments, for example), or to drop out at that or later stages, which introduces additional forms of selection or nonresponse bias (see above). The questionnaire presents eligible deployment segments in order from oldest to most recent, so that there may be up to 30 years between a deployment segment—for instance, to the 1990–1991 Gulf War—and participation in the AH&OBP Registry. Even for veterans with less time between their last deployment and participation in the registry, several years may have passed, making recall of specific exposures and their frequencies difficult (Table 5-2).
The exposure questions are further limited for use in exposure assessment by the lack of definitions of exposure or the poor descriptions of exposure in the questionnaire. For example, the questionnaire does not distinguish between the large military burn pits and other trash burning activities (such as burning trash in large drums), possibly resulting in exposure misclassification. No information was collected on the size of any burn pit, whether incinerators were in use, or other proxies for the levels and types of individual exposure, such as intensity, smoldering versus flame exposure, the proximity of work or housing locations to the burn pit, and the like. Those with higher exposures, or perceived higher exposures, may remember the event as happening more recently, occurring at a higher level, occurring more frequently, or taking place over a longer time frame. Therefore, information about both typical and peak exposures is important to collect.
Information on several relevant airborne hazards exposures to military operations taking place in Southwest Asia was not collected for each eligible deployment (such as exposure to dust and sand, for example). Questions
|Number of Years to Questionnaire Completion||Early Participants
(n = 46,444)
(n = 232,201)
|From Start Date of First Deployment||From End Date of Last Deployment||From Start Date of First Deployment||From End Date of Last Deployment|
|Less than 1 year after||0.5%||5.7%||2.3%||8.2%|
|1– <2 years after||1.8%||8.1%||2.1%||5.3%|
|2– <5 years after||11.4%||30.3%||6.4%||16.9%|
|5– <10 years after||39.1%||33.5%||21.6%||34.8%|
|10– <15years after||33.3%||8.6%||38.2%||22.6%|
|15– <20 years after||1.3%||0.2%||19.9%||5.4%|
|20– <25 years after||12.6%||7.7%||1.9%||0.9%|
|25– <30 years after||0.1%||<0.1%||5.9%||3.7%|
|30+ years after||0.0%||0.0%||1.8%||0.8%|
NOTES: 20,691 participants with missing questionnaire completion date were excluded. Prior means that the questionnaire was completed prior to the end date of the last deployment. While this should not be possible, it is what is reflected in the data.
about general military occupational exposures (section 1.3; seven questions), environmental exposures, and regional air pollution (section 1.4; six questions) do not differentiate between individual deployment segments and include exposures that appear to be outside the purview of the registry (for example, the number of days a month a person performed pesticide duties). Grouping military occupational exposures, environmental exposures, and regional air pollution is a shortcoming of the questionnaire because the exposures experienced during one deployment are not necessarily similar to the exposures experienced during other deployments and grouping exposures together makes it more difficult to estimate exposure levels or duration. To elucidate information that may potentially be used for etiologic research, well-designed questions would need to collect that level of exposure detail.
Finally, given that the AH&OBP Registry is intended to be a military exposure-based registry, several important exposures are missing. In survey design there is an inherent tension between the brevity or conciseness of the survey instrument and the completeness of the information it collects, with the result that one or the other may be compromised. One of the issues with the AH&OBP Registry questionnaire is that it is lengthy, but its length does not yield detailed information on the extent or types of deployment exposures. For example, the questionnaire does not ask about exposure to other sources of combustion products, such as burning trash and other materials, in the absence of large burn pits. Nor are there questions about high-exposure jobs other than duties that included the burn pit. Other important sources of military exposures during deployment are also missing, such as diesel exhaust, welding, paint, other chemical fumes, or organic dusts, such as those associated with wet or water-damaged indoor environments (although some questions on these exposures are asked with regard to hobbies and activities outside of military service).
Performing a granular validation of exposures is difficult under ideal conditions, let alone for combat operations. A crude or basic exposure check or concordance of registry participants might be informative. However, the committee notes that even a most basic assessment of deployment time and place is problematic because that information comes from the Data Manpower Management Center (DMDC) (funneled through the Veterans Affairs/Department of Defense Identity Repository [VADIR]), which is supposed to be the gold standard for that information, and nearly 20% of post-9/11 deployment segments are not verified and must be manually added by the registry participant. VA told the committee that manually added segments are not reported back to DMDC or updated because DMDC has determined that location information must be provided by each of the services (VA, 2022c). For those verified and manually entered segments, VA or DoD could perform a check to determine if the segment corresponds to locations and dates that burn pits were known to be operating and whether an individual
noted exposure, and vice versa (that is, locations and dates without known burn pits should not be reported as an exposure by an individual), which could provide a crude measure of confidence in the exposure information collected by the AH&OBP Registry, such as the analysis performed in Chapter 2, Self-Reported Exposure Concordance. However, this would be a very basic analysis of concordance and would not provide information on average or cumulative levels of exposures or factors that might affect exposure levels, such as the nature and magnitude of what was burned, which likely changed from day to day, seasonally, and even yearly depending on the missions of the operations. The committee is aware of only one assessment (Liu et al., 2016), summarized earlier in this chapter, that attempted to crudely validate the self-reported exposure information provided by the AH&OBP Registry participants. Although some environmental monitoring was conducted by DoD at a few locations with operating burn pits and for limited times, several flaws have been noted regarding the sampling methods and the chemical components measured (IOM, 2011). As such, using these monitoring data would result in an incomplete and flawed analysis that would be of limited use for validation-type analyses with self-reported exposure information from the AH&OBP Registry.
If a purpose of the AH&OBP Registry is hypothesis generation related to exposures to airborne hazards and health concerns, identifying those with gradations of exposure, including those likely to have been among the most highly exposed, would be important. Although there are a number of ways that these persons may be identified, such as through additional linkages with DoD records of deployment locations, number of deployments, length of deployments, and, potentially, military occupation specialty, such targeted efforts would require compatibility between the AH&OBP Registry’s architecture and additional sources and databases.
The committee finds that the exposure questions are not useful for etiologic research that would make meaningful contributions to the understanding of airborne exposures and health outcome relationships.
Health Outcome Assessment of Adequate Quality
How health outcomes are collected or measured is an important consideration when examining causality between environmental exposures and health outcomes. Higher confidence is given to relationships that rely on outcomes that are objectively measured, tested, or collected by a medical professional using a standardized examination or technique than to outcomes based on self-report. For example, electrocardiograms are objective biologic indicators that can be used to diagnose certain cardiovascular conditions, but more subjective health outcomes and symptoms, such as pain or headache, can only be identified through self-reports, although standardized scales may be used to assess their severity. For some common health conditions, such as allergies, a formal diagnosis is not always pursued. Self-reported measures are often validated by comparisons with medical records to increase the confidence in the self-reported information. Changes in diagnostic or measurement criteria can have important implications for findings of association as well.
Similar to exposure misclassification, health outcome misclassification occurs when individuals are placed into an incorrect category with respect to the outcome of interest. If the misclassification occurs differently for people with and without an exposure of interest, it is said to be differential misclassification, which may lead to an association between an exposure and an outcome being either exaggerated or underestimated. In nondifferential outcome misclassification, the misclassification is not related to exposure status, and the effect estimates tend to underestimate the true effect. How the health outcomes (and exposures) were assessed affects confidence in the findings of the strength of those effect estimates.
The quality of the health condition questions in section 2: Symptoms and Medical History of the AH&OBP Registry questionnaire is discussed in Chapter 3 and detailed in the initial assessment report (NASEM, 2017). The assessment and diagnosis of the included conditions are dependent on different criteria, measures, and tests, with some being more objective than others; there are no questions about the onset or severity of the condition following individual deployment segments. The initial assessment report provided several examples of the many questions that are overly general, cannot be answered accurately, or that appear to have little relevance for health
outcomes research. The committee is not aware of any changes that have been made to the health outcome questions despite several recommendations from the initial assessment to do so.
The outcomes covered by the health conditions questions range from nonspecific symptoms (such as fatigue, hay fever, and allergies) to very specific diagnoses or technical terms (idiopathic pulmonary fibrosis, constrictive bronchiolitis, and angina pectoris). Only a select few respiratory and cardiovascular outcomes are addressed in the questionnaire. If a respondent indicates “other lung condition,” no additional information is collected, nor is the participant able to write in any other conditions. As a proxy for diagnosis, all questions eliciting specific diagnoses begin with “Have you ever been told by a doctor or other health care professional that you had . . . ?” While responses to such questions can be useful in environmental health research, they are not substitutes for diagnoses based on a clinical examination, which the questionnaire does not include. Other important respiratory conditions that are not included in the questionnaire are reduced lung function, eosinophilic pneumonia, other lung infections (such as tuberculosis), lung scarring or fibrosis (a more inclusive diagnosis than idiopathic pulmonary fibrosis), bronchiolitis other than constrictive bronchiolitis (respiratory or obliterative), sarcoidosis/hypersensitivity pneumonitis, sinusitis, rhinitis, and vocal cord dysfunction. Questions on functional limitations, symptoms, or health conditions do not capture the specificity, time of onset, severity, or other factors that would be necessary to estimate and draw conclusions about the presence or absence of a specific diagnosis or functional limitation in the registry population following individual deployment segments.
Question 2.2.1.I is the only question in the respiratory conditions section that asks about the timing of diagnosis. When participants indicate at least one respiratory condition from the group of such conditions (asthma, emphysema, chronic bronchitis, COPD, or some other lung disease), they are asked whether they had the condition before, during, or after deployment. Because participants may select multiple responses for both the condition and the timing, the timing of each diagnosis is unclear. Question 2.2.2.F is the corresponding version of this question for selected cardiovascular outcomes, as is Question 2.2.3.F for “neurological or immune problems, chronic multisymptom illness, or liver condition.” For persons who indicated that the onset of the respiratory condition(s) was before deployment, a follow-up question (2.2.1.J) asks whether the lung disease got better, worse, or about the same during deployment. This question is problematic because it assumes a respondent has a single lung disease and does not indicate whether any change in disease status was determined by a clinician or is self-report only. Additionally, because many of the respondents have had multiple deployment segments, specifying before, during, or after deployment does not clarify the temporality of condition onset. There are wording problems with many questions that make it problematic to use the questionnaire responses in etiologic research.
There continues to be substantial scientific uncertainty about the health conditions that may result from service members’ exposure to airborne hazards in the Southwest Asia theater, and some studies have shown that organs and organ systems other than the respiratory system can be affected (IOM, 2011). A questionnaire focused primarily on respiratory and cardiovascular outcomes is not sufficient for hypothesis generation, let alone etiologic research, as it will not capture health conditions or effects in other organ systems (e.g., diabetes). Single questions that ask whether a respondent had “problems” or “conditions” related to a broad group of health outcomes or systems (e.g., immune system or liver condition) are not useful for evaluating associations between exposures and these outcomes.
Although the ability to follow participants prospectively or allow them to update health information is an important (though not always necessary) aspect of etiologic research, at present AH&OBP Registry participants cannot update any of their information after submitting their responses. A prospective follow-up of participants and updated health information could yield insights into early-onset rare diseases, such as constrictive bronchiolitis or idiopathic pulmonary fibrosis, which may be related to specific exposures, and long latency diseases, such as cancer. Prospective follow-up could be accomplished by having individuals periodically complete a questionnaire module to update their information; VA could inform participants of when to expect such updates. Other, more passive methods could be used to collect information that would not rely on proactive contributions from participants—and be subject to additional potential sources of selection and information biases. Those methods might include linking to longitudinal data sources, such as administrative health care records for VHA users.
However, the committee finds that such linkages or use of active methods to solicit updates to participants’ exposure or health status would not overcome the other limitations of the AH&OBP Registry in its current form or enable it to be used for etiologic research.
To use registry data for assessing etiologic relationships between exposures and health outcomes, it is critical to know with certainty who is and is not exposed or who has the health outcomes under investigation. Self-reported diagnoses that cover a select set of outcomes, especially those that are not common, and that are not validated in a more objective manner do not provide this certainty. The committee examined the frequency of 12 questionnaire health outcomes for registry participants who use VA health care, and found that many of the self-reported health outcomes were not found in the participant’s VA health record (Table 5-3). The range of true positives (sensitivity) varied from a high of 87.8% for hypertension to a low of 13.2% for idiopathic pulmonary fibrosis, whereas the range of true negatives (specificity) was narrower and the values much higher for all health conditions, except for allergies and hypertension. Thus, measurement error in self-reported health outcome information is systematic—registry participants are more likely to report having a health condition which does not appear in their VA medical record resulting in an overestimate of all health conditions (except for allergies and hypertension) when based on self-reports relative to an estimate based on medical records.
The committee’s analyses are consistent with previous comparisons of the AH&OBP Registry data with medical records. For example, two abstracts described the degree of agreement among respiratory symptoms and conditions self-reported in the AH&OBP Registry, documented in VHA medical records, and reported on the clinician template of the AH&OBP Registry optional health evaluation. The agreement between respiratory symptoms and conditions was poor for each comparison (Jani et al., 2019a; Molina et al., 2019). These results show that health outcomes self-reported in the registry are not useful for etiologic research, and this lack of agreement is exacerbated by having information from only a self-selected subset of potentially exposed individuals. Even if the health data
|Self-Reported Health Condition
(n = 42,640)
|ICD-10 positive||ICD-10 negative||Sensitivity (ICD-10 as gold standard)||Specificity (ICD-10 as gold standard)|
|Questionnaire positive||Questionnaire negative||Questionnaire positive||Questionnaire negative|
|Idiopathic pulmonary fibrosis||20||131||86||37,372||13.2||99.8|
|Coronary artery disease||432||654||673||38,357||39.8||98.3|
NOTE: Responses of don’t know, do not wish to answer, and missing were excluded.
could be validated for all registry participants, given the other data issues previously discussed, including selection bias, lack of representativeness, and lack of an adequate comparison group, the use of AH&OBP Registry data for etiologic research would remain problematic and unadvisable.
Identification of Other Relevant and Contributing Factors
An important aspect of evaluating the quality of individual studies that are based on registry data is whether the results reflect a true association rather than confounding bias. Confounding could occur, for example, if exposure to a source, such as an open burn pit, is associated with personal or situational attributes that may also be associated with the health outcome under study. Therefore, a key component of the study design should be to identify information on relevant factors that may contribute to the observed effect. For example, perhaps individuals who are assigned to working near a burn pit are more likely to have other risk factors for asthma, such as prior work with other known respiratory irritants. If the investigators do not account for the confounder of prior exposure, then the results (estimated effects) of the study may suggest that working near burn pits is associated with developing asthma compared with not working near a burn pit when the actual association is between asthma and the prior exposure. If, in this example, individuals with a history of working with the prior respiratory irritant were excluded from the comparison group (i.e., those not working near a burn pit), then the potential for confounding by the prior exposure would be removed.
Service-related characteristics, other than exposure to burn pits, may act as confounders when assessing the association between airborne exposures and health outcomes. These confounders include deployment outside of the United States and, specifically, deployment to a combat area, both of which can exert strong effects on the risk for adverse health outcomes. However, a well-conducted study would adjust for the presence of known confounders in the statistical analysis. The use of a suitable comparison group (described earlier in this chapter) will also help mitigate the effect of unmeasured or unobserved confounders. Ideally, a high-quality etiologic research study will make statistical adjustments to control for confounding by measuring all the characteristics that may differ between the exposed and comparison group and that may influence both the exposure and the outcome.
Effect modification is also an important consideration in epidemiologic studies. Effect modification occurs when the strength of the association between an exposure and a health outcome differs depending on the presence of another variable or variables. Consequently, the magnitude of the association may vary across studies, based on the level or presence of such variables. A common solution to addressing effect modification is to examine the association separately for each level of a third variable (e.g., examine the association between exposure to a burn pit and asthma for each service branch). While helpful, this solution depends on whether data on the other variables are collected (e.g., whether all service branches were identified), and whether the statistical power of the study (e.g., how many individuals were in each service branch) is sufficient to address the effect modifier. Many factors may contribute to effect modification, such as number of deployments to the Southwest Asia theater, the presence of other health conditions that are not exclusionary for deployment, and concurrent stressors.
The registry contains several questions on some of the most common behavioral and environmental factors associated with respiratory conditions, including smoking habits, nonmilitary occupation(s), hobbies, and prior residences. However, as the initial assessment committee observed, the questionnaire contains a substantial number of questions on nonmilitary, nondeployment factors, which are of little relevance to the registry’s stated purpose—exploring possible associations between deployment exposures to airborne hazards and health outcomes in service members and veterans—and should be removed (VA, 2021a). This recommendation is reviewed later in this chapter under “Initial Assessment Recommendations.” While critical disease influences such as smoking need to be considered, there is little basis for queries about potential or subtle exposures associated with other jobs, environments, and lifestyle factors. The registry’s purpose is not served by the poorly worded, often vague, and nonspecific questions about nondeployment-related factors. Moreover, it is not clear how such information
would be used for any analysis and what assumptions, if any, about exposure risk can be made by asking about broad domains such as the participant’s primary residence before age 13 years.
A number of other factors (not all of which are confounders) are important to obtain more details about if AH&OBP Registry data are to be appropriately used for etiologic research. These factors include more detailed information on smoking habits and products and specific military occupational exposures. Smoking and tobacco use are important confounders for etiologic research, especially when considering respiratory and cardiovascular outcomes. The questionnaire includes five questions about cigarette smoking and four questions about the use of other tobacco products. Among participants in the committee’s dataset, 36.4% reported being ever smokers and 9.3% were current smokers (reported smoking some days or every day). Although environmental tobacco smoke exposure, more commonly termed secondhand smoke, may contribute to disease development, it is much more difficult to quantify, and the generality of the question about it (2.5.J) does not lend itself to use in etiologic models of exposure–health outcome relationships. Two questions specific to smoking while deployed (2.7.A and 2.7.B) are separated from the more general smoking questions. Other than asking if there was a change to the amount one smoked during deployment, the questionnaire does not ask about what was being smoked (cigarettes, cigarillos, or cigars)—or chewed or vaped—and does not quantify the number of cigarettes or packs smoked per day. Smoking behaviors have been shown to increase during deployment (Nieh et al., 2021; B. Smith et al., 2008; Talcott et al., 2013). Of the 88,714 ever smokers in the committee’s dataset who were eligible to answer the two deployment-related smoking questions, 15,591 (17.6%) reported that they started smoking while deployed, and 35,092 (40.6%) participants reported that they smoked more while deployed. Smoking would be a confounder if, for example, smokers are more likely to have burn pit duties or if they spend more time outside because of their tobacco use as they would also be more likely to be exposed to airborne hazards on the whole.
The AH&OBP Registry does not have several characteristics that are necessary for an exposure registry to be used for etiologic research, notably a representative sample of the eligible population, an appropriate comparison group, an adequate quality of exposure assessment, and an adequate quality of health outcome assessment. It does have a sufficient sample size for precise estimates (although the estimated effects computed from the registry data are precise around a likely biased causal effect) and it includes some information on other relevant and contributing factors (although this is not complete).
As the committee finds that the AH&OBP Registry does not exhibit several of the characteristics that are necessary for an exposure registry to be used for etiologic research, it therefore concludes that AH&OBP Registry data are not appropriate for etiologic research. The registry has major design and data quality issues that cannot be overcome. Even substantial changes to the questionnaire would be insufficient to make the AH&OBP Registry appropriate for etiologic research of airborne hazards exposures and health outcomes.
Given that the AH&OBP Registry data are not of sufficient quality to support etiologic research on airborne hazards exposures and health outcomes, the committee considered whether other currently available data sources might be better suited to perform or support etiologic research on the same eligible population. The question is how well each alternative data source fulfills the six characteristics (described in the previous section) that determine its potential for use in etiologic research: a sufficient sample size for precise estimates, a sample that is representative of the eligible population, identification of an appropriate comparison group, exposure assessment of adequate quality, health outcome assessment of adequate quality, and identification of other relevant and contributing factors. The alternative sources to the AH&OBP Registry under consideration are: the Individual Longitudinal Exposure Record (ILER), the Millennium Cohort Study, the Gulf War Registry, previous epidemiologic studies of this population, and the development and design of new epidemiologic studies. Because the Statement of Task
specifically named ILER and the Millennium Cohort Study as other means or methods that should be considered for burn pit exposures research, these are presented first. The Gulf War Registry is then considered because it has many similarities to the AH&OBP Registry. Previous and ongoing epidemiologic studies are considered last and as a group since none of them were designed with exactly the same purpose as the AH&OBP Registry and ongoing studies are not yet available for a detailed assessment.
Individual Longitudinal Exposure Record
One of the goals of ILER is to provide researchers with greater access to hazard and exposure events information and to more efficiently develop research cohorts by linking exposure information based on location data with health outcomes from summaries of electronic health records (see Chapter 4 for more details on ILER). Research using these ILER-generated cohorts may inform clinical care or identify areas where further investigation is needed. The suitability of ILER for etiologic research depends on the extent to which it becomes part of a research platform that can fulfill the six characteristics presented. Such research, however, is only speculative as ILER is not as yet totally functional. Before ILER can be used for research, validation studies comparing ILER exposure extrapolations with other exposure indicators of known accuracy will be most informative to ensure that ILER accurately compiles and presents exposure information adequately at the individual and group levels. Potential research applications of ILER would include a range of exposures of concern that occur during deployment, but the committee’s focus is on its application to address the health consequences of exposure to open burn pits and other airborne hazards. Because ILER compiles data that can also have errors, the accuracy and completeness of ILER compilations is unknown and is a further reason that validation studies are necessary.
The committee finds that, when fully operational with linkages to DoD and VA health care data, ILER may have many of the characteristics necessary for prospective etiologic research to address some health concerns—a sufficient sample size for precise estimates, be representative of the eligible population, allow the identification of appropriate comparison groups, an exposure assessment of adequate quality, a health outcome assessment of adequate quality, and identification of other relevant and contributing factors.
Depending on the research question, a representative sample of the eligible population may not be feasible since DoD health record information is available only during active service and VA health record information is only for a subset of VA health care users, respectively. In addition, most exposure records in ILER are available only beginning in 2006 and, therefore, exclude exposure information from earlier times, including from the 1990–1991 Gulf War and the first few years of post-9/11 operations. Most airborne hazard and open burn pit exposures that occurred during that period (1990–2005) would not be included.
Based on the available information about ILER, which included presentations from DoD and VA representatives, a live demonstration of ILER, and several responses to information requests from the committee, ILER in its current form has not been shown to fulfill the six characteristics necessary for it to be used for etiologic research. ILER compiles information from many sources which may be incomplete, depending on the time period of interest. As such, ILER provides only a limited capability for determining exposures at the level of individual service members or veterans, only making estimates about aggregate exposure based on where they were deployed. Location information (from DMDC) is available in ILER from 2001 forward, so that ILER does not completely cover the same eligible population as the AH&OBP Registry (i.e., 1990–1991 Gulf War and Peacetime eras are excluded). As new capabilities are added and validated exposure and health information becomes available, ILER could become a complementary tool for VA and DoD efforts to understand the health effects of military exposures, but at this time it should not be viewed as an alternative source for conducting etiologic research to determine associations between exposures encountered during military service and health outcomes. Neither does ILER make it possible to conduct retrospective exposure assessments for individual service members or veterans. The details of ILER are presented in Chapter 4; its attributes for use in etiologic research are summarized here.
Sufficient Sample Size for Precise Estimates
ILER contains deployment location data for approximately 10.3 million service members and veterans, and, of those, more than 3 million individuals have at least one exposure-related report associated with at least one of their deployments (DoD, 2021).4 The committee presumes that the 3 million individuals include most, if not all, of the more than 2.77 million service members and veterans of all components (active duty, reserve, and National Guard) and service branches who have deployed after September 2001 in support of post-9/11 operations and continuing missions to Southwest Asia (Wenger et al., 2018). This satisfies the characteristic of sufficient sample size for precise estimates for exposure–health outcome relationships depending on the exposure and time period of interest.
Sample Representative of the Eligible Population
ILER’s source data do not include all service members and veterans, particularly not all of those who are eligible for the AH&OBP Registry. Location information (compiled from DMDC) is not available in ILER before 2001, which excludes both the 1990–1991 Gulf War and the Peacetime (1992–2000) era veterans. In addition, an evaluation of AH&OBP Registry data shows that over 20% of post-9/11 deployments were user-entered, particularly deployments to Iraq and Afghanistan (see Table 2-6). These registry data indicate that a potential limitation of ILER may be the accuracy of the DMDC deployment or location information, as veterans who served prior to 2001 or in some locations may not have DoD administrative records that accurately reflect their actual service, which would lead to errors in the assignment of exposure. Therefore, ILER is representative of the eligible population of those who served after 2000, but not of the population of those who served before then. Moreover, health care information is available only for those individuals who use DoD health care or who are eligible for and use the VA health care system. Therefore, the representativeness of the eligible population of service members and veterans is restricted to those who use either VA or DoD health care systems and may not be representative of the full eligible veteran population.
Appropriate Comparison Group
Because ILER is intended to have the capability to compile both deployment-related and in-garrison (United States and abroad) locations and general exposures as well as more granular exposure data, the ability to passively identify exposed and unexposed service members and veterans in ILER means that appropriate comparison groups could be identified in order to conduct more precise analyses of health effects and specific exposures. The size and composition of these comparison groups will depend on the exposures that are being assessed and the health outcomes that are available in the VA and DoD health records and group characteristics that may be of interest, such as sex, age, and service branch or component.
Exposure Assessment of Adequate Quality
Through data sharing and accessibility, a fully functional ILER may, in the future, offer an incremental improvement to currently available deployment exposure information depending on the exposure of interest. ILER has the potential to improve location-based exposure assessments for future service members and veterans, and to a lesser degree it may also benefit service members and veterans who have served post-2012 (see Table 4-2) by providing access to location data in DMDC, exposure information in the Defense Occupational and Environmental Health Readiness System–Industrial Hygiene (DOEHRS-IH), and VA and DoD clinical record information on one platform. DOEHRS-IH and Military Exposure Surveillance Library (MESL) exposure event data are available from January 2001 (January 2000 for MESL), but it is not clear how detailed the exposure–location relationships are or whether the exposure information for an individual’s total military service is complete. The available event
4 Personal communication, Larry Vandergrift, Defense Health Agency, chief information officer/deputy assistant director, Information Operations. July 13, 2021.
reports do not include in-garrison exposures at this time.5 ILER may be particularly useful for identifying or validating potential exposures by location and date for post-9/11 deployments (see the committee’s analysis on self-reported exposure concordance in Chapter 2). For example, ILER could confirm that a burn pit was operational on a specific base at a particular time; however, whether ILER could identify specific exposures at that time and location is unclear (e.g., burn pit operations versus improvised explosive device encounters versus frequency of dust storms versus particulate matter in total). Location and exposure information is also incomplete for classified deployments and may also be limited for certain groups, such as special operations forces. These information gaps affect ILER data completeness and quality and can also influence exposure assessments for some individuals and the representativeness of the population of exposed individuals.
The committee finds that ILER may be used to identify where there is a need to improve the documentation of exposures encountered during military service, but it is unclear how the quality of this information compares to self-reported exposures.
Health Outcome Assessment of Adequate Quality
Although several sources of health information are or will be available in ILER, the earliest available data on health outcomes from DoD sources are from October 2012; VA electronic health record information earlier than 2001 is considered to be unreliable, and the earliest dates for VA health information in ILER are unclear. Pre- and post-deployment health assessments are also only available beginning in 2012 (after most open burn pits had ceased operation). ILER is expected to be able to compile more information on health conditions and health diagnoses in 2022.6
DoD and VA administrative health systems and databases were not designed for research purposes, and ILER will not compile an individual’s complete VA or DoD electronic health record. Rather the health information from these sources is limited to clinical encounter summaries (dates, type of encounter, ICD-9 and ICD-10 codes, and disposition). These summaries may be useful for validating some self-reported health outcomes or to identify new-onset conditions in individuals seen at a DoD or VA health care facility during or after 2012. For symptoms and some health outcomes identified on the AH&OBP Registry questionnaire that do not correspond to a single ICD code, there would be no clinical encounter summary in ILER. While there are well-established algorithms for using VA clinical data for research purposes, a broader array of information is drawn upon, and the ICD-code assignments need to be validated. Because there are no plans to incorporate Medicare, Medicaid, or private health insurance data into ILER, health information that is not part of the VA or DoD electronic health record and captured in the clinical encounter summary will be missing in ILER, further limiting its application to health assessments and its generalizability for etiologic research. Moreover, with at most 10 years of DoD clinical summary data currently available for individuals, use of ILER to examine health outcomes with longer latencies, including cancer, may not be possible. The quality of health outcome information is limited by the availability of details contained in clinical encounters and the time period for which the clinical encounter information is available.
Identification of Other Relevant and Contributing Information
ILER contains self-reported information from periodic and deployment-related health assessments (available from 2012 forward) on some health behaviors, such as tobacco use and alcohol consumption, that may be considered as other relevant and contributing factors for etiologic research (VA, 2021d). Basic demographics and military characteristics such as gender, age, birth month, military occupation specialty, unit, and detached unit are available in ILER via DMDC (DoD, 2021). Although some basic demographic, military, and behavioral characteristics may
5 Personal communication, Larry Vandergrift, Defense Health Agency, chief information officer/deputy assistant director, Information Operations. July 13, 2021.
6 Personal communication, Larry Vandergrift, Defense Health Agency, chief information officer/deputy assistant director, Information Operations. October 13, 2021.
be available in ILER, their quality and completeness are unknown. Therefore, ILER might satisfy this characteristic for etiologic research, but the availability of accurate covariate information has not yet been demonstrated.
Millennium Cohort Study
The Millennium Cohort Study, begun in 2001 and expected to continue until 2068, is an ongoing epidemiologic study of active-duty service members to assess the health of participants after they separate from the military. Compared with many other studies of post-9/11 veterans, it offers a unique opportunity for etiologic research because of its regular follow-up of participants through standardized surveys completed approximately every 3 years.
The committee finds that the Millennium Cohort Study fulfills the six characteristics for conducting etiologic research on the general deployment-related health effects of post-9/11 service members and veterans—its intended purpose. The committee also finds that with some modifications to the number and type of exposure questions the Millennium Cohort Study asks, it could capture the airborne hazards of interest in the AH&OBP Registry questionnaire, and it could be an improved resource for conducting etiologic research on airborne hazards exposures for post-9/11 veterans and service members.
At present, the Millennium Cohort Study samples participants while they are still on active duty and systematically oversamples under-represented groups, such as women and reservists and National Guard, to enhance statistical power while ensuring that the cohort is representative of the entire population of post-9/11 service members and veterans. Because participants are randomly sampled within strata, the study population is more representative than the AH&OBP Registry, which is based solely on self-selection. However, the Millennium Cohort Study was not designed to capture individuals who did not serve in the post-9/11 operations, and therefore the findings from the panels are not generalizable to those who served only in the 1990–1991 Gulf War or Peacetime eras. To have the same coverage as the AH&OBP Registry, the Millennium Cohort Study sampling strategy would need to be redesigned to include veterans of the 1990–1991 Gulf War and the Peacetime eras, which is outside of its intended scope. Low response rates and potential biases related to participation and loss to follow-up are additional considerations that can affect representativeness.
The Millennium Cohort Study captures post-9/11 service members and veterans with a range of deployment experiences, including subsets who have had burn pit or airborne hazards exposures as well as those who have been deployed elsewhere or not deployed at all. It also includes information on participants who served in the military before the post-9/11 operations began, thereby providing a structure for identifying appropriate comparison groups of service members and veterans for both longitudinal and cross-sectional etiologic investigations.
As with the AH&OBP Registry, the Millennium Cohort Study collects self-reported exposure and health information. The survey consists of about 450 validated questions on mental, physical, behavioral, and functional health (including many questions on other relevant and contributing factors of health) and incorporates several standardized instruments (the Patient Health Questionnaire, the Medical Outcomes Study Short Form-36 for Veterans, and the Posttraumatic Stress Disorder [PTSD] Checklist–Civilian Version) (T.C. Smith et al., 2007). However, a drawback to the survey is that it only contains nine questions on deployment exposures and only one specific question on exposure to smoke from burning trash and/or feces (Crum-Cianflone, 2013). All exposures are self-reported, and some are validated by deployment status or proximity (subject to data availability), but similar to the AH&OBP Registry, questions are not asked about the magnitude, frequency, or duration of any exposures. In a presentation to the committee about the Millennium Cohort Study, it was stated that 50% of participants reported exposure to burning trash and feces, which may have included burn pits (Rull, 2021). Separately, over 12,500 Millennium Cohort Study participants also participate in the AH&OBP Registry, but it is unclear how many of the Millennium Cohort Study participants who also participate in the AH&OBP Registry were exposed to burn pits (VHA, 2021). In order to address AH&OBP Registry exposures more effectively, the Millennium Cohort Study would need to expand the number and granularity of the exposure questions to properly assess the association between specific exposures (at a more granular level than country and date of deployment) and health outcomes.
The Millennium Cohort Study does not include a clinical or physical health examination, but self-reported health outcome data can be linked to VA and DoD data sources, including ILER. These linkages permit validation of self-reported health information and provide additional objective information on deployments, demographics, medical history, health care use, and mortality, making it a robust resource for etiologic research and surveillance activities (Rull, 2021; VHA, 2021).
Gulf War Registry
Both the Gulf War Registry and the AH&OBP Registry were established by congressional legislation with the intent of determining associations between deployment-related exposures and health outcomes. The respective laws directing the development and maintenance of both registries do not provide a sunset date, nor does it appear that VA has determined what an end state to either registry would be. VA’s Health Outcomes Military Exposures reported that several publications have used Gulf War Registry data, although most are not recent. Some of these studies are methodologic and examined factors of participation and selection bias (e.g., Gray et al., 1998; T.C. Smith et al., 2002); others focused on patterns of health care use (e.g., Miller et al., 2006), specific diagnoses (e.g., Murphy et al., 1999; T.C. Smith et al., 2004), or clusters of symptoms and diagnoses (e.g., Hallman et al., 2003; Stuart et al., 2002).
The committee finds that the Gulf War Registry satisfies a few but not all of the characteristics necessary for an exposure registry to be used in etiologic research, and therefore it is not appropriate for use in etiologic research on airborne hazards exposures and health outcomes.
It has many of the same limitations as the AH&OBP Registry, including a self-selected population that is not representative of the eligible population and use of self-reported exposure information (see Chapter 4). The Gulf War Registry has an advantage over the AH&OBP Registry in that a comprehensive health examination conducted by a VA provider is required for participation. Participants are also offered diagnostic testing and specialty referral(s) as clinically indicated. While this is an improvement over the AH&OBP Registry, the Gulf War Registry still does not have all of the characteristics to make its data appropriate for use in etiologic research.
The most significant deficiency for use of the Gulf War Registry for etiologic research is that its participants are not representative of its entire eligible population, especially as the eligible population has expanded (see Chapter 4, Table 4-1), nor is there a readily apparent appropriate comparison group (e.g., unexposed deployed group or deployed outside of the Southwest Asia theater). When eligibility for the Gulf War Registry was expanded to include those who served in OIF and Operation New Dawn (OND), a majority of the population eligible for the AH&OBP Registry also became eligible for the Gulf War Registry; those who served only in Afghanistan, Djibouti, Syria, or Uzbekistan would continue to be excluded. Based on country of deployment information for the eligible post-9/11 population (see Chapter 2, Table 2-4), 15.6% of the eligible population deployed to Afghanistan only and is excluded from participating in the Gulf War Registry. (Information was not available on the proportion of deployments to Djibouti, Syria, or Uzbekistan in the eligible population, but the total excluded proportion would be more than 15.6%.) Comparing the eligible population with the committee’s AH&OBP Registry dataset in Table 2-4, 26.1% of AH&OBP Registry participants served only in Afghanistan and would continue to be excluded from the Gulf War Registry. As of May 2022, 60,816 veterans who served post-9/11 in the Southwest Asia theater participate in the Gulf War Registry; 3,589 registry participants served in both the 1990–1991 Gulf War and OIF/OND conflicts, and over 240,000 veterans have already joined both the AH&OBP and Gulf War registries (VA, 2022d). However, it is unclear whether they are representative of the entire eligible population, and issues of selection bias and factors that determine adequate power make any estimates using the Gulf War Registry data questionable.
Epidemiologic Studies of Military and Veteran Populations
The AH&OBP Registry is not a substitute for the VA and DoD epidemiologic studies that assessed the relationship between deployment-related airborne hazards exposures and health outcomes. Many epidemiologic studies
have examined the association between health outcomes and either deployment in general or specific deployment-related exposures. Most of these that were conducted prior to 2019 have been reviewed by NASEM committees in the Gulf War and Health series or in other stand alone reports of post-9/11 service members and veterans. Several examples of such studies are summarized in Chapter 4, particularly those that were conducted by VA (and to a lesser extent DoD). Some of the health outcomes and deployment exposures collected by the AH&OBP Registry have not been the focus of epidemiologic studies conducted to date. Unlike the AH&OBP Registry, the epidemiologic studies are intentionally designed for purposes of etiologic research. If a new epidemiologic study were to be designed with the same focus as the AH&OBP Registry, it could easily incorporate the most salient exposures and collect more granularity on them, which would provide a better source for etiologic research and causal estimates than the current AH&OBP Registry, which cannot be used in any practical way for etiologic research. Furthermore, a new epidemiologic study could incorporate objective health outcome assessments, including the identification and collection of other relevant and contributing factors for exposure–health outcome relationships, such as smoking and body mass index, and effect modifiers for populations of service members and veterans might include service branch, military rank, pay grade, occupation, and number and lengths of deployments.
There are ongoing DoD- and VA-funded studies that may offer potential alternative approaches to conducting etiologic research on airborne hazards and exposures to burn pits specifically. Chapter 2 describes the process and criteria that the committee used to search for funded studies. Although these studies appear to be ongoing and results have not been published, creative or novel methods or data sources may offer new insights to inform the knowledge base or etiologic estimates of airborne hazards exposures and health outcomes.
The committee recommends that VA support the conduct of epidemiologic studies to examine the associations between exposures to airborne hazards and open burn pits and health outcomes. The studies should be designed specifically to fulfill the characteristics needed for etiologic research—a sufficient sample size for precise estimates, a sample that is representative of the eligible population, identification of an appropriate comparison group, an exposure assessment of adequate quality, a health outcome assessment of adequate quality, and identification of other relevant and contributing factors.
The sampling design could target representative samples of the deployed population at specific time points, apply novel methods for a retrospective assessment of airborne hazards exposures, and include more granularity on those deployment- or military-service exposures than the AH&OBP Registry questionnaire does. It could also include some objective means of health outcome assessment as well as methods to identify other relevant and contributing factors to use in adjusting estimates of exposure–health outcome relationships. One such design that may be considered is the three-tiered study design presented in the first National Academies’ report on the health effects of exposure to burn pits (IOM, 2011). Ongoing service member and veteran health research programs, most notably the Millennium Cohort Study, have a number of relevant strengths and could be adapted to address the exposures of concern by targeting the population of interest and incorporating more detailed exposure information regarding burn pits and airborne hazards.
The initial assessment committee offered nine recommendations for the AH&OBP Registry. Two of the recommendations pertained to etiologic research; the other seven appear in the chapters for which they are most applicable. The recommendations were not numbered, and are presented below in the same order as they were in the initial report (NASEM, 2017). Each of the initial assessment recommendations is followed by VA’s verbatim response to the recommendation and then the reassessment committee’s response to VA’s response.
Initial Assessment Recommendation: The committee recommends that VA eliminate the questionnaire sections addressing locations of previous residences (Section 4), non-military work history (5) and home environment, community, and hobbies (6), which collect data that might only be useful in epidemiologic studies of the population.
VA Response: The VA did not accept this recommendation. Sections 4, 5, and 6 are not major impediments in completion of the survey as shown by the data in the table of last section view [as cited in Participation and Completion by Questionnaire Section]. In evaluating potential lung disease, factors outside deployment should be considered. This is why smoking questions are included with the online section. In addition, prior locations before military, certain occupations such as asbestos remediation and certain hobbies could contribute to lung disease [as determined by VA]. [Table 5-4] on hobbies is presented below. It only contains hobbies that may be significant to lung disease. The majority (81%) quickly left this section by answering “none of the listed hobbies.”
Reassessment Committee Response: VA did not accept this recommendation and therefore did not adequately address it. The reassessment committee concurs with the recommendation of the initial assessment committee that the questionnaire sections that collect information on potential contributing factors and confounders, specifically sections 4 (previous residences), 5 (nonmilitary work history), and 6 (home environment, community, and hobbies) should be eliminated because they are burdensome to participants and do not contribute to the AH&OBP Registry in supporting etiologic research. For comparison to VA’s data on response to hobby questions, the committee examined responses to these questions using its own dataset (column 3 of Table 5-4). Although fairly similar proportions of participants report the listed hobbies, fewer report them in the committee’s dataset of more than 278,000.
VA’s response suggests that the fact that few people answer these questions is an indication of low respondent burden. However, an alternative view held by the committee is that a relatively minor subset of participants have these hobbies. During the questionnaire demonstration, the committee was told that residential histories (section 4) are not being used for research, but the motivation for including the question was its use as a potential confounder of respiratory disease (VA, 2021e). If the information is not being used for research purposes, then such questions are superfluous. Even if there was interest in using some of the information from sections 5 or 6 of the registry questionnaire to create a job or hobby exposure matrix, the selection bias of participants, use of unvalidated questions, and responses that are reported by few participants would not confer utility for this purpose. Not only do several of these questions have high rates of missing responses overall, but the rates of missing responses have increased—some substantially—between early and late participants (data not shown for missing stratified by early and late participation). For example, question 5.3.B.1 asks respondents to select the occupational category that best describes their job with the longest dust exposure. Among early participants eligible to answer this question, nonresponse was 1.5%, whereas among late participants the nonresponse rate was 21.8%. Long-term retrospective recall bias also affects the responses to these questionnaire sections; for example, questions on previous residence up to 13 years of age and such responses would not be useful in etiologic research models of airborne hazards exposures and respiratory outcomes. If the purpose of the questions is to examine diagnoses of lung and respiratory diseases, then these questions might be better asked as part of the registry health evaluation rather than as part of the self-assessment questionnaire.
|Hobbies Examined||Number (%)
(as of July 1, 2020)
(as of February 1, 2022)
|Woodwork||22,314 (10.7%)||30,710 (11.0%)|
|Indoor swimming/indoor ice skating||14,905 (7.1%)||19,441 (7.0%)|
|Metal work||6,794 (3.2%)||9,098 (3.3%)|
|Welding||6,328 (3.0%)||8,369 (3.0%)|
|Epoxies||4,840 (2.3%)||6,694 (2.4%)|
|Pottery||498 (0.2%)||649 (0.2%)|
|Stained glass||237 (0.1%)||308 (0.1%)|
|None of the listed hobbies||167,887 (80.3%)||224,907 (80.7%)|
SOURCES: Column 2 is VA-generated, received September 2020; column 3 is from the committee’s dataset.
Related to this initial assessment recommendation is a second recommendation by the initial assessment committee that VA clarify the intent and purpose of the registry. The initial committee also recommended developing a specific plan for more seamlessly integrating relevant VA and DoD data sources with the registry’s data with the goals of reducing future participant burden, increasing data quality by restructuring questions to minimize recall and other biases, and improving the usefulness of the registry database as an information source for health care professionals and researchers (first initial assessment recommendation in Chapter 3). The committee’s assessment of VA’s response to this recommendation appears in Chapter 3; however, although the initial committee recommended refinements to the questionnaire that would be beneficial for its use in etiologic research (as it may potentially improve exposure or health outcome assessment), that purpose cannot be met because it still will not satisfy all six committee-identified characteristics of etiologic research.
Initial Assessment Recommendation: The committee recommends that other means for evaluating the potential health effects associated with airborne hazards and open burn pit exposures be developed, such as a well-designed epidemiologic study.
VA Response: VA and DoD have ongoing epidemiological and clinical research projects to evaluate airborne exposure and disease in terms of clinical evaluation, associations, natural history of disease, and tertiary prevention. A main effort by VA to promote such research is establishment of the Airborne Hazards and Burn Pits Center of Excellence in May 2019. This is the VA epicenter for all AHOBP research. More information on this center can be found at: https://www.warrelatedillness.va.gov/WARRELATEDILLNESS/AHBPCE/index.asp.
In addition, VA partners with the NASEM. A report titled Gulf War and Health (Volume 12) Respiratory Health Effects of Airborne Hazards Exposures in the Southwest Asia Theater of Military Operations was published on 11 September 2020. It is expected that this document will help direct VA’s future research agenda on airborne hazards, burn pits, and health outcomes.
Reassessment Committee Response: VA appears to be pursuing different types of research efforts to evaluate the potential health effects associated with airborne hazards and open burn pit exposures, which the committee commends. The committee is aware of some published studies, particularly on respiratory health issues that were funded or conducted by VA or the War Related Illness and Injury Study Center and AHBPCE investigators specifically (for example, Butzko et al., 2019; Falvo et al., 2016a,b; Garshick, 2019; Jani et al., 2017a; Lindheimer et al., 2019). However, based on the published literature and other information presented on VA’s websites, including that of AHBPCE, it is not clear whether epidemiologic studies on airborne hazards and health outcomes—as opposed to clinical investigations or case series reports—are being conducted, which is a key part of the initial assessment committee’s recommendation. AHBPCE has established the AIMES program to improve the veteran experience with the AH&OBP Registry health evaluation, improve the consistency and reliability of these evaluations across VHA, and enhance scientific understanding of airborne hazards, in part, through the curation of a merged database of AH&OBP Registry and clinical data from the VA Corporate Data Warehouse (VA, 2021b; VA AHBPCE, 2021). Although these activities appear to be positive steps, they are not characteristics of etiologic research at present.
The intent of the initial assessment recommendation was to encourage an epidemiologic study that was designed specifically to overcome many of the limitations that have affected the studies on these airborne hazards exposures–health outcome associations to date. The sampling design could focus on samples of the deployed population at specific time points, apply novel methods for a retrospective assessment of airborne hazards exposures, and include more granularity on those deployment- or military-service exposures than does the AH&OBP Registry questionnaire. It could also include some objective means of health outcome assessment as well as include methods to identify the other most relevant and contributing factors for use in adjusting for estimates of exposure–health outcome relationships.
One of VA’s and Congress’s goals for the AH&OBP Registry is to support research about the health effects caused by deployment-related airborne hazards, including burn pits, but the AH&OBP registry cannot be used for etiologic research. AH&OBP Registry data appear to have only been used by internal VA researchers for primarily descriptive analyses. Furthermore, as the AH&OBP Registry does not have many characteristics that an exposure registry to be used for etiologic research would need to have, it is not capable of supporting etiologic research. This has created a chasm between the expectation that it will be used for research and the reality of what it can actually be used to do.
Therefore, the committee concludes that the AH&OBP Registry data are not appropriate for etiologic research. The design and data quality issues of the registry are major and cannot be overcome. Even substantial changes to the questionnaire would not be sufficient to make the AH&OBP Registry appropriate for etiologic research of airborne hazards exposures and health outcomes.
Because the AH&OBP Registry cannot be used to perform etiologic research, other information sources were considered that might fulfill this function. This is one component of the committee’s Statement of Task, “The report should make a recommendation on the best way to utilize resources for airborne hazards research. Does, for example, a more cost-effective way than the AH&OBP Registry exist to discover associations between airborne hazards and burn pit exposures and disease?” The committee considered ILER, the Millennium Cohort Study, the Gulf War Registry, and other previous and new epidemiologic studies of the population eligible for the AH&OBP Registry. Although the Gulf War Registry satisfies some of the needed characteristics for use in etiologic research, it does not fulfill them all and therefore is not appropriate for use in etiologic research on airborne hazards exposures and health outcomes. Each of the alternatives to the AH&OBP Registry has strengths over the registry, but none of them in their current form is an acceptable substitute in terms of the population covered or containing or replicating all of the exposures and health outcomes collected by the registry.
One of the goals of ILER is to provide researchers with greater access to hazard and exposure events information and to more efficiently develop research cohorts by linking exposure information based on location data with health outcomes from clinical summaries of electronic health records. Research using these ILER-generated cohorts may inform clinical care or identify areas where further investigation is needed. ILER’s use for etiologic research depends on the extent to which it becomes part of a research platform that has the needed characteristics. Such research, however, is only speculative at this time as ILER is not yet entirely functional, and the accuracy and completeness of ILER compilations are unknown, so further validation studies are needed. The quality of data compiled by ILER is compromised by missing values, incomplete and unreliable data, the lag time for data input, use of inappropriate sampling measures or methods for exposure data, single point-in-time measurements, lack of validation or reconciliation of disparate sources, continued viability and accessibility of many data sources, and the extent of data system interoperability. The variability and completeness in time periods covered by the different sources for location, exposure, and health information also complicates using ILER for etiologic research on the same population eligible to participate in the AH&OBP Registry. The available location data do not include deployments prior to 2000, thereby excluding individuals who served in the 1990–1991 Gulf War or the subsequent Peacetime period. Most DOEHRS-IH exposure data are only available from 2006 onward (which misses much information related to exposures to open burn pits), and there are even fewer exposure data available from the 1990–1991 Gulf War (though specific exposures such as Khamasiyah and oil-well fire smoke are available) and no exposure data at all prior to then. In ILER, electronic health record information is only available after 2012 for DoD and after 2000 for VA, and there is no health information for individuals who receive care outside of VA or DoD. Based on the available information about ILER, which included presentations from DoD and VA representatives, a live demonstration of ILER, and several responses to information requests from the committee, ILER in its current form has not been shown to fulfill the characteristics necessary for it to be used for conducing etiologic research. As such, it offers only a limited capability for determining exposures at the level of individual service members or veterans, only allowing estimates about aggregate exposure based on where they were deployed.
The committee finds that, when fully operational with linkages to DoD and VA health care data, ILER may have many of the characteristics necessary for prospective etiologic research to address some health concerns, that is—a sufficient sample size for precise estimates, be representative of the eligible population, allow the identification of appropriate comparison groups, an exposure assessment of adequate quality, a health outcome assessment of adequate quality, and identification of other relevant and contributing factors.
As new capabilities are added and validated exposure and health information becomes available, ILER could become a complementary tool for VA and DoD efforts to understand the health effects of military exposures, but at this time it should not be viewed as an alternative resource for conducting etiologic research to determine causal associations between exposures encountered during military service and health outcomes.
The Millennium Cohort Study offers a unique opportunity for etiologic research compared with many other studies of post-9/11 veterans because of its regular follow-up of participants through standardized surveys completed approximately every 3 years.
The committee finds that the Millennium Cohort Study fulfills the six characteristics for conducting etiologic research on the general deployment-related health effects of post-9/11 service members and veterans—its intended purpose. The committee also finds that with some modifications to the number and type of exposure questions the Millennium Cohort Study asks, it could capture the airborne hazards of interest in the AH&OBP Registry questionnaire, and it could be an improved resource for conducting etiologic research on airborne hazards exposures for post-9/11 veterans and service members.
In order to address AH&OBP Registry exposures more effectively, the Millennium Cohort Study would need to expand the number and granularity of the exposure questions to properly assess the association between specific exposures (at a more granular level than country and date of deployment) and health outcomes. The Millennium Cohort Study does not include a clinical or physical health examination, but the self-reported health outcome data are linked to DoD and VA health data sources, which improves the information available for etiologic research analyses.
VA and DoD epidemiologic studies have assessed relationships between deployment-related airborne hazards exposures and health outcomes. Some of the health outcomes and deployment exposures in the AH&OBP Registry have not been the focus of epidemiologic studies conducted to date. The epidemiologic studies are intentionally designed for purposes of etiologic research, unlike the AH&OBP Registry, and if a new epidemiologic study were to be designed with the same focus as the AH&OBP Registry, it could easily incorporate the most salient exposures and collect more granularity on them, which would be an improved source of etiologic research and causal estimates. Furthermore, a new epidemiologic study could incorporate objective health outcome assessments including the identification and collection of other relevant and contributing factors for exposure–health outcome relationships.
There are several ongoing DoD- and VA-funded studies that may offer potential alternative approaches to conducting etiologic research on airborne hazards and exposures to burn pits specifically. Although these few studies appear to be ongoing and their results have not been published, creative or novel methods or data sources may offer new insights to inform the knowledge base or etiologic estimates of airborne hazards exposures and health outcomes.
The committee recommends that VA support the conduct of epidemiologic studies to examine the associations between exposures to airborne hazards and open burn pits and health outcomes. The studies should be designed specifically to fulfill the characteristics needed for etiologic research—a sufficient sample size for precise estimates, a sample that is representative of the eligible population, identification of an appropriate comparison group, an exposure assessment of adequate quality, a health outcome assessment of adequate quality, and identification of other relevant and contributing factors.
The sampling design could target representative samples of the deployed population at specific time points, apply novel methods for retrospective assessment of airborne hazards exposures, and include more granularity on those deployment- or military-service exposures than the AH&OBP Registry questionnaire does. It could also include some objective means of health outcome assessment as well as methods to identify the other relevant and contributing factors for use in adjusting estimates of exposure–health outcome relationships. One such design that may be considered is the three-tiered study design presented in the first National Academies’ report on the health effects of exposure to burn pits (IOM, 2011). Ongoing service member and veteran health research programs, most notably the Millennium Cohort Study, have a number of relevant strengths and could be adapted to address the exposures of concern by targeting the population of interest and incorporating more detailed exposure information regarding burn pits and airborne hazards.
Bith-Melander, P., J. Ratliff, C. Poisson, C. Jindal, Y. M. Choi, and J. T. Efird. 2021. Slow burns: A qualitative study of burn pit and toxic exposures among military veterans serving in Afghanistan, Iraq, and throughout the Middle East. Annals of Psychiatry and Clinical Neuroscience 4(1):1042.
Brumback, B. A., M. A. Hernán, S. J. P. A. Haneuse, and J. M. Robins. 2004. Sensitivity analyses for unmeasured confounding assuming a marginal structural model for repeated measures. Statistical Medicine 23:749–767.
Butzko, R. P., A. M. Sotolongo, D. A. Helmer, J. C. Klein-Adams, O. Y. Osinubi, A. R. Berman, R. Ortiz-Pacheco, and M. J. Falvo. 2019. Forced oscillation technique in veterans with preserved spirometry and chronic respiratory symptoms. Respiratory Physiology & Neurobiology 260:8–16.
CRS (Congressional Research Service). 2019. U.S. war costs, casualties, and personnel levels since 9/11. CRS Report IF11182. https://crsreports.congress.gov/product/pdf/IF/IF11182 (accessed June 8, 2022).
Crum-Cianflone, N. F. 2013. The Millennium Cohort Study: Answering long-term health concerns of U.S. military service members by integrating longitudinal survey data with Military Health System records. In J. Amara and A. Hendricks (eds.), Military Health Care: From pre-deployment to post-separation. New York: Routledge. Pp. 55–77.
DoD (Department of Defense). 2021. Response to the Committee to Reassess the Department of Veterans Affairs Airborne Hazard and Open Burn Pit Registry information request: ILER Clarifications. Provided by Larry Vandergrift, Defense Health Agency, Chief Information Officer (CIO)/Deputy Assistant Director, Information Operations. July 13. Available from the project public access file at https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BPH-20-06.
Falvo, M. J., J. H. Abraham, O. Y. Osinubi, J. Klein, A. M. Sotolongo, D. Ndirangu, L. A. Patrick-DeLuca, and D. A. Helmer. 2016a. Bronchodilator responsiveness and airflow limitation are associated with deployment length in Iraq and Afghanistan veterans. Journal of Occupational and Environment Medicine 58(4):325–328.
Falvo, M. J., D. A. Helmer, J. C. Klein, O. Y. Osinubi, D. Ndirangu, L. A. Patrick-DeLuca, and A. M. Sotolongo. 2016b. Isolated diffusing capacity reduction is a common clinical presentation in deployed Iraq and Afghanistan veterans with deployment-related environmental exposures. Clinical Respiratory Journal 12:795–798.
Garshick, E. 2019. Service and Health Among Deployed Veterans (SHADE). Update: VA cooperative study #595: Pulmonary health and deployment to Southwest Asia and Afghanistan. VA Boston Healthcare System. Presentation before the Committee on the Respiratory Health Effects of Airborne Hazards Exposures in the Southwest Asia Theater of Military Operations. October 3. https://www.nationalacademies.org/event/10-03-2019/committee-on-the-respiratory-health-effects-of-airborne-hazards-exposures-in-the-southwest-asia-theater-of-military-operations-workshop-and-meeting-3 (accessed August 12, 2022).
Gray, G. C., A. W. Hawksworth, T. C. Smith, H. K. Kang, J. D. Knoke, and G. D. Gackstetter. 1998. Gulf War veterans’ health registries. Who is most likely to seek evaluation? American Journal of Epidemiology 148(4):343–349.
Hallman, W. K., H. M. Kipen, M. Diefenbach, K. Boyd, H. Kang, Leventhal, H. and D. Wartenberg. 2003. Symptom patterns among Gulf War Registry veterans. American Journal of Public Health 93(4):624–630.
Hill, A. B. 1965. The environment and disease: Association or causation? Proceedings of the Royal Society of Medicine 58:295–300.
Imbens, G. W., and D. B. Rubin. 2015. Causal inference in statistics, social, and biomedical sciences. Cambridge, U.K.; Cambridge University Press.
IOM (Institute of Medicine). 2011. Long-term health consequences of exposure to burn pits in Iraq and Afghanistan. Washington, DC: The National Academies Press.
Jani, N., M. Falvo, A. Sotolongo, O. Osinubi, C.-L. Tseng, M. Rowneki, M. Montopoli, S. Morley, V. Mitchell, and D. Helmer. 2017a. Self-reports of constrictive bronchiolitis among service members participating in the Veterans Administration and Department of Defense Airborne Hazards & Open Burn Pit Registry. Chest 152(4):A822.
Jani, N., M. J. Falvo, A. Sotolongo, O. Y. Osinubi, C. L. Tseng, M. Rowneki, M. Montopoli, S. W. Morley, V. Mitchell, and D. A. Helmer. 2017b. Blast injury and cardiopulmonary symptoms in U.S. veterans: Analysis of a national registry. Annals of Internal Medicine 167(10):753–755.
Jani, N., F. Molina, M. Rajan, A. M. Sotolongo, M. Falvo, and D. Helmer. 2019a. Agreement between lower respiratory symptoms and conditions across three modalities of assessment among veteran participants in the Airborne Hazards & Open Burn Pit Registry. Paper read at American Thoracic Society 2019 International Conference, May 17–22, Dallas, TX.
Jani, N., M. Rajan, M. Falvo, F. Molina, A. Sotolongo, and D. Helmer. 2019b. Pulmonary evaluation for U.S. veterans before vs after an optional, targeted registry examination. Chest 156(4):A862.
Jani, N., M. J. Falvo, M. Arjomandi, S. E. Hines, S. D. Krefft, J. J. Osterholzer, E. Shuping, and A. M. Sotolongo. 2020a. Examination of insomnia and other sleep conditions among deployed U.S. service members. Paper read at American Thoracic Society 2020 International Conference, May 15–20, Philadelphia, PA. https://www.atsjournals.org/doi/epdf/10.1164/ajrccm-conference.2020.201.1_MeetingAbstracts.A4342.
Jani, N., M. J. Falvo, J. J. Osterholzer, S. D. Krefft, S. E. Hines, M. Arjomandi, E. Shuping, and A. M. Sotolongo. 2020b. Self-reported sleep apnea among female U.S. service members in the Airborne Hazards & Open Burn Pit Registry. Paper read at American Thoracic Society 2020 International Conference, May 15–20, Philadelphia, PA. https://www.atsjournals.org/doi/abs/10.1164/ajrccm-conference.2020.201.1_MeetingAbstracts.A4339.
Jani, N., M. Falvo, M. Arjomandi, S. Krefft, J. Osterholzer, S. Hines, E. Shuping, and A. Sotolongo. 2021. COVID-19 infection among Airborne Hazards & Open Burn Pit Registry participants utilizing the VA. Paper read at American Thoracic Society International Conference, May 14–19. https://www.atsjournals.org/doi/abs/10.1164/ajrccm-conference.2021.203.1_MeetingAbstracts.A3086.
Lindheimer, J. B., D. B. Cook, J. C. Klein-Adams, W. Qian, H. Z. Hill, G. Lange, D. S. Ndirangu, G. R. Wylie, and M. J. Falvo. 2019. Veterans with Gulf War illness exhibit distinct respiratory patterns during maximal cardiopulmonary exercise. PLOS One 14(11):e0224833.
Liu, J., N. Lezama, J. Gasper, J. Kawata, S. Morley, D. Helmer, and P. Ciminera. 2016. Burn pit emissions exposure and respiratory and cardiovascular conditions among Airborne Hazards & Open Burn Pit Registry participants. Journal of Occupational and Environmental Medicine 58(7):e249–e255.
Miller, R. N., D. J. Costigan, H. A. Young, H. K. Kang, N. Dalager, R. W. Mathes, H. C. Crawford, W. F. Page, and S. Thaul. 2006. Patterns of health care seeking of Gulf War Registry members prior to deployment. Military Medicine 171(5):370–375.
Molina, F., N. Jani, M. Rajan, A. Sotolongo, M. Falvo, and D. Helmer. 2019. Agreement between dyspnea symptoms and lower respiratory conditions across three modalities of assessment among veteran participants in the Airborne Hazards & Open Burn Pit Registry. Paper presented at American Thoracic Society International Conference, May 17–22, Dallas, TX. https://www.atsjournals.org/doi/abs/10.1164/ajrccm-conference.2019.199.1_MeetingAbstracts.A2775.
Murphy, F. M., H. Kang, N. A. Dalager, K. Y. Lee, R. E. Allen, S. H. Mather, and K. W. Kizer. 1999. The health status of Gulf War veterans: Lessons learned from the Department of Veterans Affairs Health Registry. Military Medicine 164(5):327–331.
NASEM (National Academies of Sciences, Engineering, and Medicine). 2016. Gulf War and health: Volume 10: Update of health effects of serving in the Gulf War, 2016. Washington, DC: The National Academies Press.
NASEM. 2017. Assessment of the Department of Veterans Affairs Airborne Hazards and Open Burn Pit Registry. Washington, DC: The National Academies Press.
NASEM. 2018. Veterans and Agent Orange: Update 11, 2018. Washington, DC: The National Academies Press.
NASEM. 2020a. Assessment of long-term health effects of antimalarial drugs when used for prophylaxis. Washington, DC: The National Academies Press._
NASEM. 2020b. Respiratory health effects of airborne hazards exposures in the Southwest Asia Theater of Military Operations. Washington, DC: The National Academies Press.
Nieh, C., J. D. Mancuso, T. M. Powell, M. M. Welsh, G. D. Gackstetter, and T. I. Hooper. 2021. Cigarette smoking patterns among U.S. military service members before and after separation from the military. PLoS One 16(10):e0257539.
NRC (National Research Council). 2010. Review of Department of Defense Enhanced Particulate Matter Surveillance Program. Washington, DC: The National Academies Press.
Poisson, C., S. Boucher, D. Selby, S. P. Ross, C. Jindal, J. T. Efird, and P. Bith-Melander. 2020. A pilot study of airborne hazards and other toxic exposures in Iraq War veterans. International Journal of Environmental Research and Public Health 17(9):3299.
Powell, T. A., V. Mysliwiec, J. K. Aden, and M. J. Morris. 2020. Burn pit exposure in military personnel: Is there an effect on sleep-disordered breathing? Sleep and Breathing 25:479–485.
Rull, R. 2021. The Millennium Cohort Study. Presentation before the Committee to Reassess the Department of Veterans Affairs Airborne Hazards and Open Burn Pit Registry: Naval Health Research Center. June 23. Available from the project public access file at https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BPH-20-06.
Smith, B., M. A. Ryan, D. L. Wingard, T. L. Patterson, D. J. Slymen, and C. A. Macera, for the Millennium Cohort Study Team. 2008. Cigarette smoking and military deployment: A prospective evaluation. American Journal of Preventative Medicine 35(6):539–546.
Smith, T. C., B. Smith, M. A. Ryan, G. C. Gray, T. I. Hooper, J. M. Heller, N. A. Dalager, H. K. Kang, and G. D. Gackstetter. 2002. Ten years and 100,000 participants later: Occupational and other factors influencing participation in U.S. Gulf War health registries. Journal of Occupational and Environmental Medicine 44(8):758–768.
Smith, T. C., D. L. Jimenez, B. Smith, G. C. Gray, T. I. Hooper, G. D. Gackstetter, J. M. Heller, N. A. Dalager, H. K. Kang, K. C. Hyams, and M. A. Ryan. 2004. The postwar hospitalization experience of Gulf War veterans participating in U.S. health registries. Journal of Occupational and Environmental Medicine 46(4):386–397.
Smith, T. C., B. Smith, I. G. Jacobson, T. E. Corbeil, and M. A. Ryan. 2007. Reliability of standard health assessment instruments in a large, population-based cohort study. Annals of Epidemiology 17(7):525–532.
Stuart, J. A., K. M. Murray, R. J. Ursano, and K. M. Wright. 2002. The Department of Defense’s Persian Gulf War Registry year 2000: An examination of veterans’ health status. Military Medicine 167(2):121–128.
Talcott, G. W., J. Cigrang, D. Sherrill-Mittleman, D. K. Snyder, M. Baker, J. Tatum, D. Cassidy, S. Sonnek, C. Balderrama-Durbin, R. C. Klesges, J. O. Ebbert, A. M. Slep, and R. E. Heyman. 2013. Tobacco use during military deployment. Nicotine Tobacco Research 15(8):1348–1354.
VA (Department of Veterans Affairs). 2021a. VA Airborne Hazards and Open Burn Pit Registry. https://www.publichealth.va.gov/exposures/burnpits/registry.asp (accessed March 10, 2022).
VA. 2021b. Response to the Committee to Reassess the Department of Veterans Affairs Airborne Hazard and Open Burn Pit Registry information request. Provided by Dr. Eric Shuping, Director, Post-9/11 Era Environmental Health Program, Health Outcomes Military Exposures, VA. January 26. Available from the project public access file at https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BPH-20-06.
VA. 2021c. VA Maryland health care. https://www.maryland.va.gov/services/Post_Deployment_Cardiopulmonary_Evaluation_Network_PDCEN.asp (accessed May 23, 2022).
VA. 2021d. Response to the Committee to Reassess the Department of Veterans Affairs Airborne Hazard and Open Burn Pit Registry information request. Provided by Dr. Eric Shuping, Director, Post-9/11 Era Environmental Health Program, Health Outcomes Military Exposures, VA. April 14. Available from the project public access file at https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BPH-20-06.
VA. 2021e. Department of Veterans Affairs Airborne Hazards and Open Burn Pit Registry questionnaire demonstration. Provided by Dr. Eric Shuping, Director, Post-9/11 Era Environmental Health Program, Health Outcomes Military Exposures, VA. July 28. Available from the project public access file at https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BPH-20-06.
VA. 2022a. Airborne hazards and burn pits: What you need to know? Webinar presented by Dr. Anays Sotolongo, Dr. Michelle Robertson and Paulette Brower. February 24. https://www.tn.gov/content/dam/tn/veteranservices/learning/powerpoints/opportunity-training/airborne-hazards--what-you-need-to-know-02-24-2022/FINAL_ABH_Presentation_2022.pdf (accessed August 9, 2022).
VA. 2022b. WRIISC: About VA Airborne Hazards and Burn Pits Center of Excellence. https://www.warrelatedillness.va.gov/WARRELATEDILLNESS/AHBPCE/index.asp (accessed June 30, 2022).
VA. 2022c. Response to the Committee to Reassess the Department of Veterans Affairs Airborne Hazard and Open Burn Pit Registry information request. Provided by Dr. Eric Shuping, Director, Post-9/11 Era Environmental Health Program, Health Outcomes Military Exposures, VA. January 6. Available from the project public access file at https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BPH-20-06.
VA. 2022d. Response to the Committee to Reassess the Department of Veterans Affairs Airborne Hazard and Open Burn Pit Registry information request. Provided by Dr. Eric Shuping, Director, Post-9/11 Era Environmental Health Program, Health Outcomes Military Exposures, VA. May 24. Available from the project public access file at https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BPH-20-06.
VA AHBPCE (Department of Veterans Affairs Airborne Hazards and Burn Pits Center of Excellence). 2021. Airborne Hazards and Burn Pits Center of Excellence—Airborne Hazards and Open Burn Pit Registry efforts. Presentation by Dr. Nisha Jani and Dr. Michael Falvo to the Committee to Reassess the Department of Veterans Affairs Airborne Hazards and Open Burn Pit Registry. June 23. Available from the project public access file at https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BPH-20-06.
VA OIG (Department of Veterans Affairs Office of Inspector General). 2022. Airborne Hazards and Open Burn Pit Registry exam process needs improvement. Report #21-02732-153. July 21. https://www.oversight.gov/sites/default/files/oig-reports/VA/VAOIG-21-02732-153.pdf (accessed August 16, 2022).
VHA (Veterans Health Administration). 2021. Cooperative Studies Program #505: Millennium Cohort Study. Presentation by Dr. Edward Boyko to the Committee to Reassess the Department of Veterans Affairs Airborne Hazards and Open Burn Pit Registry. June 23. Available from the project public access file at https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BPH-20-06.
Wenger, J. W., C. O’Connell, and L. Cottrell. 2018. Examination of recent deployment experience across the services and components. Santa Monica, CA: RAND Corporation. https://www.rand.org/pubs/research_reports/RR1928.html (accessed July 18, 2022).
This page intentionally left blank.