Redesigning the Process for Establishing the Dietary Guidelines for Americans (2017) / Chapter Skim
Currently Skimming:
6 Current Approaches to Examining the Evidence: Key Findings
6 Current Approaches to Examining the Evidence: Key Findings
Pages 133-188
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 133... ...
how systematic reviews are conducted on longstanding DGA recommendations, including whether scientific studies are included from scientists with a range of viewpoints."3 To respond to these requests, this chapter summarizes the approach taken by this National Academies committee to review and evaluate the processes used in examining the evidence that underlies the DGA recommendations. This chapter is divided into sections to reflect the types of analyses tradi ionally used by the Dietary Guidelines Advisory Committee (DGAC)
|
From page 134... ...
existing systematic reviews, metaanalyses, and reports, (3) food pattern modeling, and (4)
|
From page 135... ...
It does not offer all possible types of analyses that could be used to answer these questions. DGAC = Dietary Guidelines Advisory Committee; NHANES = National Health and Nutrition Examination Survey; USDA = U.S.
|
From page 136... ...
The NEL was introduced in the 2010 cycle. The 2015 DGAC used the following types of analyses: • NEL systematic reviews: Comprehensive reviews of the literature that adhere to established principles, as well as updates of exist ing systematic reviews • Existing sources of evidence: Evaluations of sources of evidence such as published systematic reviews, meta-analyses, and reports • Food pattern modeling: A type of sensitivity analysis that incor porates various data inputs, constraints, goals, and assumptions to inform food patterns and resulting nutrient profiles, as well as to answer various questions regarding the effects of modifica tions to food patterns
|
From page 137... ...
. The use of systematic reviews has varied across cycles with respect to the use of original and existing systematic reviews, meta-analyses, and reports.
|
From page 138... ...
According to NEL protocol, systematic review questions are developed and prioritized in advance of the decision to use an existing systematic review, meta-analysis, or report, or to conduct a de novo systematic review. The process of identifying, evaluating, and deciding whether or not an existing systematic review should be included or excluded requires a different set of considerations than conducting an original systematic review (see "Assessment of the NEL Process for Using Existing Systematic Reviews, Meta-Analyses, and Reports" beginning on page 167)
|
From page 139... ...
For original NEL systematic reviews, the review team is composed of the following: • A DGAC subcommittee (four to seven members) , for the purpose of providing expertise specific to the review topic and knowledge of the field; • One or more NEL analysts, who assist the DGAC in planning, facilitating, conducting, and documenting the systematic review to ensure alignment with NEL methodology; • One or more NEL librarians, who manage the development, implementation, refinement, and documentation of the search strategy; and • NEL abstractors, individuals with advanced degrees in nutrition or a related field, who assist in data extraction and risk of bias assessment.
|
From page 140... ...
For each suggested topic, the rationale for review, target population, and public health outcomes of interest are outlined, and the approach for examining the evidence for the topic is recommended by the DGAC. These steps apply to all topics, regardless of the type of analysis used to examine the evidence (i.e., original NEL systematic review; existing highquality systematic reviews, meta-analyses, and report(s)
|
From page 141... ...
NEL protocol specifies the methods used at each of these steps, which are outlined in detail in the text. b If no existing systematic reviews, meta-analyses, or reports are identified in the literature search, this step is omitted.
|
From page 142... ...
For topics that are selected to be addressed with systematic reviews -- either original NEL systematic reviews, or existing high-quality reports when available -- questions are developed by the DGAC with assistance from federal staff according to the PICO (population, intervention/exposure, comparator, and outcome of interest) framework.
|
From page 143... ...
. Existing systematic reviews and meta-analyses are identified in a duplication assessment and may be used at the discretion of DGAC subcommittees to replace or augment an original NEL systematic review.
|
From page 144... ...
If an existing systematic review, meta-analysis, or report is identified during the search and screening process, the DGAC subcommittee is responsible for determining if and how it should be used, based on the report's relevance to the systematic review question of interest, the quality of the report, the timeliness of the report, and with consideration for reference overlap. The assessment is based on PICO elements, AMSTAR rating, and the date range of the existing systematic review (see "Assessment of the NEL Process for Using Existing Systematic Reviews, Meta-Analyses, and Reports" beginning on page 167 for more information)
|
From page 145... ...
. The NEL BAT was developed to assess the risk of bias of individual studies included in NEL systematic reviews, and is based on existing risk of bias tools, including those developed by the Agency for Healthcare Research and Quality and Cochrane, and follows a question/answer format (Higgins and Green, 2011; USDA/HHS, 2016; Viswanathan et al., 2012, 2013; West et al., 2002)
|
From page 146... ...
This review is facilitated by a series of probing questions provided by the NEL analyst, referred to as the evidence portfolio worksheet, which are independently completed by each DGAC subcommittee member. These questions vary in their focus and aim and are intended to aid the DGAC members in comparing and contrasting the studies reviewed, and to assist with subsequent development of a conclusion statement along with a grade of the overall quality of the evidence.
|
From page 147... ...
Per NEL protocol, conclusion statements are not to address areas outside of the body of evidence reviewed and are not intended to express implications. Drafting conclusion statements, as with the evidence description and synthesis, is an iterative process involving both DGAC subcommittee and NEL staff.
|
From page 148... ...
Step 6: Identification of Research Recommendations Research recommendations are initially developed and drafted during the evidence description and synthesis step to reflect gaps and/or limitations in the body of evidence, but can be revised and updated to reflect the continued discussions concerning conclusions and grading of evidence before being finalized. Emerging topics in particular can be included in the DGAC Scientific Report with a rationale describing the need for additional research.
|
From page 149... ...
For example, the NEL systematic reviews are documented in their entirety and, following the completion of the review and the publication of the DGAC Scientific Report, are posted on the NEL website (NEL.
|
From page 150... ...
As a result, the NEL has two separate protocols for conducting systematic reviews: one for DGAC-requested systematic reviews, and one for non-DGAC systematic reviews.6 The two protocols have many similarities and use the same steps, but there are key differences. The fundamental difference between the two protocols is that for DGAC-requested systematic reviews, the DGAC is the approver and "authors" the systematic review (USDA/HHS, 2017)
|
From page 151... ...
. To assess the NEL process, the systematic review process from the 2015 DGAC Scientific Report was outlined by this National Academies committee according to systematic review steps adapted from the report Finding What Works in Health Care: Standards for Systematic Reviews (IOM, 2011)
|
From page 152... ...
systematic reviews; 4–7 o NEL staff was members were included on supported by each subcommittee, based on abstractors, who expertise; 2 subcommittees were trained by added consultants to provide NEL staff to review additional subject-matter individual research expertise. (Working groups were articles included similarly organized to address in NEL systematic crosscutting topics later in the reviews.
|
From page 153... ...
of the team conducting the systematic review. 2.1 Require each • Certain USDA employees • DGAC members were required team member hold positions requiring to disclose potential financial to disclose them to file the annual COIs annually through OGE potential COI OGE Form 450, which Form 450; professional and and professional includes disclosure of intellectual biases were noted to or intellectual financial interests and be considered in the selection of bias.
|
From page 154... ...
3.1 Protect the Public comments about the systematic review design and independence of conduct were received and reviewed by the 2015 DGAC. the review team to make the final decisions about the design, analysis, and reporting of the review.
|
From page 155... ...
Federal staff reviewed every comment and filtered providing out any duplicate, blank, or irrelevant comments. input into the systematic review.
|
From page 156... ...
. • Information collected during topic identification included a brief description of the topic and rationale that explained the importance of the topic, a description of the population, interventions/exposures, comparators, and outcomes of interest (PICO)
|
From page 157... ...
review.b 5.3 Use a • Using key information • Draft systematic reviews standard format collected during topic questions were refined using to articulate identification, federal an iterative process between each question of staff assisted the DGAC the DGAC and the NEL, which interest.b with drafting systematic incorporated the various review questions using scientific perspectives of the the PICO framework.
|
From page 158... ...
. included in the search plan and results; criteria were presented during public meetings and posted publicly as part of DGAC presentations immediately after meeting and after the completion of the review (DGAC, 2014)
|
From page 159... ...
final search strategy. • NEL librarian and analysts may conduct a duplication assessment to determine whether any existing high-quality systematic reviews and/ or meta-analyses existed that address systematic review questions posed.
|
From page 160... ...
• NEL analysts reviewed extracted data for each study for quality control purposes. 6.7 Describe • Differences between the • N/A the process analyst and abstractor for identifying regarding data extraction and resolving or NEL Bias Assessment disagreement Tool responses were between resolved, and a thirdresearchers party consultation with in study an additional member selection and of the federal staff was data extraction solicited when needed decisions.
|
From page 161... ...
• The DGAC reviewed the final draft synthesis and conclusion statement to ensure that its input was interpreted correctly, to solicit responses to clarifying questions, and to request feedback on the synthesis and conclusion statement. • Step 4: The DGAC evaluated and graded the body of evidence for each conclusion using the NEL grading rubric, which is based on five elements -- internal validity, adequacy, consistency, impact, and generalizability.
|
From page 162... ...
7. Submit the The protocol for individual systematic reviews was not protocol for submitted for peer review.
|
From page 163... ...
The numbered systematic review steps are adapted from the Institute of Medicine report Finding What Works in Health Care: Standards for Systematic Reviews. The systematic review team was considered to include NEL staff, abstractors, and the DGAC members.
|
From page 164... ...
Conclusion Overall, the NEL process for conducting original systematic reviews is thorough and adheres to several of the existing systematic review standards in the field. However, the overall protocol needs to be strengthened
|
From page 165... ...
Systematic reviews including observational studies in particular will need to be carefully evaluated in the interpretation of results and development of conclusions. ASSESSMENT OF THE NEL PROCESS FOR UPDATING SYSTEMATIC REVIEWS Because the NEL has only been used to support two editions of the DGAC Scientific Report, the only opportunity to update systematic reviews conducted previously by the NEL was in the 2015 DGAC.
|
From page 166... ...
Ongoing surveillance can also identify existing systematic reviews that may replace the need to conduct an update of a systematic review.
|
From page 167... ...
Identifying Existing Systematic Reviews, Meta-Analyses, and Reports As systematic review questions were developed and prioritized, the DGAC collaborated with the NEL to develop an analytic framework. At this point, before the literature search and screening begins, existing systematic reviews, meta-analyses, and reports may have been identified a priori by DGAC subcommittee members aware of current literature.
|
From page 168... ...
Criteria for Inclusion Existing systematic reviews, meta-analyses, and reports were assessed by the NEL and the DGAC to determine if they met the predetermined inclusion/exclusion criteria. In some cases, federal DGAC support staff other than the NEL supported the DGAC in its review of existing systematic reviews, meta-analyses, and reports.
|
From page 169... ...
In some cases, targeted questions may have been prepared by the federal staff to facilitate the DGAC members' identification of themes and key findings from the systematic reviews. The review of evidence specific to each systematic review question was outlined in the 2015 DGAC Scientific Report (at the same level of detail as with original systematic reviews, including a conclusion statement and grade, implication statement, and summary of the review of evidence)
|
From page 170... ...
A common challenge in using existing systematic reviews and relying on reported information is not having the necessary information to allow independent verification of the validity of the analyses and conclusions. Without the abilities to verify, the user of an existing systematic review has to trust the veracity of the reported information.
|
From page 171... ...
Conclusion In summary, use of existing systematic reviews, meta-analyses, and authoritative reports from leading organizations is generally appropriate and encouraged by this National Academies committee, with the understanding that they ought to be relevant, timely, and of high quality. Efficiency and use of time and resources must be weighed carefully in using an existing systematic review compared to conducting a de novo review (Smith et al., 2011; Whitlock et al., 2008)
|
From page 172... ...
In effect, food pattern modeling shows how diets could be developed to meet those constraints. Three different patterns developed by USDA were featured in the 2015–2020 DGA Policy Report -- "Healthy US-Style," "Healthy Mediterranean-Style," and "Healthy Vegetarian" -- as "examples of healthy eating patterns that can be adapted based on cultural and personal preferences" (HHS/USDA, 2015a)
|
From page 173... ...
NOTES: The traditional USDA food groups are vegetables, fruits, grains, dairy, protein foods, oils, and calories for other uses. DGAC = Dietary Guidelines Advi ory s Committee; IOM = Institute of Medicine; USDA = U.S.
|
From page 174... ...
That is, by design, the nutrient profile associated with each food group corresponded to average nutrient values associated with a population-weighted mix of pure foods in that group. The exact procedure involved in food pattern modeling depends on the specific question being addressed, but the general process is outlined in Figure 6-1 and described below.
|
From page 175... ...
replace meat, poultry, and eggs in the Healthy Vegetarian Eating Pattern. BOX 6-1 Food Groups and Subgroups in USDA's Healthy Eating Patterns Vegetables Dark-green vegetables Red and orange vegetables Legumes (beans and peas)
|
From page 176... ...
Food pattern modeling could benefit from such a review. Role of the DGAC in Food Pattern Modeling The other steps shown in Figure 6-1 are initiated by the DGAC (i.e., review findings from systematic reviews along with proposed eating patterns and suggesting new goals for some nutrients and other options
|
From page 177... ...
These steps may also be conducted iteratively, as they relate to evaluating the patterns and their possible modifications, based on the DGAC's assessment of findings from the systematic reviews and other descriptive data analyses. Many of the questions shown in Appendix C involve these steps.
|
From page 178... ...
Recent iterations of USDA eating patterns have allowed soy beverages to substitute for animal milk, but that does not entirely address the mismatch between the guidance and the food and beverage preferences of many individuals. The output of the food pattern modeling is in terms of the total daily quantity of foods from each group.
|
From page 179... ...
A key result of the food pattern modeling is demonstrating there is a very small allowance for discretionary calories when trying to meet nutrient goals within energy levels appropriate for most Americans. In effect food modeling was used to set limits on substances such as fat, oil, and sugars rather than using a physiologic or metabolic end point.
|
From page 180... ...
The relatively limited inclusion of foods containing salt and other sources of sodium results in patterns that approximate the UL without the addition of salt at the table, which is instructive regarding the austerity of the sodium UL. Conclusion In summary, the current process for food pattern modeling and its use in informing the DGAC review of the evidence is generally well designed for the questions it is intended to answer.
|
From page 181... ...
Different from the questions completed by the NEL systematic reviews, the questions addressed using descriptive data analyses did not go through a grading rubric and were not graded. The DGAC acknowledged this issue by taking the "strengths and limitations of data analyses into account in formulating conclusion statements" (HHS/USDA, 2015b)
|
From page 182... ...
SOURCE: HHS/USDA, 2015b. 2015 DGAC Scientific Report were not available, the sources of data were included in the DGAC's description of the evidence (HHS/USDA, 2015b)
|
From page 183... ...
In reviewing descriptive data analyses conducted to inform past DGACs by various federal agencies and offices based on availability and relevance, this National Academies committee sees the introduction of the DAT as an opportunity for improving reporting of data analyses and centralizing efforts. For example, a central Web-based location that includes links to all the data analyses would be useful in increasing transparency and comparability of descriptive data analyses across DGA cycles.
|
From page 184... ...
QUALITY OF DIETARY DATA ACROSS ALL TYPES OF EVIDENCE Dietary intake data are central to the development of dietary guidelines, and form the basis of most studies examined in the systematic evidence-based reviews and analyses of food pattern modeling and consumption patterns of the population. The nutrition field has relied on respondent self-report as a means of assessing dietary intake to capture the totality of the diet.
|
From page 185... ...
CONCLUSION The types of analyses used in the DGAC Scientific Report include original systematic reviews; existing systematic reviews, meta-analyses, and reports; food pattern modeling; and descriptive data analyses. All types of analyses will continue to provide important information in the DGA going forward, and they need to be based on validated, standardized, and up-to-date methods and processes.
|
From page 186... ...
2016. When and how to update systematic reviews: Consensus and checklist.
|
From page 187... ...
2012. A series of systematic reviews on the effects of nutrition education on children's and adolescents' dietary intake.
|
From page 188... ...
2008. Using existing sys tematic reviews in complex systematic reviews.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.