2

Methodological Approach to the Task

The Committee on Evaluating the Process to Develop the Dietary Guidelines for Americans, 2020–2025 (2020–2025 DGA) approached its work by addressing each of three subtasks within the Statement of Task (see Chapter 1, Box 1-2) as separate tasks. The committee devised a set of tools for assessing each of the first two subtasks, referred to herein as task 1 and task 2. In this chapter, the committee describes its interpretation of the task, its methodological approach to it, and the tools used to gather relevant evidence and assess the evidence to support its findings and conclusions.

INTERPRETATION OF THE TASKS

The committee’s task (see Box 2-1) was initially described by Congress (see Chapter 1). The U.S. Department of Agriculture (USDA) and the National Academies of Sciences, Engineering, and Medicine (the National Academies) then developed questions to guide the committee’s response.

The committee interpreted task 1 as a request to compare what was recommended in the 2017 National Academies report (NASEM, 2017) with what was done by the federal agencies responsible for the DGA process. The committee further interpreted this within the context of the background information for each recommendation in the 2017 National Academies report as well as the guiding principles articulated in that report. The questions developed from USDA and the National Academies included a focus on “scientific methodologies, review protocols, and evaluation processes” (see Box 2-1). These terms did not apply to all

recommendations but were used when relevant. To clarify possible issues as well as to be able to draw useful conclusions, the committee developed key questions. These questions were posed to representatives of USDA and the U.S. Department of Health and Humans Services (HHS) both before and during open sessions (see Appendix B).

The committee interpreted task 2 as a request to compare the scientific studies used to establish the DGA to the most current and rigorous studies on this topic (see Box 2-1). Therefore, the committee sought to assess the quality of the systematic review processes used to select the studies included in systematic reviews conducted for the 2020 DGAC Scientific Report (DGAC, 2020). The committee conducted its analysis within a broad definition of diet-related chronic diseases. The question developed by USDA and the National Academies requested information about specific attributes (“current,” “rigorous,” and “generalizable”) of the evidence base within the specific context of providing guidance on public health nutrition. As was the case for task 1, the committee developed key

questions related to task 2 that were posed to representatives of the federal agencies before and during open sessions (see Appendix B).

Two important concepts from the 2017 National Academies report (NASEM, 2017), namely “transparency” and “scientific rigor,” were integral to the committee’s work. Transparency is an essential component of trustworthiness and, thus, is essential for the public’s acceptance of dietary guidance (NASEM, 2017). Transparency is achieved by “requiring each step of the process be documented and updated, and that such documentation be readily available to the public” (NASEM, 2017, p. 41). This committee used this description as a standard when assessing the processes used to develop the 2020–2025 DGA. Scientific rigor is an essential component of developing evidence-based guidelines. The authors of the 2017 National Academies report stated that

The committee used this standard when considering the recommendations and tasks related to the systematic reviews and other data sources used in the 2020 DGAC Scientific Report (DGAC, 2020), as well as the recommendations related to food pattern modeling and the inclusion of systems science.

The committee gave priority to task 1 as well as key elements of task 2, which it viewed as providing time-sensitive information to USDA and HHS for the development of the 2025–2030 DGA.

METHODOLOGICAL APPROACH TO TASK 1

Overview

In task 1 (see Box 2-1), the committee was asked to compare the scientific methodologies, review protocols, and evaluation processes used by USDA and HHS to develop the most recently issued 2020–2025 DGA (USDA and HHS, 2020) against each of seven recommendations made in the report Redesigning the Process for Establishing the Dietary Guidelines for Americans (NASEM, 2017) and aimed at enhancing the scientific rigor and transparency of those processes. Additionally, the committee con-

ducted literature searches in PubMed to identify relevant peer-reviewed published literature.

Approach to Assessing the Evidence

To guide its assessment of the evidence for task 1, the committee developed a flowchart (see Figure 2-1). To gather evidence on the responses made to each recommendation, the committee formulated key questions to pose to representatives of USDA and HHS. In response to these questions, the sponsor, the USDA Food and Nutrition Service, provided summary documents, including the Scientific Report of the 2020 Dietary Guidelines Advisory Committee (DGAC, 2020) and the Dietary Guidelines for Americans, 2020–2025 (USDA and HHS, 2020); numerous website links; as well as documents provided in response to dialogue with the committee in two open sessions held with representatives of USDA and HHS (see Appendix B).

Some of these key questions were dropped or reformulated after additional information was acquired. When it seemed necessary, the committee also consulted the scientific literature for additional information about the background of an issue or to go beyond what the information contained in federal reports and websites. In particular, recommendation 4 calls for the processes for conducting systematic reviews to align with “best practices.” To evaluate progress toward this recommendation, the committee identified practices used by leading organizations (see Appendix E) and compared them to those used by USDA for its systematic reviews. Briefly, the practices were identified by the committee as they were used by organizations that are regarded as leaders in the systematic review process, including the Cochrane Library, the Agency for Healthcare Research and Quality (AHRQ), the U.S. Preventive Services Task Force (USPSTF), and a National Academies report (AHRQ, 2018; Higgins et al., 2021; IOM, 2011; USPSTF, 2018).

The process of analysis and assessment of the USDA and HHS response to the 2017 National Academies recommendations was guided by a flowchart developed by the committee to compare each of the USDA and HHS responses to a set of criteria relevant to a specific recommendation, and linked to the relevant evidence. The committee then developed a summary of the extent to which each recommendation was implemented. In the case of partial implementation, the committee discussed the importance of partial implementation relative to the guiding principles and goals of the 2017 National Academies report (NASEM, 2017). In the final report, the committee includes a discussion of the outcomes and implications for future DGA reviews.

Approach to the Comparative Analysis

Based on the 5-year DGA cycle, the committee formulated a timeline (see Figure 2-2) for comparing the steps in the process to develop the 2015–2020 DGA and the 2020–2025 DGA against the 2017 National Academies recommendations. This allowed the committee to readily identify where changes were made to existing steps in the process that were consistent with those recommendations. In instances where USDA and HHS determined that time and/or resources did not permit the implementation of a recommendation, or only partial implementation was feasible, the committee identified what the agencies were able to do instead. This analysis was carried out using the same information resources (to the extent that they were available) for the process to develop the 2020–2025 DGA as noted above.

METHODOLOGICAL APPROACH TO TASK 2

Overview

For task 2 (see Box 2-1), the committee was asked to compare the scientific studies used to develop the DGA to determine how the dietary

needs of Americans with diet-related metabolic diseases compared to the most current and rigorous scientific studies on diet and diet-related metabolic diseases available.

Approach to Assessing the Evidence

The committee began with the assumption that the body of evidence used by the 2020 Dietary Guidelines Advisory Committee (DGAC) is affected by several key steps in the systematic review process. Its approach to task 2, therefore, was to determine whether the body of evidence used to develop the DGA included those studies with diet-related metabolic disease and were the most current and rigorous scientific studies available. Although evidence from existing (not updated) systematic reviews is used to develop the DGA (see Table C-5), the committee elected to focus on the new and updated systematic reviews.¹ The committee defined diet-related metabolic diseases as the nutrition-related chronic health conditions identified in the 2020 DGAC Scientific Report. These include overweight, obesity, metabolic syndrome, prediabetes, diabetes mellitus, reduced muscle strength and bone mass, gestational diabetes mellitus, chronic liver disease, cancer, cardiovascular disease, dental health, and food allergy. In defining these diseases, the committee was aware other systematic reviews could indirectly address diet-related metabolic conditions, such as preventing cognitive decline or maintaining neurocognitive health and development (DGAC, 2020).

The committee developed a flowchart (see Figure 2-3) to guide its assessment of the Nutrition Evidence Systematic Review (NESR) methodology that informed the search plan and inclusion and exclusion criteria used to identify the body of evidence. Concurrently, a consultant to the committee, guided by the committee’s needs, developed a systematic review matrix summary (see Appendix C) that contained the details of each of the new and updated systematic reviews used by the 2020 DGAC. This summary was used by the committee in its evaluation of the appropriateness of the NESR methods. This summary allowed the committee to compare specific data fields across systematic reviews. Additional tables and figures were created using data collected in the summary to address specific questions. The data included questions answered by the systematic reviews, detailed inclusion and exclusion criteria, participant details, conclusions, and grading of evidence in the systematic reviews.

___________________

¹ This text, and similar text throughout the report, was modified after release of a prepublication version of the report to the sponsor to clarify that the committee focused on new and updated systematic reviews, but evidence is also utilized from existing, not updated systematic reviews.

Approach to the Comparative Analysis

The committee used its framework (described under task 1) together with information from the systematic review matrix summary (see Appendix C) to develop decision criteria relevant to task 2. The final decision criteria for specific recommendations germane to task 2 included

1. The appropriateness of the NESR approach to the search strategy, including date range, types of research included, and the health status of the individuals included in research included; and
2. The application of the search strategy to specific systematic review questions examined and published to inform the 2020 DGAC Scientific Report.

The decision criteria were then applied across the NESR systematic reviews used in the cycle to produce the 2020–2025 DGA to determine whether committee-identified practices (described in Appendix E) were consistently implemented, partially implemented, or not implemented or whether implementation could not be assessed because of a lack of information. The comparative analyses carried out for task 1 and task 2 were used to formulate the committee’s findings and conclusions presented in Chapters 3 and 4, respectively.

REFERENCES

This page intentionally left blank.