3

Analysis of the Scientific Methodologies, Review Protocols, and Evaluation Processes of the Dietary Guidelines

INTRODUCTION

The first task of the Committee Evaluating the Process to Develop the Dietary Guidelines for Americans 2020–2025 was to conduct a comparative analysis of the scientific methodologies, review protocols, and evaluation processes used to develop the 2020–2025 edition of the Dietary Guidelines for Americans (2020–2025 DGA) (USDA and HHS, 2020) against the seven recommendations for improving the process cited in the report Redesigning the Process for Establishing the Dietary Guidelines for Americans (2017 National Academies report) (NASEM, 2017a). The committee was asked specifically to respond to the question: How did the process used to develop the Dietary Guidelines for Americans, 2020–2025, including scientific methodologies, review protocols, and evaluation processes, compare to the seven recommendations in the 2017 National Academies report? In this chapter, the committee provides its analysis of the implementation of those recommendations in the process to develop the 2020–2025 DGA.

Key Considerations

The overarching goal of the 2017 National Academies recommendations was that the DGA process be “universally viewed as valid, evidence-based, and free of bias … to the extent possible,” in particular by the U.S. population (NASEM, 2017a, p. 35) (see Box 3-1). The 2017 National Academies report recognized that change in the scope and process for

developing the DGA has been ongoing since the guidelines’ inception in 1980 (see Chapter 1). Thus, the committee views recommendations for the improvement of future versions of the DGA as a continuation of this ongoing process. To illustrate, the adoption of food-pattern modeling began with the 2005–2010 edition of the DGA (HHS and USDA, 2005), and the 2020–2025 edition was the first to include chapters specific to pregnant or lactating individuals, infants, and children up to 24 months of age¹ (USDA and HHS, 2020).

As noted in Chapter 1, the DGA underpin all federal nutrition policy and programs (HHS, 2021). The public sees the results of the DGA process in their daily lives in the form of dietary recommendations aimed at improving population health. The DGA are also used by health professionals, educators, the food industry, and others. Thus, to maintain their relevance the DGA must be based on a preponderance of current scientific evidence.

The 2017 National Academies report was preceded by the report Optimizing the Process for Establishing the Dietary Guidelines for Americans: The Selection Process (NASEM, 2017b). The recommendations in this report were intended to be implemented rapidly, to the extent possible, so that improvements to the process of nominating and selecting the Dietary

___________________

¹ Text in this paragraph was modified after release of a prepublication version of the report to the sponsor to correct the year the first DGA was published and to clarify that pregnant/lactating individuals, infants, and children up to 24 months of age were included in previous DGA editions; however, this was the first edition to include chapters specific to these populations.

Guidelines Advisory Committee (DGAC) could be made early in the process to develop the 2020–2025 DGA (NASEM, 2017b). The 2017 committee framed its recommendations in the second report (NASEM, 2017a) around five values (see Box 3-1). These five values guided the 2017 committee in developing its recommendations to improve the transparency and rigor of the subsequent DGA processes (NASEM, 2017a). However, in contrast to the National Academies (2017b) report, no time line was specified for the implementation of the seven 2017 National Academies recommendations (NASEM, 2017a). The committee recognized that implementation of these recommendations would take time to be accomplished—time for initial analysis and consideration by the secretaries of the U.S. Department of Agriculture (USDA) and the U.S. Department of Health and Human Services (HHS) and, if adopted, time for deliberative planning, budgeting, and staffing to support implementation. Thus, in its analysis of the DGA process, the committee gave consideration to the fact that full implementation of the 2017 National Academies recommendations would occur over the course of subsequent DGA cycles.

APPROACH TO TASK 1

As discussed in Chapter 1, the committee’s charge was to assess whether the seven 2017 National Academies recommendations had been considered by USDA and HHS in developing the 2020–2025 DGA, and, if so, the extent to which implementation of the recommendations over the course of the DGA cycle “met the spirit” of the report and were consistent with the five key values for improving the integrity of the process (see Box 3-1). The methodological approach taken to carry out the task is described in detail in Chapter 2. A summary of this approach is outlined below.

In brief, the committee began by considering at what point in the course of the 5-year DGA cycle the 2017 National Academies recommendations would be implemented (see Chapter 2, Figure 2-2). The committee recognized that some recommended changes could not reasonably be expected to be implemented within a cycle that was already in process at the time of the release of the 2017 National Academies report. In response, the committee developed a flowchart that could be used to assess the progress of each of the seven recommendations. The flowchart (see Chapter 2, Figure 2-1) allowed the committee to assess whether specific recommendations from the 2017 National Academies report were implemented fully, partially, or not at all. If a recommendation was partially implemented, the committee identified which components of the recommendation were implemented. Throughout the assessment, the committee remained mindful that the indicator of success is the aggregate response by the federal agencies to the seven 2017 National Academies

recommendations and that the overall changes led to improvements in the 2020–2025 DGA process so that it is, or comes close to being, “universally viewed as valid, evidence-based, and free of bias … to the extent possible” (NASEM, 2017a, p. 35).

COMPARING THE DIETARY GUIDELINES FOR AMERICANS PROCESS TO RECOMMENDED REVISIONS

Background

The Committee to Review the Process to Update the Dietary Guidelines for Americans (2017 committee), in its report, Redesigning the Process for Establishing the Dietary Guidelines for Americans (NASEM, 2017a), recommended redesigning the process to prioritize topics to be reviewed in the DGA cycle and to redistribute the current functions of the DGAC to three separate groups (see Box 3-2, recommendation 1). This committee evaluated the 2017 National Academies recommendation 1 in light of whether the changes that were made support the remaining recommendations.

Assessment of Recommendation 1

Background

Inherent in recommended changes to the DGA process is the separation of functions related to transparency, management of conflicts of

interest, and conducting a more deliberative process. However, since separating the functions of USDA’s Nutrition Evidence Systematic Review (NESR) (formerly Nutrition Evidence Library [NEL]) and the DGAC would also mean reassigning some functions of the DGAC, new names were recommended in 2017. These were the Dietary Guidelines Planning and Continuity Group (DGPCG) and the Dietary Guidelines Scientific Advisory Committee (DGSAC) (closely akin to the DGAC but not identical in its functions) (see Table 3-1).²

The 2017 committee was aware that recommendation 1, if implemented in whole or part, would require a considerable investment in planning, staff time, and resources. It is noteworthy that, given the complexities involved in process redesign that would be required to implement recommendation 1 (e.g., a new committee, the DGPCG, which, if a federal advisory committee, would require a new charter; the reassignment of some functions previously performed by the DGAC or NESR; and further

TABLE 3-1 Groups Proposed in the 2017 National Academies Report and Their Functions

NOTE: CVD = cardiovascular disease; HHS = U.S. Department of Health and Human Services; NEL = Nutrition Evidence Library; P/B-24 = pregnant women and children from birth to 24 months; PICO = population, intervention/exposure, comparator, outcome of interest; TEP = technical expert panel; USDA = U.S. Department of Agriculture.

SOURCE: NASEM, 2017a, p. 59.

___________________

² Text in this paragraph was modified after release of a prepublication version of the report to the sponsor to clarify that the 2017 National Academies report called for management rather than avoidance of conflicts of interest and to clarify that NEL is now NESR. Similar clarifications have been made throughout the report.

efforts to define the optimal roles of the technical expert panels [TEPs]), there was no deadline specified for this recommendation.

The National Academies (2017a) report was released to the USDA and HHS secretaries after the cycle to develop the 2020–2025 DGA report had been initiated (see Chapter 2, Figure 2-2). NESR was already in the processes of conducting systematic reviews as part of the Pregnancy and Birth to 24 Months (P/B-24) project; however, the systematic reviews conducted by the 2020 DGAC were initiated after the release of the 2017 National Academies report.³ Improvements in how the members of the DGAC would be selected and appointed to minimize conflicts of interest, expand input, and improve transparency had been requested in 2017 so the input could be incorporated into the process to develop the 2020–2025 DGA. The remaining recommended changes were not required to be implemented in 2017, which suggests that the federal agencies knew that they probably could not incorporate the recommendations during that DGA cycle; nevertheless, they would be useful for future planning. As discussed in Chapter 2, to assess recommendation 1, the committee compared the time line proposed by the 2017 committee against the actual time line for the process to develop the 2020–2025 DGA (see Chapter 2, Figure 2-2) and assessed the extent to which progress toward implementation of the 2017 National Academies recommendation had been made.

Review of Evidence

The committee identified topic prioritization as a focus of its analysis of recommendation 1 because the 2017 committee observed that analyses tended to be repeated across DGA cycles, even when new research was limited (NASEM, 2017a). Thus, the process could become more efficient by prioritizing those topics with significant new evidence, and using the released time and effort to address new topics of public health concern that had emerged since the previous DGA cycle. As specified in the 2017 National Academies recommendation, topic prioritization would take place before the appointment of the DGAC (NASEM, 2017a). Given the limited 2-year term of the DGAC, the concept of a new committee, the DGPCG, was proposed as a mechanism to “pre-advise” on the priority topics for the new DGSAC. To assess implementation of this recommendation the committee considered three overarching questions:

___________________

³ This text was modified after release of a prepublication version of the report to the sponsor to clarify which systematic reviews had started prior to the release of the 2017 National Academies report and which had not yet been initiated.

1. What was the process used to prioritize topics for the 2020–2025 DGA?
2. Were any topics prioritized for the next edition of the DGA?
3. Was input from the public received relative to selection of the topics for review? If so, how was this input used? If not, why not?

In response to these questions, the committee’s review of evidence showed that the process for prioritizing topics for the development of the 2020–2025 DGA included topics developed as part of the Birth to 24 (B-24) Topic Identification and P/B-24 projects, as well as those from prior DGA cycles that were relevant to other life stages (Raiten et al., 2014; Stoody et al., 2019). USDA and HHS prioritized the list of topics with input from other federal agencies as well as public comments before the DGAC was established (DGAC, 2020). To elicit feedback from the public, USDA and HHS posted a list of proposed topics and scientific questions for public comment (DGAC, 2020). Following the public comment period, USDA and HHS released the original and a second list of topics. The second list was organized differently than the original list in that it included a more global description of the topics, except for those topics related to P/B-24 (DGA, n.d.-a).⁴ However, in the final list, “Some topics are not included … because they are addressed in existing evidence-based Federal guidance” (DGA, n.d.-a). To avoid duplication, the 2020–2025 DGA referenced this guidance (USDA and HHS, 2020). Several examples of existing guidance (e.g., guidance on food safety, health risk of excessive alcohol use, gestational weight gain, physical activity, Dietary Reference Intakes [DRIs]) were provided.

Before examining the specific responses of USDA and HHS to recommendation 1 and its subcomponents a, b, and c, the committee considered that

However, establishing a new advisory committee is an extensive and time-consuming process. For the 2015–2020 DGA cycle, USDA and HHS

___________________

⁴ This text was modified after release of a prepublication version of the report to the sponsor to clarify that both the original and the second list of topics were posted to the DGA website. In addition, a sentence that inaccurately stated that the public comments were not available to the DGAC was removed. Public comments were available to the DGAC through the DGA website throughout the process. This sentence was also removed from the section relating to recommendation 1a below.

were not able to establish an additional advisory committee (see Appendix B-5).

As a result, none of the three recommended groups was established during the process to develop the 2020–2025 DGA.

Recommendation 1a

Recommendation 1a (see Box 3-2) called for the creation of a new group, the DGPCG, that would be responsible for developing a brief report of the criteria and logic for the list of topics and associated research questions recommended. This group would operate across DGA cycles to provide strategic planning of future reviews (NASEM, 2017a). To assess the implementation of this recommendation, the committee considered the following two questions:

1. How were the proposed functions of this group carried out and by whom?
2. What are the processes for identifying and prioritizing strategic plans? What is the availability of information about those processes to the public?

In its assessment of the first question, the committee determined that the proposed role of the DGPCG was to prioritize evidence for review by USDA and HHS, with additional input from other federal agencies and the general public. The 2020 DGAC Scientific Report supported use of a “continuous model to identify and evaluate evidence as it is published” (DGAC, 2020). For the second question, the committee found that the DGAC provided recommendations for topics that should be considered by future advisory committees (DGAC, 2020).

The committee notes, however, that USDA and HHS recognized that “identifying the topics and questions prior to establishing the [DGAC] has potentially the greatest effect on the [DGAC’s] timeline” (see Appendix B-3). Moreover, “In the future, USDA and HHS hope to refine this step to continue to support maximizing the time the [DGAC] has available to examine the evidence” (see Appendix B-3).

Recommendation 1b

The 2017 National Academies report proposed that the TEPs should serve “as a flexible mechanism to supplement the technical insights in beginning stages of any type of evidence analysis” (NASEM, 2017a, p. 61). The TEPs were expected to include both federal and nonfederal experts with relevant methodologic expertise and a diversity of scientific view-

points. The committee developed and considered two questions relative to this recommendation

1. Which persons or groups performed these functions described above and how?
2. Did the approach used provide consultation during the evidence evaluation or were these persons or groups providing other services instead of or in addition to this one?

Recommendation 1c

The 2017 National Academies report described the DGSAC as being “charged with synthesizing and interpreting scientific evidence, as well as developing conclusions for USDA and HHS to consider in the DGA update” (NASEM, 2017a, p. 62). The report also envisioned that the DGSAC would integrate all data inputs (which could have come from food pattern modeling or new approaches to data analysis as well as from the systematic reviews prepared by NESR) or request new analyses (see Table 3-2). The committee developed and considered the following question:

1. How were the proposed functions of the DGSAC carried out?

TABLE 3-2 Comparison of Functions of the Dietary Guidelines Advisory Committee with the Proposed Dietary Guideline Scientific Advisory Committee^a

NOTE: DGAC = Dietary Guidelines Advisory Committee; DGPCG = Dietary Guidelines Planning and Continuity Group; DGSAC = Dietary Guidelines Scientific Advisory Committee; HHS = U.S. Department of Health and Human Services; TEP = technical expert panel; USDA = U.S. Department of Agriculture. SOURCES: DGAC, 2020; NASEM, 2017a.

^a The text in this table was modified after release of a prepublication version of the report to the sponsor. The title of this table was changed to refer to functions of the DGAC and proposed DGSAC to maintain consistency with the terminology used in the 2017 National Academies report. Information relating to the functions of the DGAC was modified to clarify the way in which the DGAC identified new topics and to remove a reference to a function of developing questions and topic briefs.

Findings for Recommendation 1

Prioritization of Topics

The committee found that the topics that were prioritized for the 2020 DGA cycle included those developed as part of the P/B-24 projects as well as those from prior DGA cycles that were relevant to other life stages (see Appendix B-3). USDA and HHS prioritized the list of topics with input from other federal agencies as well as public comment before the DGAC was established (see Appendix B-3). Following the public comment period, USDA and HHS released the final list of topics (see Appendix B-3).

Recommendation 1a

The committee’s analysis further revealed that none of the three groups recommended in the 2017 National Academies report was created. In light of the time frame for the cycle to develop the 2020–2025 DGA, this recommendation was issued too late in the process for implementation as envisioned for the DGPCG. Rather, USDA and HHS redistributed some of the current functions of the DGAC to “Federal nutrition scientists, existing Federal committees, and NESR technical expert collaboratives” (see Appendix B-5).

Both USDA and HHS viewed the recommendation as requiring the creation of at least two discretionary federal advisory committees, a time-intensive process that they regarded as infeasible under the circumstances (see Appendix B-5). However, they did redistribute some of the recommended functions to other groups. In general, USDA and HHS chose to look internally to the staff of other federal agencies for the various functions that the 2017 National Academies report had described would be carried out by ad hoc (i.e., Federal Advisory Committee Act-regulated) groups and the TEPs (see Appendix B-5).

Recommendation 1b

The committee’s analysis of evidence related to recommendation 1b found that in lieu of using a TEP, NESR conducted its systematic reviews directly with the 2020 DGAC (Stoody et al., 2019). For the second question, the committee’s analysis revealed that the process used by USDA and HHS did not provide consultation during the evidence evaluation in the way described in the 2017 National Academies report. The 2017 National Academies report proposed that TEPs could assist with developing the systematic reviews by, for example, “operationaliz[ing] the research questions formulated by the DGPCG by helping to set the eli-

gibility criteria for the literature review and to clarify systematic review question elements” (NASEM, 2017a, p. 61).

Recommendation 1c

The committee’s analysis found that interpretation of the scientific evidence and drawing conclusions were carried out by NESR and the DGAC, but their efforts were not partitioned as described in the 2017 National Academies report. Recommendation 1 was addressed by redistributing some of the current functions of the DGAC to “Federal nutrition scientists, existing Federal committees, and NESR technical expert collaboratives” (see Appendix B-5). As discussed later in this chapter, there was more collaboration between these NESR and the DGAC in the development of the 2020 DGAC Report than desirable to minimize the risk of bias and ensure transparency.

Conclusions for Recommendation 1

The committee concluded that prioritization of the topics for systematic reviews used in the 2020 DGA cycle was completed by federal staff, with input from public comments, before the DGAC began its deliberations. The committee concluded that, although the response by USDA and HHS to recommendation 1 is understandable under the circumstances related to the timing of the recommendation and resource constraints, it may have been possible to create greater differentiation between the roles of NESR and the DGAC. It may also have been possible to seek and use additional consultation from external experts while the evidence was being evaluated by the DGAC. Furthermore, the reorganization and separation of the functions of the DGAC into three separate groups (DGCPG, DGSAC, and TEPs) was not implemented. Some internal shifts in functions by the staff of federal agencies were made instead. Overall, the committee concluded that full implementation of recommendation 1 is essential if the guiding principles of the 2017 National Academies report

are to be achieved. Full implementation of this recommendation would provide improved transparency and reduce possible conflicts of interest, which are needed to establish public trust in the DGA.

Assessment of Recommendation 2

Background

The 2017 National Academies report recommended a clear explanation by USDA and HHS about which recommendations made by the DGAC are accepted, revised, or discarded (see Box 3-3, recommendation 2). This recommendation specifically focused on transparency when the 2020 DGAC Scientific Report and 2020–2025 DGA differ, as well as when USDA and HHS describe whether and why the DGA recommendations were different.

Review of Evidence

To review and assess the evidence on implementation of 2017 National Academies recommendation 2, the committee’s aim was to compare recommendations by the DGAC against the final recommendations in the DGA to determine if the differences were clearly explained by USDA and HHS. The committee’s criteria for identifying an area where potential differences may occur included asking whether a change to the DGAC recommendations would support the goal of encouraging healthy dietary patterns across the life cycle.

Specifically, the committee sought to determine whether there was transparency in the decision-making processes in developing the DGA. To carry out its assessment of the recommendation, the committee considered four questions:

1. Did USDA and HHS clearly identify differences between 2020 DGAC Scientific Report and the 2020–2025 DGA?
2. Did USDA and HHS identify a process they followed for deciding whether or not to accept any recommendations from the 2020 DGAC Scientific Report?
3. Were USDA and HHS clear and transparent in explaining why they did not accept any recommendation from the 2020 DGAC Scientific Report?
4. Did USDA and HHS identify a process they followed for deciding to include guidance in the 2020–2025 DGA that was not in the 2020 DGAC Scientific Report?

TABLE 3-3 Selected Examples of Potential Differences Between the 2020 Dietary Guidelines Advisory Committee Scientific Report and the 2020–2025 Dietary Guidelines for Americans

NOTES: DGA = Dietary Guidelines for Americans; DGAC = Dietary Guidelines Advisory Committee. Further information in Appendix D.

SOURCES: DGAC, 2020; USDA and HHS, 2020.

In the analysis of the first question, the committee assessed if USDA and HHS identified differences between the 2020 DGAC Scientific Report and the 2020–2025 DGA (DGAC, 2020; USDA and HHS, 2020). In an open session, USDA and HHS representatives stated that approximately 95 percent of the recommendations made by the DGAC were reflected in the 2020–2025 DGA, and that the recommendations that were not included were those for added sugars and alcohol only (NASEM, 2021a). The committee then examined whether the 2020 DGAC Scientific Report and the 2020–2025 DGA diverged in areas other than alcohol and added sugars. Comparisons between the recommendations in the 2020 DGAC Scientific Report and the final 2020–2025 DGA found several additional areas of potential differences (see Table 3-3; see Appendix D). Instances in which the DGAC made a recommendation that did not appear to be clearly presented in the 2020–2025 DGA (e.g., choice of high omega-3 fish at multiple life stages) and where the DGAC cited limited evidence in support of a given area of advice (e.g., omega-3 fatty acid supplements in pregnancy) were identified (DGAC, 2020; USDA and HHS, 2020). It was not the task of the committee to evaluate the scientific merits of the 2020 DGAC Scientific Report or the 2020–2025 DGA, but rather identify differences and determine whether these were adequately explained.

In its assessment of the first three questions, the committee looked to the public record to determine whether USDA and HHS provided a clear explanation for why the 2020–2025 DGA differed from the 2020 DGAC Scientific Report on the added sugars and alcohol recommendations. The following USDA and HHS response (DGA, n.d.-b) highlights why these two recommendations differ:

For the fourth question, the committee developed tables comparing selected examples of excerpts from the DGAC and DGA recommendations (see Table 3-3; see Appendix D) and identified instances where the

2020–2025 DGA included a recommendation for a subject that the 2020 DGAC Scientific Report did not discuss (e.g., vitamin and mineral supplement use in pregnancy, donor breast milk).

Findings for Recommendation 2

The committee found that USDA and HHS provided, in the cases of added sugars and alcohol, a clear, transparent, and public account that the 2020–2025 DGA recommendations differed from the recommendations in the 2020 DGAC Scientific Report and why. Consistent with federal law, the agencies highlighted the “preponderance of current scientific and medical knowledge” as the basis for decision making (USDA and HHS, 2020).

Conclusions for Recommendation 2

The committee concluded that recommendation 2 was partially implemented. Even though the 2020 DGAC Scientific Report (DGAC, 2020) and the 2020–2025 DGA (USDA and HHS, 2020) differed in some instances, efforts were made by the secretaries of USDA and HHS to explain their reasons for these differences. However, these explanations were not fully transparent. Consistent identification and justification for all differences between the 2020 DGAC Scientific Report and the 2020–2025 DGA would further enhance transparency.

Assessment of Recommendation 3

Background

The 2017 National Academies report recommended separating the roles of the USDA NEL staff and the DGSAC (see Box 3-4, recommendation 3) as a way to ensure that the process used by the NESR, formerly called NEL, follows best practice principles and a rigorous approach to the conduct of systematic reviews (NASEM, 2017a). This recommendation outlined an approach to assigning tasks between the NESR staff and the DGSAC and emphasized the importance of peer review in the systematic review process. As noted in the discussion of recommendation 1, the DGSAC was not created, so the analysis for this recommendation refers to the 2020 DGAC.

Review of Evidence

To review and assess the evidence on implementation of recommendations 3a, 3b, and 3c, the committee compared the time line of steps in the process to develop the 2020–2025 DGA with the NESR systematic

review process (see Chapter 2, Figure 2-2). Specifically, the committee’s aim was to determine whether the quality of the overarching systematic review process was followed consistently in the cycle to develop the 2020–2025 DGA. The committee further sought to elucidate the reasons for the decision by USDA and HHS not to implement some components of recommendation 3 in the 2017 National Academies report.

Recommendation 3a

In the discussion of recommendation 3a (see Box 3-4), the 2017 National Academies report indicates that in the systematic review process the TEPs should serve as “a mechanism to supplement the technical insights in the beginning stages of any type of analysis” (NASEM, 2017a, p. 11). The recommendation further indicates that the TEPs should include both federal and nonfederal experts with relevant methodologic expertise and a diversity of scientific viewpoints. To carry out its assessment of recommendation 3a, the committee considered three specific questions

1. Was a TEP constituted, and if so, was it the same or a different TEP that assisted in the conduct of systematic reviews for the P/B-24 questions and for all other questions?
2. Was the risk-of-bias process conducted by NESR, and if so what tools were used and did they align with best practices?
3. In what manner did the NSER staff assist the DGAC in the conduct of the systematic reviews?

In its analysis of the first question, the committee found that the P/B-24 systematic reviews grew out of the B-24 process conducted by the National Institutes of Health and NESR.⁵ This process identified and developed the systematic review questions that needed to be answered for children from birth to 24 months old to prepare for the inclusion of this age group in the DGA (Raiten et al., 2014). Subsequently, Congress passed the Agricultural Act of 2014,⁶ mandating the 2020–2025 DGA edition include guidance for infants, toddlers, and pregnant women. Therefore, the scope of B-24 expanded to include pregnant women (hence, P/B-24) (Stoody et al. 2019). To conduct the resulting systematic reviews, the NEL developed several Technical Expert Collaboratives (TECs),⁷ each with a set of specific topics to consider, which were composed of federal and nonfederal scientists (Obbagy, 2019; Stoody et al., 2019). The resulting systematic reviews were published in the peer-reviewed literature (Stoody et al., 2019). For the non-P/B-24 topics, the role of a TEP for establishing research questions, the systematic review framework, and the inclusion and exclusion criteria for each topic were done by NESR staff in collaboration with the 2020 DGAC (DGAC, 2020).

For the second question, “NESR used a dual, independent process for risk of bias assessments” (DGAC, 2020, Part C, p. 33). Additionally, for each article included, two NESR analysts independently completed the risk-of-bias tool appropriate for the study’s design. “The analysts’ responses were compared, and disagreements, if any, were discussed and reconciled. If a disagreement could not be resolved by the 2 analysts, an additional member of the NESR staff was asked to provide a third-party consultation” (DGAC, 2020, Part C, p. 34). The committee noted that the same NESR members who completed data extraction and created evidence tables with input from DGAC also did the risk-of-bias analysis (DGAC, 2020).

For the third question regarding how NESR assisted the DGAC in the systematic review process, it was noted in the report that “the [DGAC] makes all substantive decisions” during each step (DGAC, 2020), but most steps, except final conclusion statements, were contributed to by both

___________________

⁵ This text was modified after release of a prepublication version of the report to the sponsor to clarify that this project was a joint effort between the National Institutes of Health and NESR.

⁶ Agricultural Act of 2014. H.R. 2642; 113th Cong. (2014). See https://www.congress.gov/bill/113th-congress/house-bill/2642/text (accessed December 8, 2021).

⁷ The Technical Expert Collaborative is a group of experts that participate in NESR projects. This group is analogous to the TEP and is composed of both federal and nonfederal staff. It completes systematic reviews but does not provide advice or recommendations to the government. The TECs were used for the P/B-24 systematic reviews.

groups. Additional information is provided in the discussion of recommendation 3c below.

Recommendation 3b

To address recommendation 3b, the committee developed and considered the following question:

1. For peer review of the systematic reviews, how were reviewers selected and how was the process operationalized?

The committee’s review of evidence showed that for systematic reviews completed after release of 2017 National Academies report, a process for internal federal peer review was developed and coordinated by the co-executive secretary for the DGAC from USDA’s Agricultural Research Service (ARS) (DGAC, 2020). In this process, federal scientists were invited to carry out the peer review by first self-selecting their scientific areas of expertise (based on content expertise, not related systematic review methodology) (DGAC, 2020). All federal peer reviewers were provided with an overview of the NESR systematic review methodology used by the DGAC in conducting the reviews (DGAC, 2020). Then, the co-executive secretary from ARS selected two peer reviewers for each of the DGAC systematic reviews based on expertise (Klurfeld, 2020; see Appendix B). The peer reviewers were provided with a draft copy of the systematic review, and asked to peer review the document following procedures similar to those used by peer-review journals (DGAC, 2020; Klurfeld, 2020). The peer reviewers were anonymous to NESR, the DGAC, and the second assigned peer reviewer assignments were known only to the co-executive secretary from ARS (see Appendix B). The peer-review comments were reviewed initially by NESR staff, who addressed any editorial comments, then shared the comments with the DGAC, noting any substantive content-related comments (DGAC, 2020). The DGAC determined how to respond.

NESR supported them in making any necessary revisions and in preparing a response to the peer reviewers. The co-executive secretary from ARS was provided with these responses, and shared them with the respective peer reviewers (DGAC, 2020). The DGAC also used several existing NESR systematic reviews that were conducted as part of the P/B-24 project. Those reviews had been published in the American Journal of Clinical Nutrition, which underwent an external peer-review process (Obbagy et al., 2019; Stoody et al., 2019).

Recommendation 3c

To address recommendation 3c, the committee developed and considered the following question:

1. What were the specific roles of NESR and DGAC in the various steps of the systematic review process?

As noted in the 2017 National Academies report, the TEPs were⁸ expected to review and provide feedback to NESR to refine the systematic review materials. The DGAC was responsible for establishing protocols, synthesizing and grading evidence, and drafting conclusions in its systematic reviews. For the grading process, “predefined criteria … [were] used to evaluate and grade the strength of the evidence supporting each conclusion statement” (DGAC, 2020, Part C, p. 36), and then applied a criterion provided by NESR (DGAC, 2020). The NESR prepared a draft description which was of the studies included in each systematic review as a first step in the synthesis of the evidence. The DGAC then reviewed and synthesized the evidence and drafted conclusion statements (DGAC, 2020). NESR staff supported the process by drafting text and evidence tables that documented and provided transparency to the DGAC’s review and synthesis (DGAC, 2020).

Findings for Recommendation 3

Recommendation 3a

The committee’s analysis of the evidence on the NESR systematic review process to develop the 2020–2025 DGA found that, first, TEPs were not implemented for any of the systematic reviews conducted following the release of the 2017 National Academies recommendations. An external TEC was only constituted for the P/B-24 systematic reviews, which was carried out with NESR staff (Obbagy et al., 2019; Stoody et al., 2019). The reason for not establishing a TEP for the rest of the NESR systematic reviews, for the 2020–2025 DGA, was identified as not having sufficient time and resources as the DGAC had already been constituted (Stoody et al., 2019).

For the second question, the committee found that the risk-of-bias assessments were conducted by NESR using appropriate standard tools for randomized controlled trials (RCTs), non-RCTs, and observational

___________________

⁸ This text was modified after release of a prepublication version of the report to the sponsor to correct an error where the DGSAC was incorrectly mentioned instead of TEPs.

studies (DGAC, 2020). Limitations to systematic review practices identified by the committee are discussed in recommendation 4.

The 2017 National Academies recommendation that NESR assists the DGAC was implemented as the two groups worked collaboratively on many aspects of the systematic process outside of data extraction and assembling of the evidence tables (Obbagy, 2019). However, although each step of the systematic review process is described in the 2020 DGAC Scientific Report, the procedure does not provide for “a clear delineation of roles in order to minimize the introduction of bias and allow for an objective, evidence-based review” (NASEM, 2017a, p. 76), consistent with the 2017 National Academies recommendations.

Recommendation 3b

The committee found that each systematic review for the 2020–2025 DGA underwent a type of peer review, in contrast to previous DGA cycles in which no peer review was included in the process (DGAC, 2020). The committee noted that the NESR systematic review process began before the release of the 2017 National Academies report. For systematic reviews conducted after the release of 2017 National Academies report, NESR systematic reviews were reviewed internally to the federal government (by HHS, USDA, Department of Defense, and Department of Veteran Affairs) and the results were posted on the USDA website. However, no details were provided, for example, about whether a template for reporting of the reviews was provided to reviewers. Overall, the committee found an external peer-review process that would be consistent with the National Academies recommendation (NASEM, 2017a) was not implemented.

Recommendation 3c

The procedure described in the 2020 DGAC Scientific Report and in the NESR process suggest that NESR contributed to organizing the synthesis of the evidence as well as drafting the text for the synthesis of the evidence before it was submitted to the DGAC (DGAC, 2020). However, the committee was not able to determine whether NESR or DGAC played the larger role in summarizing and interpreting the evidence, and to what extent the process was consistent across the systematic reviews. Furthermore, drafting of the conclusion statement and grade assignments appear to have been completed solely by the DGAC. It is also unclear whether the process was consistent across systematic reviews.

Conclusions for Recommendation 3

The committee concluded that the recommended use of TEPs was not implemented. Regarding subrecommendations 3b and 3c, the process

of external peer review and the DGAC synthesizing and interpreting the results were both only partially implemented, which raises concerns about transparency. However, a clear delineation of roles (separation of function) between NESR (formerly NEL) and the DGAC was not evident in the 2020 DGAC Scientific Report. Further progress on this recommendation requires the formation of the three separate groups as described in recommendation 1.

Assessment of Recommendation 4

Background

One of the central tenets of the 2017 National Academies report is that the quality of the NESR (formerly NEL) systematic reviews underpins the strength of DGA recommendations. The 2017 National Academies report further states that “the NEL process for conducting original systematic reviews is thorough but has not been updated to reflect recent advances in systematic review methodology” (NASEM, 2017a, p. 28). The 2017 National Academies report also states that “one possible improvement would be to invite systematic review experts to periodically assess the NEL process, as well as to learn from other leading organizations (e.g., AHRQ [Agency for Healthcare Research and Quality] and Cochrane)” (NASEM, 2017a, p. 80). Although the application of best practices for conducting systematic reviews can differ according to the intent of the review and available literature that informs the review, there is general agreement on the elements of best practices. By examining practices from leading systematic review organizations, the committee compiled information on practices from authoritative groups, such as AHRQ, the Institute of Medicine (IOM), and Cochrane. These are described in detail in Appendix E and are summarized in Table 3-4. Given that recommendation 4 (see Box 3-5) focuses only on whether

TABLE 3-4 Summary of Committee-Identified Systematic Review Practices^a

NOTES: COI = conflict of interest; GRADE = Grading of Recommendations Assessment, Development and Evaluation; PICO = population, intervention/exposure, comparator, and outcome of interest. See Appendix E for further explanation.

SOURCES: Ayorinde et al., 2020; Chandler et al., 2021; Higgins et al., 2021; IOM, 2011; USPSTF, 2018.

^a The title of this table was modified after release of a prepublication version of the report to the sponsor to clarify that the table is the committee’s compilation of identified systematic review practices and that the list is not comprehensive.

the already thorough NESR systematic reviews remain up to date, the committee’s assessment highlighted only those items where there were differences between NESR’s processes and committee-identified practices (see Appendix E).

Review of Evidence

The 2017 National Academies recommendation 4 included four subrecommendations to ensure and enhance the training of NESR staff, promote continuous quality improvement, engage external experts in a peri-

odic review of the NESR review process, and introduce new advances in technology as they become available. In its assessment of recommendation 4, the committee reviewed the evidence on progress in each of these four areas (see Box 3-5, recommendation 4).

The committee compared the NESR processes to those used by leading organizations, which are described here as “committee-identified practices.” The committee also compared NESR systematic review practices from the cycles to develop the 2015–2020 and 2020–2025 DGA to determine if the changes made to the process align more closely with committee-identified practices as recommended in the 2017 National Academies report (see Appendix F). NESR has maintained a Continuous Quality Advancement (CQA) program to ensure that staff are adequately trained.⁹ The committee also reviewed the CQA program and other resources identified in an open dialogue with USDA and HHS (see Appendix B-3–B-5). The committee’s assessment focused on the extent to which USDA and HHS addressed the subrecommendations 4a, 4b, 4c, and 4d related to training, engagement/collaboration with external experts, external expert evaluation, and technology investment.

Recommendation 4a

Recommendation 4a was a process recommendation for training the NEL/NESR staff in alignment with best practices. The section of this training plan describing roles and responsibilities explains that

Although the NESR website indicates that training occurs, documentation to assess the extent of the training program was not identified. Ideally, documentation would be provided that describes a systematic approach to the overall training program (e.g., orientation and ongoing professional development) and the outcomes of the training program (e.g., specific kinds of training completed and improvements made to NESR processes).

___________________

⁹ See https://nesr.usda.gov/continuous-quality-advancment (accessed December 8, 2021).

¹⁰ See https://nesr.usda.gov/roles-and-responsibilities (accessed December 8, 2021).

Recommendation 4b

Recommendation 4b relates to enabling engagement and learning from external groups to promote best practices. As described above, a CQA program is briefly described on the USDA website and additional information relevant to this recommendation was provided to the committee in an open dialogue with USDA and HHS (see Appendix B-3). NESR forms CQA interest groups, which are small, self-directed teams of NESR staff that are asked to complete a specific time-limited assignment that addresses a topic related to evidence synthesis methodology (see Appendix B-3). USDA and HHS stated

In its dialogue with the committee, USDA indicated that it is routinely in contact with other organizations that conduct systematic reviews concerning nutrition or dietary guidance (see Appendix B). The NESR staff also noted the Cochrane Library, Health Canada, and the Scandinavian government agencies as expert groups that they have studied (NASEM, 2021b), although no specific examples of the topics that were discussed or improvements that were made based on these collaborations or from the CQA interest groups were provided.

Recommendation 4c

Recommendation 4c concerns the use of external experts with specific expertise in the systematic review process to periodically review the NESR process in the context of best practices. The committee reviewed the processes used by NESR staff to incorporate input received from these experts. The details of the NESR CQA program, as described on the NESR website, identify the 2017 National Academies reports in their external review by experts.¹¹ In addition, NESR has a long-standing, robust after-action analysis that is informed by participants in the systematic review process. This after-action analysis has identified potential improvements in the process, which has been documented by peer-reviewed publications (Bailey et al., 2021; Dewey et al., 2021; NASEM, 2017a; Stoody et

___________________

¹¹ See https://nesr.usda.gov/continuous-quality-advancment (accessed December 8, 2021).

al., 2019). A comparison of the methodology from the cycle to develop the 2015–2020 DGA (HHS and USDA, 2015) with the cycle to develop the 2020–2025 DGA (USDA and HHS, 2020) identified the changes that were implemented (see Appendix F). The NESR staff made several updates to its systematic review methods between the process to develop the 2015–2020 and 2020–2025 DGA (see Table 3-5).

Recommendation 4d

Recommendation 4d concerns the need to invest in technological infrastructure aligned with best practices. Updates included implementation of a new software program, DistillerSR, for managing search results and data extraction of systematic review for the 2020 DGA cycle (DGAC, 2020).

Findings for Recommendation 4

The committee found that the majority of the NESR methods are similar to committee-identified practices (see Appendix E). However, there are still some important differences (e.g., peer review, management of systematic review protocols, processes for updating existing systematic reviews, and description of process for committee deliberations).

Recommendation 4a

Although the objectives of the 2017 National Academies report were placed on the NESR website, the committee was not able to determine which professional development activities were systematically selected to ensure continued alignment of NESR processes with best practices. Overall, the committee was unable to assess the extent of the implementation of recommendation 4a.

Recommendation 4b

The committee found that recommendation 4b was partially implemented with concerns about transparency. USDA stated they routinely interact with other agencies that conduct and use systematic reviews (Stoody et al., 2019). However, the nature of collaboration with other leading organizations conducting systematic reviews and the work completed by the CQA to refine NESR methods continually is not clear. Additional documentation that provides the topics selected by the interest group and outcomes from the agency’s work would be helpful for determining the effectiveness of the CQA program.

Recommendation 4c

The committee found that recommendation 4c appears to have been partially implemented with concerns about transparency. However, further documentation of protocols and the accomplishments of USDA’s CQA program, and the improvements made based on findings of their after-action report, would be needed to evaluate progress. The committee could only estimate changes by comparing methods from the process to develop the 2015–2020 DGA and the 2020–2025 DGA (see Table 3-5). Although updates to NESR practices occurred between the cycles to produce the 2015–2020 and 2020–2025 DGA, it was not clear to this committee whether NESR training practices are sufficient to ensure updating of these practices. USDA considered the 2017 National Academies report to be the periodic review by external experts (Stoody et al., 2019). Evidence

TABLE 3-5 Examples of Methodological Changes to the Process to Develop the 2020–2025 Dietary Guidelines for Americans from the Process to Develop the 2015–2020 Dietary Guidelines for Americans^a

NOTES: DGA = Dietary Guidelines for Americans; DGAC = Dietary Guidelines Advisory Committee; GRADE = Grading of Recommendations Assessment, Development and Evaluation; HHS = U.S. Department of Health and Human Services; NESR = Nutrition Evidence Systematic Review; PICO = population, intervention/exposure, comparator, and outcome of interest; USDA = U.S. Department of Agriculture. Additional information in Appendix F.

^a The title of this table was modified after release of a prepublication version of the report to the sponsor to clarify this is not a comprehensive list.

of invitation of other experts to evaluate specific aspects of the NESR methodology is not publicly available at this time.

Recommendation 4d

The committee found that technological infrastructure changes between the process to develop the 2015–2020 DGA and the 2020–2025 DGA included the shift to using the DistillerSR program (DGAC, 2020). Only limited evidence of other investment or requests for funding for technological infrastructure was available at the time this report was completed (see Appendix B-3).

Conclusions for Recommendation 4

The committee concluded that the NESR systematic review processes were generally aligned with committee-identified practices. Although the committee was unable to assess the implementation of the effectiveness of the training fully, outcomes from the CQA, and benefits of collaboration with other experts and investments in technology, indicate that the NESR practices included recent improvements. Notably, there are opportunities to enhance the transparency and, thus, the perceived integrity of the systematic review through external peer review as well as improved management of systematic review protocols (including more clearly defined processes for updating existing systematic reviews and fully describing the process for creating DGAC recommendations). Updating these processes as well is essential for ensuring the integrity of the DGA.

Assessment of Recommendation 5

Background

The 2017 National Academies recommendation 5 addressed enhancing food pattern modeling to reflect the complex interactions involved, variability in intakes, and range of possible healthful diets (see Box 3-6, recommendation 5). Food pattern modeling is used to translate nutrient requirements into food combinations and estimate how well various eating patterns meet the DRIs, and recommendations in the DGA (NASEM, 2017a). Food pattern modeling, as carried out in the DGA process, was found to address an important but narrow set of questions with appropriate methodologies. However, it was limited by the food groupings, assumptions, and constraints of the models. The 2017 National Academies report recommended the use of systems science to enable more key questions involving different assumptions to be addressed (NASEM, 2017a).

The 2017 National Academies report noted that food pattern modeling has focused on representing the overall population using population energy and nutrient requirements, typical food choices, a traditional set of food groups, and that its applicability to individuals who follow a different consumption pattern has been limited (NASEM, 2017a). To illustrate, food choices can vary by cultural or other factors, with some individuals or groups not consuming some foods. The 2017 National Academies report suggested that advancing the methods used to account for the complexity of dietary exposures would strengthen the understanding of relationships between diet and health outcomes. Such an approach would also account for variability in food patterns, “providing flexibility in food and taste preferences, cultural norms, and other individual factors” (NASEM, 2017a, p. 86).

In the 2017 National Academies report, complex systems models were suggested to represent the mechanisms involved and to enable consideration of a greater array of food combinations and amounts, as well as to address the complex mechanisms and dynamic characteristics of diets (NASEM, 2017a). Deterministic models that employ set estimates for inputs were recommended to be replaced by stochastic systems models, such as simulation models, that account for variability and uncertainty, including variability in food composition and consumption patterns (NASEM, 2017a). Sensitivity analyses were also recommended to address the use of average composition values, which reflect nutrient values associated with food groups rather than information on variability in food composition, and to assess combinations of recommendations to meet nutrient targets (e.g., excluding dairy foods or replacing wheat with rice as a primary grain food) (NASEM, 2017a).

Lee et al. (2021) suggested that the difference between top-down modeling, which uses statistical methods to identify possible patterns, associations, and correlations, versus bottom-up systems modeling, which aims to represent the actual cause-and-effect mechanisms involved, is important for food pattern modeling. The 2017 National Academies report

included recommendations for both types of food pattern modeling, noting that accounting for variability and uncertainty using stochastic systems models could enhance the transparency of the dietary recommendations (NASEM, 2017a). Similarly, the 2017 National Academies report indicated food pattern modeling could make better use of the most current evidence available, be more flexible, and reflect new research.

Finally, the 2017 National Academies report recommended that TEPs be employed to supplement the expertise of food pattern modeling groups, specifically, to help verify key assumptions in the development of food patterns. This would include making the modeling results available for the DGAC when it first convenes (NASEM, 2017a). The implementation of the TEPs to inform the cycle to produce the 2020–2025 DGA is addressed above in recommendations 1, 3, and 4.

Review of Evidence

To review and assess the evidence on food pattern modeling, the committee utilized the DGAC report to clarify whether food pattern modeling had been improved in the process to develop the 2020–2025 DGA (DGAC, 2020). The committee then sought to elucidate the reasons for not implementing some elements of the recommendation. To address these issues, the committee considered the following four questions:

1. Were efforts made to better reflect the complexity of dietary exposures within the food pattern modeling?
2. Was the food pattern modeling enhanced to address variability in intakes?
3. Did the food pattern modeling address a range of possible healthful diets?
4. Are the assumptions underlying the modeling transparent and flexible?

The 2020 DGAC Scientific Report identified assumptions, strengths, and limitations of food pattern modeling. The 2020 DGAC Scientific Report indicated the food pattern modeling was based on profiles of nutrient-dense foods and U.S. population-based dietary intake data, with two key assumptions (DGAC, 2020):

1. Population-based patterns reflect the evidence on the relationship between diet and health in ways that might be adopted by the American public; and
2. Modeling tests assume population-wide compliance with all food intake recommendations; food pattern modeling is hypothetical and does not predict the behaviors of individuals.

Additionally, the 2020 DGAC Scientific Report identified that using a life-stage approach allows for examination of opportunities to identify specific advice for improving intakes within specific age-sex subgroups, going “beyond universal advice for adopting a healthy eating pattern to specific, tailored, and actionable advice on small but meaningful cumulative changes to typical food intake patterns common for a given life stage” (DGAC, 2020, Part D, p. 3).

In the food pattern modeling procedure, nutrient profiles for food groups were based on a weighted average of nutrient-dense forms of foods (DGAC, 2020). For the 2020–2025 DGA, a life-stage approach was adopted to test how proposed changes to food pattern components affect the number of servings from a food group as well as nutrient adequacy across the life span (see Appendix B-5). Nutrient profiles were tailored to each life-stage group, and for those ages 2 and older, 12 energy intake levels and over 30 nutrients were considered¹² (DGAC, 2020). Modifications were tested, including the amount of food of the number of servings from the food groups; food group nutrient profiles; and inclusion and exclusion of certain foods or food groups (e.g., vegetarian) in a given pattern (DGAC, 2020). As well, current data on dietary intakes (2015–2016 What We Eat in America [WWEIA]/National Health and Nutrition Examination Survey [NHANES]) (USDA, 2020) and food composition data (Food and Nutrient Database for Dietary Studies 2015–2016, Food Patterns Equivalents Database 2015–2016, National Nutrient Database for Standard Reference Release 28)¹³ were used in the modeling (DGAC, 2020; see Appendix B-2).

The 2020 DGAC Scientific Report noted that, although the USDA food patterns do not address cultural variations in intake patterns, they do offer flexibility for tailoring to cultural and taste preferences (DGAC, 2020). This is because, although the food group and subgroup amounts are prescriptive, the types of foods to be consumed are not. The report also identified the need to understand how to incorporate beverages that are not part of the USDA food groups or subgroups into the food pattern modeling (DGAC, 2020).

For each food pattern modeling question, the DGAC developed and made available a protocol with an analytic framework describing the scope, approach, and data and analyses to be conducted. The protocols were posted online, discussed at public meetings of the DGAC, and made

___________________

¹² This text was modified after release of a prepublication version of the report to the sponsor to clarify that these nutrient profiles are for those ages 2 and older. Nutrient profiles for toddlers age 12–23 months differed.

¹³ See https://www.ars.usda.gov/northeast-area/beltsville-md-bhnrc/beltsville-human-nutrition-research-center/food-surveys-research-group (accessed December 8, 2021).

publicly available on the DGA website (DGA, n.d.-c). In the 2020 DGAC Scientific Report the key definitions (including food groups and subgroups, food pattern components, nutrient profiles, item cluster, nutrient-dense representative food, typical choice representative food, essential calories, solid fats, and added sugars) were noted along with the general process for developing and updating the USDA food patterns (DGAC, 2020).

Findings for Recommendation 5

The committee’s analysis for the first question found that, in the 2020 DGAC Scientific Report, food pattern modeling was used to assess the ability to meet nutrient recommendations for each life stage through variations in USDA food patterns and the relationship between consumption of added sugars and intake recommendations (DGAC, 2020). Additionally, food pattern modeling informed the development of two new food patterns for children 12–23 months of age (DGAC, 2020). The committee found that the DGAC relied on the top-down modeling approaches used in previous DGA cycles. These approaches did not incorporate systems modeling, as discussed below under recommendation 7. Moreover, the committee noted that although food pattern modeling can be used to examine hypothetical scenarios, it can also be used to examine and explore nonhypothetical situations, such as better understanding the relationships between different factors, or identifying gaps in data.

For the second and third questions, the committee found that, using the traditional modeling approach, refinements were made to account for variability in intakes. The committee further found that the 2020 DGAC Scientific Report stated that the need to add or modify the USDA food patterns was explored based on systematic reviews (DGAC, 2020). Three USDA food patterns were carried forward, namely the Healthy U.S. style, the Healthy Vegetarian style, and the Mediterranean style (DGAC, 2020).

To the fourth question, the committee found that the assumptions for the food pattern modeling were made transparent through the protocols and analytic frameworks posted online and discussed at DGAC meetings (DGA, 2020).

Conclusions for Recommendation 5

Some enhancements were made to the food pattern modeling process, particularly to address variability in intakes. There was also attention to transparency in the food pattern modeling protocols and assumptions. However, the analytic methods used in the process to develop the DGA did not change substantially from the process to develop the 2015–2020 DGA. The committee concluded that recommendation 5 was only par-

tially implemented, which presents serious concerns about adequate rigor of the guidelines given the diversity of food patterns in the U.S. population. Moreover, this is an important missed opportunity to create a better informed and more useful product.

Assessment of Recommendation 6

Background

The 2017 National Academies report recommended that methods and criteria for establishing nutrients of concern, which are based on the prevalence of inadequate and excess intakes in the population (often relative to the DRIs [IOM, 2006]), should be standardized (see Box 3-7, recommendation 6). Specifically, the 2017 National Academies report identified the need to improve “the accuracy and efficiency of data analyses … by standardizing and validating the processes used, both within and between DGA cycles to identify nutrients of concern” (NASEM, 2017a, p. 15). The report further noted:

In addition, the report noted that “descriptive data analyses provide key insights to understanding the context and landscape of dietary patterns and population health and disease, including both current intakes and prevalence of disease” (NASEM, 2017a, p. 86).

Additionally, the 2017 National Academies report noted that TEPs could supplement the expertise of groups conducting descriptive data analyses (e.g., to refine research questions) with the goal of having results available for the DGAC when it first convenes. The DGAC would then independently evaluate the evidence and develop conclusions, consulting with the appropriate methodologists when needed (NASEM, 2017a).

The implementation of the TEPs to inform the 2020–2025 DGA cycle is addressed above in recommendations 1, 3, and 4.

Review of Evidence

The committee reviewed and assessed the available evidence on the implementation of recommendation 6 (see Box 3-7). Specifically, the committee aimed to determine whether the methods and criteria used to identify nutrients of concern improved from the previous cycle to develop the 2015–2020 DGA. The committee recognized that some of the recommendations are amenable to implementation in the near term, including standardization of terminology, definitions and thresholds for nutrients of concern, and documentation of descriptive data analyses commonly used over previous DGA cycles. However, others require serial application and the expansion of research to identify biomarkers reflective of nutrient intake and chronic disease. In addition, the committee notes that recommendation 6 pertained to within and between cycles but the assessment was limited to the cycle to develop the 2020–2025 DGA (see Chapter 2, Figure 2-2).

The committee considered the following four questions in its analysis of recommendation 6:

1. Were the terminology, thresholds, analytic methods, and interpretation related to nutrients of concern standardized and validated within and between cycles?
2. Where biochemical and health-related data were integrated, was this done in a systematic and consistent manner?
3. Were improvements in efficiency incorporated by identifying questions earlier and initiating data analyses before convening the DGAC and by documenting descriptive data analyses used across previous DGACs?
4. Was there transparency regarding the analytic methods and interpretation related to nutrients of concern?

The committee reviewed data from Bailey et al. (2021), who proposed a framework (see Figure 3-1) that was used by the DGAC to harmonize DGA terminology and provide quantitative thresholds for prevalence estimates obtained from dietary data as well as streamline the work of future DGACs. In the Bailey et al. (2021) paper, the phrase nutrients or food components (NFCs) was proposed instead of the term nutrients to incorporate non-nutrient food components, such as fiber, and be more consistent with the 2017 National Academies report definition of non-nutritive food substances. The paper further proposed the phrase NFCs of public health

concern to consolidate the terms under-consumed, over-consumed, public health concern, and public health significance (Bailey et al., 2021).

The Bailey et al. (2021) paper then identified nutrients or food components that posed special challenges, a term used to identify at-risk groups for which it was difficult to develop recommended intake levels. To establish consistent quantitative thresholds as a means of identifying nutrients of concern, the authors defined under-consumed as “equal to or greater than 5 percent of the population or in specific groups relative to EAR [Estimated Average Requirement], AI [Adequate Intake], or other quantitative authoritative recommendations from the diet alone” (Bailey et al., 2021, p. 1199). An over-consumed NFC was defined as “consumed in potential excess of the UL [Tolerable Upper Intake Level] CDRR [Chronic Disease Risk Reduction Intake], or other quantitative authoritative recommendation by equal to or greater than 5 percent of the population, or in specific groups from the diet alone” (Bailey et al., 2021, p. 1199).

While acknowledging that the 5 percent cutoff is arbitrary for inadequacy or excess, the authors determined this to be sufficiently low to have adequate sensitivity as a screening criterion (Bailey et al., 2021). These arbitrary thresholds will need further evaluation, including for their consequences for different uses. The committee identified the three-pronged approach for key data sources used by the 2020 DGAC, also drawing upon Bailey et al. (2021). This approach consisted of

1. Dietary intake data from WWEIA/NHANES;
2. Biological endpoints; and
3. Clinical health consequences (prevalence of health conditions measured directly or indirectly through validated surrogate markers related to NFC exposures).

To account for special nutritional needs, the NFC analysis focused on a life-stage approach (i.e., infancy, early childhood, pregnancy and lactation, adolescence, reproduction, and aging). The analyses also examined race/ethnicity and income (Bailey et al., 2021).

The evaluation of dietary intake data included examinations of means and distributions of the usual intake of energy, macronutrients, and other selected food components. Analysis of these data included the consideration of individuals aged 1 year and older, pregnant and lactating individuals, and infants receiving human milk or those receiving infant formula (Bailey et al., 2021). The National Cancer Institute method was used to estimate usual intake distributions, appropriately accounting for within-person variation in intake (Bailey et al., 2021; NCI, 2009). Prevalence of potential inadequacy or excess was based on the DRIs or other authoritative recommendations when such standards were available (Bailey et al.,

2021). Intake from food and beverages alone was first examined and when available, distributions of total usual intake including dietary supplements were also examined (Bailey et al., 2021). If available, the DGAC also considered biological endpoints or validated surrogate endpoints (DGAC, 2020). For most concentration biomarkers, the authors relied on 15- to 18-year-old data in the Second National Report on Biochemical Indicators of Diet and Nutrition from NHANES 2003–2006, as these biomarkers have not been collected in nationally representative data since that time (Bailey et al., 2021). With respect to biochemical and health-related data, the DGAC considered dietary metrics and biochemical or clinical indicators and associated health conditions (Bailey et al., 2021).

To assess the prevalence of health conditions, the DGAC considered nutrition-related conditions and how they vary by sex, age, race/ethnicity, income level, and weight status (DGAC, 2020). Chronic disease conditions that were considered included cardiovascular disease; metabolic syndrome, prediabetes, and diabetes; growth, size, and body composition, including overweight and obesity; reduced muscle strength and bone mass; gestational diabetes mellitus; chronic liver disease; cancer; dental health; and food allergies (DGAC, 2020).

Bailey et al. (2021) acknowledges that the development and implementation of the framework (see Figure 3-1) also led to the identification of gaps in data, including those related to the DRI, the availability of biomarkers, the lack of alignment between cutoffs for biomarkers and the DRI, lack of data for infants and pregnant and lactating individuals, an emphasis on foods and beverages alone rather than total intake inclusive of supplements, and the need for updated biochemical estimates.

The Data Analysis Team, including staff from USDA’s Agricultural Research Service and HHS’s Centers for Disease Control and Prevention and National Institutes of Health, coordinated data needs for the DGAC, including NHANES analyses (DGAC, 2020). Before convening the 2020 DGAC, federal staff documented and refined the process for identifying nutrients of public health concern and initiated discussions regarding data analyses, including those specific to pregnancy and lactation (see Appendix B-5). USDA and HHS also noted that they explored other federal processes for evaluating potential nutrients of concern, such as the U.S. Food and Drug Administration’s Evaluation Process of Public Health Significance of Essential Vitamins and Minerals, to standardize methods and criteria for establishing nutrients of concern across governmental agencies (see Appendix B-5). In addition to documenting the three-pronged approach in the peer-reviewed literature, a protocol that described the approach to identifying nutrients of public health relevance and that detailed the data and subsequent analysis to be conducted was presented and discussed at DGAC meetings (DGA, 2020). The commit-

tee’s understanding was that there was reliance on the 2017 National Academies report for initially documenting “all the descriptive data analyses commonly used across previous DGACs” (NASEM, 2017a).

Findings for Recommendation 6

The committee’s analysis of the evidence on the methods and criteria for establishing nutrients of concern found the recommendation was partially implemented and there were some enhancements to the process to develop the 2020–2025 DGA. For the first question, the committee found that the 2020 DGAC Scientific Report and a later peer-reviewed publication documented a framework that standardized the terminology, thresholds, analytic methods, and interpretation related to nutrients and food components of public health relevance (Bailey et al., 2021; DGAC, 2020). This built on the approach used by the 2015 DGAC and proposed an approach for future committees, which would address the recommendation to standardize both within and between cycles. The current thresholds for identifying nutrients of concern, however, are arbitrary according to Bailey et al. (2021), and will need further evaluation, including for their consequences for different uses.

For the second question, the committee found that the framework builds on the integration of biochemical and health-related data (Bailey et al., 2021), though it is noted that up-to-date biochemical data are lacking.

For the third question, the committee found that the DGAC, working with federal partners, initiated the identification of potential NFCs of public health concern prior to examining data. They established a three-pronged approach which included a framework to identify nutrients of public health relevance that detailed the data and subsequent analysis to be conducted (Bailey et al., 2021). However, the federal agencies have not publicly documented the descriptive data analyses commonly used across previous DGACs. Lastly, with respect to the fourth question related to transparency, the committee found that a protocol was developed with

the 2020 DGAC and posted to DietaryGuidelines.gov for public input before any data were reviewed and synthesized (see Appendix B).

Conclusions for Recommendation 6

The 2020 DGAC Scientific Report documented and published a peer-reviewed framework standardizing the terminology, thresholds, analytic methods, and interpretation related to nutrients of concern, building on the approach used by the 2015 DGAC and proposing an approach for future committees. The 2020–2025 DGA initiated data analyses early in the process and also showed greater transparency. Therefore, the committee concluded that recommendation 6 was nearly completely implemented with only minor remaining concerns.

Assessment of Recommendation 7

Background

The 2017 National Academies report recommended integrating more systems approaches into the DGA process (see Box 3-8, recommendation 7). This recommendation is overarching and crosscutting, stemming from the number of complex systems that affect diets, dietary behaviors, and dietary intake, as well as the ways that various nutrients may affect both short- and long-term health outcomes. The 2017 National Academies report offered an example of how complex systems are involved in the relationship between saturated fatty acid intake and coronary artery disease. It described how excessive saturated fatty acid intake can lead to lipid deposition within blood vessel walls, which in turn may initiate a cascade of inflammatory and immune reactions resulting in coronary artery disease, and how this pathway is straightforward with multiple intermediate steps and potential modifying factors (NASEM, 2017a, p. 95).

The DGA process begins with the data that are available and then looks for trends in those data. Deterministic statistical and epidemiological approaches can help identify associations and trends and make some predictions, but they do not necessarily elucidate nor represent the actual mechanisms involved (Macy and Willer, 2002). Randomized or controlled trials may be able to answer specific efficacy questions but may focus on particular factors and processes without representing all or even most of the interactions and mechanisms that are operative in the real world (i.e., a given complex system).

The cultural shift described in the 2017 National Academies report would require using approaches and methods that better account for all the different components and processes in a system (NASEM, 2017a). When systems approaches and methods are fully integrated into the DGA

process, they could influence the DGA recommendations based on an expanded knowledge of the diet–health relationships of interest, inform the translation of the guidelines to maximize impact, and identify relevant connections across stakeholders (NASEM, 2017a).

The 2017 National Academies report acknowledged that it may require a few years for systems approaches to be incorporated in the DGA process and that this process could be gradual and iterative (NASEM, 2017a). But it also suggested that USDA and HHS should start as soon as possible to integrate systems approaches and methods into the DGA process. To illustrate, the report recommended beginning to assemble relevant data, identifying modelers with appropriate experience and expertise, and identifying, assembling, and developing initial systems maps and/or models (NASEM, 2017a).

Review of Evidence

To review and assess the evidence on systems approaches that were incorporated into the process to develop the 2020–2025 DGA, the committee broke the recommendation down into two components: (1) supporting and advancing the integration of systems science into nutrition science that will become part of the future evidence base for DGA, and (2) integrating specific systems mapping and modeling strategies to inform and enhance the DGAC process. Specifically, the committee sought to elucidate any planning for—or implementation of—this long-term shift in the types of science that would be conducted and the evidence that would be considered by the DGAC. The committee developed and considered two questions in relation to this recommendation:

1. How was recommendation 7 implemented?
2. What were the challenges associated with implementing recommendation 7?

In its analysis for the first question, the committee found that although a rationale has been provided for how innovative methodological approaches (including systems science) could be adopted by the DGAC, no evidence for starting any specific process for integration of systems methods and data has been provided by USDA and HHS. USDA and HHS have stated that they have done some initial exploration into systems approaches and will continue to explore this option (NASEM, 2021a; see Appendix B).

Although the 2020 DGAC Scientific Report notes that HHS and USDA have enacted many of the 2017 National Academies recommendations, it also acknowledged a lack of progress on the systems science recommendation, and thus encouraged further work:

In its analysis of the second question, the committee reviewed materials that revealed a number of real or perceived challenges, which explain the lack of progress on recommendation 7 (see Appendix B-3). These challenges included limitations in staffing, expertise, funding, as well as time line challenges that restricted the ability of USDA and HHS to start planning a long-term process for integrating systems thinking and methods into the DGA process. USDA and HHS also stated that “In short, the barriers have included time, resources, and the need to develop the methodology for informing guidance on diet and health” (see Appendix B-3).

The most specific explanation provided for the lack of progress on the systems science recommendations is a lack of funding. In a response to dialogue with the committee, USDA and HHS stated that they do not have the funding to do this work, but they have been requesting funding since fiscal year 2018:

The 2020 DGAC Scientific Report recognized the ambitious nature of the goals laid out in recommendation 7, noting that there will be a need to draw on broader scientific expertise: “The nature of the recommendation acknowledges that moving to incorporate systems thinking will require different types of scientific expertise outside of nutrition and food science to evaluate diet and health relations in a broader body of evidence” (Schneeman et al., 2021, p. 1056). In addition to new scientific expertise, the DGAC points out that other branches of the federal government will need to be involved (DGAC, 2020). Finally, expertise to guide this process also takes the form of new types of data. Although USDA and HHS do not specify what kinds of data they would most like to have access to, they reported to the committee that “availability of federal datasets” is an important consideration (see Appendix B-3).

The committee’s dialogue with USDA and HHS indicated the perception that systems methods in nutrition science are not advanced enough to

be integrated into the DGAC process; “the DGA are mandated to reflect the preponderance of scientific evidence, and as such, emerging methodologies can be explored but only integrated when they are advanced enough to support the development of national nutrition public health guidance” (see Appendix B-5). Additionally, the agencies stated that “agencies adopt new methodologies when they are advanced enough to support national nutrition public health guidance,” which was done when they adopted food pattern modeling into the 2005 DGAC (see Appendix B5).

Findings for Recommendation 7

Although the DGAC, USDA, and HHS acknowledge the importance of the recommendation to integrate systems science into the nutritional evidence base and the DGAC processes, the committee found that no discernable planning or implementation activities have started, other than requests for additional funding (see Appendix B).

Conclusions for Recommendation 7

The committee’s analysis of evidence about developing and implementing systems approaches into the DGA thus concluded that recommendation 7 was not implemented. The committee recognized that implementation of recommendation 7 would be a long, ongoing process, with the goal of integrating a systems approach into a variety of processes used to develop the DGA. The committee notes that steps need to be initiated soon as implementation of recommendation 7 is critical to the expansion of the DGA to cover the life span, include individuals living with chronic health conditions, and account for dietary diversity when developing dietary guidelines that capture the heterogeneity of the American population.

OVERARCHING CONCLUSION

Overall, the committee found that USDA and HHS took several steps toward implementing the seven 2017 National Academies recommendations. However, the extent of implementation was not uniform within and across the seven recommendations. The changes that have been made have focused on those recommendations and subcomponents that could most readily be implemented from the time of the release of the 2017 National Academies report to that of the 2020–2025 DGA (USDA and HHS, 2020). Further changes will be needed during the preparation of the 2025–2030 DGA to implement the seven National Academies 2017 recommendations completely.

REFERENCES

IOM. 2011. Clinical practice guidelines we can trust. Washington, DC: The National Academies Press. https://doi.org/10.17226/13058.

Klurfeld, D. 2020. 2020 Dietary Guidelines Advisory Committee: Meeting 5. March 12–13, 2020 https://www.dietaryguidelines.gov/meeting-5 (accesssed December 8, 2021).

Lee, B. Y., M. C. Ferguson, S. N. Cox, and P. H. Phan. 2021. Big data and systems methods: The next frontier to tackling the global obesity epidemic. Obesity 29(2):263–264.

Macy, M. W., and R. Willer. 2002. From factors to actors: Computational sociology and agent-based modeling. Annual Review of Sociology 28(1):143–166.

NASEM (National Academies of Sciences, Engineering, and Medicine). 2017a. Redesigning the process for establishing the Dietary Guidelines for Americans. Washington, DC: The National Academies Press.

NASEM. 2017b. Optimizing the process for establishing the Dietary Guidelines for Americans: The selection process. Washington, DC: The National Academies Press.

NASEM. 2021a. Evaluating the process to develop the Dietary Guidelines for Americans, 2020–2025 Committee Meeting 1. https://www.nationalacademies.org/event/08-12-2021/evaluating-the-process-to-develop-the-dietary-guidelines-for-americans-2020-2025-committee-meeting-1 (accessed December 8, 2021).

NASEM, 2021b. Evaluating the process to develop the Dietary Guidelines for Americans, 2020–2025 Committee Meeting 4. https://www.nationalacademies.org/event/09-03-2021/evaluating-the-process-to-develop-the-dietary-guidelines-for-americans-2020-2025-committee-meeting-4-open-session (accessed December 8, 2021).

NCI (National Cancer Institute). 2009. Usual dietary intakes: SAS macros for the NCI method. http://riskfactor.cancer.gov/diet/usualintakes/macros.html (accessed December 8, 2021).

Obbagy, J. 2019. Nutrition evidence systematic review. Presentation to the Dietary Guidelines Advisory Committee: Meeting 1, Washington, DC.

Obbagy, J., J. M. Spahn, Y. P. Wong, T. L. Psota, M. K. Spill, C. Dreibelbis, D. E. Gungor, P. Nadaud, R. Raghavan, E. Callahan, L. K. English, B. L. Kingshipp, C. C. LaPergola, M. J. Shapiro, and E. E. Stoody. 2019. Systematic review methods for the Pregnancy and Birth to 24 Months Project. American Journal of Clinical Nutrition 109(Suppl 1):698S–704S.

Raiten, D., R. Raghavan, A. Porter, J. E. Obbagy, and J. M. Sphan. 2014. Executive summary: Evaluating the evidence base to support the inclusion of infants and children from birth to 24 mo of age in the Dietary Guidelines for Americans—“the B-24 Project.” American Journal of Clinical Nutrition 99(3):663S-691S.

Schneeman, B. O., J. D. Ard, C. J. Boushey, R. L. Bailey, R. Novotny, L. G. Snetselaar, J. M. de Jesus, and E. E. Stoody. 2021. Perspective: Impact of the National Academies of Sciences, Engineering, and Medicine report on the process for the 2020 Dietary Guidelines Advisory Committee. Advances in Nutrition 12(4):1051–1057.

Stoody, E. E., J. M. Spahn, and K. O. Casavale. 2019. The pregnancy and birth to 24 months project: A series of systematic reviews on diet and health. American Journal of Clinical Nutrition 109(Suppl 1):685S–697S.

USDA (U.S. Department of Agriculture). 2020. What We Eat in America (WWEIA), NHANES: Data tables. Food Surveys Research Group. https://www.ars.usda.gov/northeast-area/beltsville-md-bhnrc/beltsville-human-nutrition-research-center/food-surveys-research-group/docs/wweia-data-tables (accessed December 8, 2021).

USDA and HHS. 2020. Dietary Guidelines for Americans, 2020–2025, 9th ed. https://dietaryguidelines.gov (accessed December 8, 2021).

USPSTF (U.S. Preventive Services Task Force). 2018. Standards for guideline development. https://www.uspreventiveservicestaskforce.org/uspstf/about-uspstf/methods-and-processes/standards-guideline-development (accessed December 8, 2021).