Appendix D

Open Session Materials

During the process of information gathering, the committee held open session with members of the U.S. Department of Agriculture (USDA) and the U.S. Department of Health and Human Services (HHS), as well as members of the 2020 Dietary Guidelines Advisory Committee and the chair of the 2017 National Academies report Redesigning the Process for Establishing the Dietary Guidelines for Americans (DGA). Before the sessions, the committee submitted questions that were either answered during the open sessions, or answered via written responses provided below. The open session agendas are provided below as well as slides presented at the sessions containing pertinent information to this report.

This appendix contains the following:

• D-1: Meeting 4 Open Session Questions for USDA and USDA Responses to NASEM
• D-2: Meeting 32 Open Session Agenda
• D-3: Meeting 34 Open Session Agenda
• D-4: Meeting 34 Open Session Questions for USDA/HHS and USDA Responses to NASEM
• D-5: Meeting 34 Open Session—Select Slides

D-1: MEETING 4 OPEN SESSION QUESTIONS FOR USDA AND USDA RESPONSES TO NASEM [NATIONAL ACADEMIES OF SCIENCES, ENGINEERING, AND MEDICINE]

Below is an outline of questions from the committee for the upcoming open session:

1. Identify milestones in the DGA process:
   1. What was the initial time line for the DGA process compared to the time line after implementing NASEM report recommendations?
      1. How did the initial plan (with milestones) change with implementation of NASEM 2017 recommendations?
      2. How did the initial plan (with milestones) change for the systematic review process?

Response: Each edition of the DGA is released on a 5-year cycle. Throughout this 5-year cycle, USDA and HHS work together to support development and implementation of the current edition and planning for the next edition. Additionally, the Nutrition Evidence Systematic Review (NESR) team and food pattern modeling staff are standing entities that support the committee’s work and, in the interim, conduct other projects and continuous quality advancement activities.

At the beginning of each cycle, the Departments outline a time line. There are two elements of the time line that are fixed: (1) the DGA must be released at least every 5 years and (2) the committee is convened under a charter not to exceed 2 years. Each cycle, the Departments aim to maximize the time the committee has to review the scientific evidence and provide the departments as much time as possible to write and release the DGA.

USDA and HHS planned the 2020 process, and its time line, after taking into consideration the NASEM recommendations, along with other inputs. As such, there was not an “initial plan” that was changed. However, for comparative purpose, the table below provides dates associated with some of the major milestones of the process used to establish the last four editions of the DGA.

Dates Associated with Major Milestones of the Process Used to Establish the 2005, 2010, 2015, and 2020 Editions of the DGA

^a National Academies’ two reports on the DGA development process published in February and September 2017.

^b A 34-day federal funding gap (i.e., “government shutdown”) delayed the process to establish the 2020 Dietary Guidelines Advisory Committee.

^c The committee’s report release was delayed 1 month because of the COVID-19 pandemic.

For the 2020 process, there were many factors that affected the time line. A few of these are outlined below:

• At the initiation of the process, the National Academies’ reports were released in February and November 2017. Because the Dietary Guidelines Advisory Committee is one of approximately 1,000 federal advisory committees managed under the Federal Advisory Committee Act, the study recommendations required substantial review and discussion across the departments, including with committee management officials and legal and ethics counsel.
• At the initiation of the process, the National Academies’ reports from December 2018 to January 2019, there was a federal lapse in funding (i.e., a “government shutdown”), which delayed the announcement of the committee.
• At the initiation of the process, the National Academies’ reports beginning in March 2020 through the release of the 2020 edition, the COVID-19 pandemic affected the process in several ways, including a delay in the release of the committee’s report.

In addition to time, two other resources are important in the development process: funding and staff. The DGA is an unfunded mandate, and the 2020 edition was the first to receive congressional funding of $12.3 million over 3 years for development and release. Additionally, over 65 federal staff supported the committee’s review process and an additional 45 federal staff conducted a peer review of the NESR systematic reviews. Funding and staff are important elements to consider when working within a fixed time line, and with the increasing scope of the 2020 edition to include the birth to 24 months (B–24) population, these additional resources were key to ensuring a transparent, science-driven process within the 5-year time line.

Several of the NASEM recommendations encouraged separation of responsibility, which have time line implications:

• In some cases, the NASEM recommendations supported work already underway. For example, NESR systematic reviews were underway as part of the Pregnancy and Birth to 24 Months Project and were published as the committee began its work.
• Additionally, prior to convening the committee, federal staff explored food pattern modeling activities for individuals who are pregnant or lactating, documented and refined the process for identifying nutrients of public health concern, and met to begin discussing data analysis needs.
• Of note, identifying the topics and questions prior to establishing the committee has potentially the greatest effect on the committee’s

• time line. As a comparison, the 2015 committee spent approximately 6 months identifying topics and questions. Having the topics and questions in advance allowed the 2020 committee to initiate its review of evidence upon being established. In the future, USDA and HHS hope to refine this step to continue to support maximizing the time the committee has available to examine the evidence.

2. How were the DGA topics selected following the public comment period? What health outcomes were considered for B–24? How was chronic disease approached? Was it considered from a risk of disease or a disease state? Can the committee obtain the public comments submitted about the DGA process?

Response: Sent separately (9/1/21) (See NASEM, 2022)

1. Which topics were predetermined (e.g., B–24, chronic disease) and which were identified from the interests of the DGAC members and from the comments received from the public?

Response: In the previous response to this inquiry (9/1/21), USDA and HHS sent: (1) the initial list of topics and questions identified by the departments and posted for public comment and (2) the final list of topics and questions provided to the 2020 committee. Many of the questions identified were examined by previous DGACs or in the Pregnancy and Birth to 24 Months Project and were prioritized for inclusion using the following criteria: relevance, importance, potential effect on federal programs, and avoiding duplication.

Following agency and public comment, the general scope of the topics and questions remained similar. One of the main differences was to expand the life stages addressed in a number of topics and questions that had previously focused on a specific subset of life stages. For example, initially, the questions on dietary patterns consumed during childhood and adolescence only addressed the outcomes of growth, size, and body composition and bone health; whereas for adults, questions on dietary patterns addressed the outcomes of risk of cardiovascular disease, type 2 diabetes, and certain types of cancer. Following public and federal agency comment, the dietary patterns questions were revised and expanded to address dietary patterns and all health outcomes at each stage of life.

Additionally, some health outcomes were added to address outcomes of interest in the public and agency comments. Of note, neurocognitive health and all-cause mortality were outcomes added following the comment period.

Finally, some questions were added following public and agency comments to address areas of uncertainty. For example, the 2015 committee did not complete reviews on alcoholic beverages and health outcomes, and this was identified as an area requiring updated review. Additionally, the initial list of topics and questions did not include questions on added sugars and health outcomes, but in response to comments, USDA and HHS added questions considered by the 2015 committee on added sugars and risk of overweight/obesity, cardiovascular disease, and type 2 diabetes.

The 2020 committee received the full list of questions identified by USDA and HHS to consider in its review. The committee’s charter stated, “The committee will limit its review and advice to dietary guidance for human nutrition on the topics and scientific questions specified by the departments.” Ultimately, owing to time constraints, the committee prioritized questions from that full list, and did not complete all of the questions posed by the departments. The 2020 committee did not add additional questions beyond those provided by the departments.

3. What was the process and methodology for the 2020 systematic reviews? How were existing systematic reviews updated? Were there changes in the search strategies from the previous DGA update? Did search strategies change for all or for a subset of systematic reviews? How were the 2015 systematic reviews used in the process?

Response: Sent separately (9/1/21) (See NASEM, 2022)

4. Peer-review process for the systematic reviews: Was there a defined methodology for peer reviews? How were the reviewers chosen, and how were the systematic reviews assigned to them; for example, did the reviewers each see all of the systematic reviews or was there some other process? Is the review process published? What were the specifications for the review process?

Response: Sent separately (9/1/21) (See NASEM, 2022)

5. Food modeling process: What was the process used in food modeling? Are there multiple groups of scientists involved? If so, how were their contributions used together? What is the plan for food modeling in future editions of the DGA?

Response: Sent separately (9/1/21) (See NASEM, 2022)

6. How do the federal agencies identify when new evidence was needed for a DGA topic?

Response: When identifying topics and questions for the 2020 committee, the existence of new evidence was not a formal criterion used in selecting or prioritizing topics for review. However, federal scientists who have expertise in various topic areas are engaged in the topics-and-questions process and provide feedback on the availability of new evidence. Based on input received from the 2020 committee, the 2017 NASEM report, and new techniques used by other systematic review organizations, NESR is currently conducting continuous evidence monitoring and using evidence scans to understand the volume and characteristics of evidence available on a topic or question. This information will be useful in determining when existing reviews should be updated or when there is enough evidence available on a topic or question to proceed with a review.

1. What tools are used, and what plans are in place for incorporating new evidence into a systematic review?

Response: Sent separately (9/1/21) (See NASEM, 2022)

2. When do data need to be analyzed in innovative ways to prepare for a DGA review?

Response: Innovative approaches for examining the evidence are identified in between committee reviews. There are many rationales why new approaches are developed in advance of a committee being in place. First, DGACs are time-limited, discretionary federal advisory committees, and their time is needed to focus on the review of diet and health. Second, resources, research, and staff are needed to develop and refine methodologies. Finally, the DGA are mandated to reflect the preponderance of scientific evidence, and as such, emerging methodologies can be explored but only integrated when they are advanced enough to support the development of national nutrition public health guidance.

3. How do the agencies decide that it is time to adopt new methodological approaches to developing the DGA?

Response: As nutrition science has evolved, so have the approaches for examining scientific evidence. For example, since the 2005 committee, food pattern modeling analysis was included as a method to review evidence, and since the 2010 committee, DGACs have conducted original systematic reviews with support from USDA’s NESR (formerly Nutrition Evidence

Library [NEL]). The agencies adopt new methodologies when they are advanced enough to support national nutrition public health guidance. How this decision is made depends on the process. For example, for the past several years, NESR, through their Continuous Quality Advancement initiative, has been exploring when and how to integrate meta-analysis into their systematic reviews. For other methodological approaches, such as systems modeling, integration of new methods involves adequate funding, availability of federal data sets, and discussion among staff at USDA and HHS, including input from standing federal committees, such as the Dietary Guidelines Subcommittee of the Interagency Committee on Human Nutrition Research.

7. From what has been published to date, it appears that there were barriers to including systems modeling in the most recent edition of the DGA. What were these barriers, and what are your plans, if any, for addressing them?

Response: In short, the barriers have included time, resources, and the need to develop the methodology for informing guidance on diet and health. As noted previously, the DGA is an unfunded mandate. Funds have been requested, but not secured, to explore this methodology. The departments plan to continue to request funding to explore this approach.

8. Similarly, what influenced decision making around implementation of a continuous process improvement strategy?

Response: USDA and HHS have supported the development of the DGA since 1980 and have utilized a Dietary Guidelines Advisory Committee (DGAC) since 1985. Additionally, the fact that the process occurs on a regular 5-year cycle allows the departments to plan ahead, reflect on lessons learned, keep abreast of consensus best practices in guidance development, follow other processes for developing guidance, and maintain staff who have the institutional knowledge to continue to advance the process. As a result of our continuous process improvement strategy, USDA created the NESR (formerly NEL) prior to initiating the 2010 committee. When NESR was launched, its methodology was modeled on that of the Academy of Nutrition and Dietetics’ Evidence Analysis Library.

NESR’s Continuous Quality Advancement (CQA) initiative, which has been ongoing since NESR was launched, is critical for ensuring that we remain at the forefront of food- and nutrition-related evidence synthesis, and have the capacity to maintain, and expand, our high-quality, mission-critical work. Our CQA initiative involves the routine evaluation and

refinement of our methodology and tools to ensure that processes remain state of the art and aligned with other organizations, such as Cochrane, AHRQ [Agency for Healthcare Research and Quality] and NIH [National Institutes of Health]/NIEHS [National Institute of Environmental Health Sciences]’s National Toxicology Program, as well as other international organizations that support the development of evidence-based dietary guidance. When appropriate and feasible, refinements to our methodology are carefully planned, tested, and adopted. CQA work results in timely updates to NESR’s methodology, as well as to staff training materials. NESR’s CQA initiative also includes outreach activities designed to disseminate information about us and our work to a range of audiences.

NESR’s CQA work also aligns with and is guided by a recommendation from the 2017 National Academies report, which encouraged the secretary of USDA to ensure all NESR reviews align with best practices by:

• Enabling ongoing training of the NESR staff,
• Enabling engagement with and learning from external groups on the forefront of systematic review methods,
• Inviting external systematic review experts to periodically evaluate the NESR’s methods, and
• Investing in technological infrastructure.

NESR CQA is accomplished by “Interest Groups,” which are small, self-directed teams of NESR staff that are tasked to accomplish a specific time-limited assignment addressing a specific evidence synthesis-related topic. These topics are identified and selected based on after-action input from experts and NESR staff following the completion of each NESR review project, monitoring of evolutions occurring in the fields of systematic review and nutrition science, and from other key resources, such as the 2017 National Academies report.

NESR’s CQA model allows our staff to build capacity and expertise in a wide range of topic areas. Interest group work always involves a landscape analysis of other systematic review organizations’ approaches to the topic, followed by a thorough exploration of whether and how these other approaches can be integrated into NESR’s approach or tailored to meet NESR’s needs. CQA often involves engagement with topic-specific experts from those organizations to discuss, ask questions, and get input on NESR’s approach. Whenever feasible, NESR explores how technology can be used to enhance the efficiency and rigor of its work.

See “Advancing NESR Methodology” on NESR’s website at https://nesr.usda.gov/continuous-quality-advancment.

REFERENCE

D-2: MEETING 32 OPEN SESSION AGENDA

Evaluating the Process to Develop the Dietary Guidelines for Americans, 2020–2025

Meeting 32 Open Session

June 6, 2022

3:00–4:00 PM ET

D-3: MEETING 34 OPEN SESSION AGENDA

Evaluating the Process to Develop the Dietary Guidelines for Americans, 2020–2025

Meeting 34 Open Session

June 23, 2022

3:00–4:00 PM ET

D-4: MEETING 34 OPEN SESSION QUESTIONS FOR USDA/HHS AND USDA RESPONSES TO NASEM

Below are the questions from the committee they would like to discuss at the open session:

1. When the National Academies 2017 report was released, were there practical and other barriers to creating the Dietary Guidelines Planning and Continuity Group (DGPCG), Technical Expert Panels (TEPs), and Dietary Guidelines Scientific Advisory Committee (DGSAC)?
   1. Are there barriers to implementing these recommendations now?
   2. Please describe how you will address underlying barriers that impede the full implementation of recommendation 1.
   3. Given the effectiveness of the Technical Expert Collaboratives (TECs) for the Pregnancy and Birth to 24 Months project, what were the barriers to implementing the TEPs, which seem similar to the TECs, as proposed in recommendation 1?

Response: See Appendix D-5.

2. How do USDA and HHS decide to recommend something different in the DGA than the DGAC concluded?
   1. Is there a formal review process with specific criteria by which they compare DGAC conclusions against the DGA to identify such differences?
   2. For the 2020–2025 DGA, how did the agencies decide to specifically highlight differences in alcohol and added sugar intake?
   3. How did the agencies identify other differences between DGAC conclusions and DGA recommendations that they did not highlight?

Response: See the open session recording and all presentation slides at: https://www.nationalacademies.org/event/06-23-2022/evaluating-the-process-to-develop-the-dietary-guidelines-for-americans-2020-2025-committee-meeting-34-open-session

3. The NASEM 2017 report recommended that USDA separate the roles of the NEL (now NESR) from those of the DGASAC (still implemented as the DGAC) relative to conducting, reviewing, and synthesizing the evidence from the systematic reviews created for the development of the DGA. Do you anticipate being able to establish a DGSAC as part of the 2025–2030 DGA process?

1. 1. If not, how do you plan to address separating the roles of NESR and the DGAC in handling the systematic reviews?

Response: See Appendix D-5.

4. What actions were taken to develop a plan for integrating systems thinking and data into the DGA process?
   1. What actions have been taken to build capacity in systems science?
   2. What specific resources will be needed (staff, time, funding)?

Response: See the open session recording and all presentation slides at: https://www.nationalacademies.org/event/06-23-2022/evaluating-the-process-to-develop-the-dietary-guidelines-for-americans-2020-2025committee-meeting-34-open-session

Below are the questions the committee would like a written response to:

5. What is the process for approving a new federal advisory committee and the process for re-chartering a federal advisory committee?
   2. How long do these processes take?
   3. Who is responsible for doing this?

Response: The process for approving a new federal advisory committee or re-chartering a federal advisory committee are the same.

Requirements for Establishing Advisory Committees

The process for seeking approval and appointment of a new federal advisory committee is extensive and requires several months to a year. Under the Federal Advisory Committee Act (FACA), advisory committees can be created only when they are essential to the performance of a duty or responsibility conveyed upon the executive branch by law or Presidential Directive.

Advisory Committee Appointment

Agency officials, members of Congress, the general public, or professional societies may nominate potential candidates for membership on a committee. Selection of committee members is made based on the FACA’s requirements and the potential member’s background and qualifications. Final selection of committee members is made by the President or heads of departments or agencies. Prior to accepting an appointment with a federal advisory committee, each prospective member should clarify his/her role, obligations, duties, allowable expenses, compensation limitations, and any ethics requirements with their committee’s Designated Federal Officer and/or Committee Management Officer, as appropriate.

Role of the GSA Under the FACA

The General Services Administration (GSA) monitors committee activities government-wide. As part of this responsibility, GSA responds to requests from agencies on establishing new committees or the renewal of existing groups, among other activities. The GSA and the federal community work to eliminate the overlap or duplication of advisory bodies, terminate unnecessary or inactive committees, and develop committee management regulations, guidelines, and training in response to requirements of the Executive Branch and Congress.

Role of Committee Management Officer and Designated Federal Officer The FACA also provides that each agency sponsoring a federal advisory committee must appoint a Committee Management Officer to oversee the administration of the Act’s requirements. In addition, a Designated Federal Officer must be assigned to each committee to ensure compliance with the FACA, and any other applicable laws and regulations.

6. Given that the DGA is an “unfunded mandate,” the cost of operating these committees could also be important. How would the relative costs of running the DGPCG and DGSAC compare to running the current DGAC?
   1. What are the major drivers of any projected differences in cost?

Response: The unfunded mandate puts the Departments in the position of seeking funding every cycle when the leadership role transitions to the other Department. In FY2019, CNPP [Center for Nutrition Policy and Promotion] requested and received the first Congressional appropriations to develop the Dietary Guidelines for Americans, 2020–2025. The request was funded at $12,297,000 over three fiscal years. The funding supported contract staffing (including contractors to support systematic reviews, data analysis, and communications activities), DGAC operations (e.g., travel to public meetings, public meeting facilities, report development), development and maintenance of DietaryGuidelines.gov and NESR.usda.gov, development of the DGA publication, and limited MyPlate communications activities.

The estimated cost to HHS of supporting the 2025 DGAC, which performs functions similar to the proposed DGSAC, is over $1,000,000 per year. These funds are specific to DGAC operations (e.g., travel to public meetings, public meeting facilities, report development). HHS currently does not have funding for FY23, when the 2025 DGAC is expected to begin work. HHS requested $1,000,000 in funding in the FY23 President’s budget to support the 2025 DGAC work.

The major cost drivers greatly depend on the duties of the advisory committee. It is estimated that costs to support the DGSAC and the

DGPCG would each be similar to costs for supporting the DGAC, resulting in a doubling of the costs to the Departments over a 5-year cycle. Staff support and costs associated with transparency (e.g., public meetings and website maintenance and enhancements) are the major driver of costs for the DGAC. The DGSAC and DGPCG are both likely to require staff support, public meetings, and web maintenance and enhancements at similar levels to the current DGAC. Doubling financial support for federal advisory committees to support the DGA would not be realistic when funding is not mandated for the support of the DGAC.

7. It might be useful to have a document that matches the DGAC report recommendations with the DGA recommendations, indicating how they align or differ and, in cases where there are differences, explaining why this might be the case. What would be barriers or challenges to creating such a summary document?

Response: The DGAC’s scientific report is a detailed technical document on the current state of nutrition science on specific topics for HHS and USDA DGA developers. The DGA is an evidence-based policy document for programs, policy makers, and health professionals. They have different purposes and different audiences. Additionally, the DGAC scientific report and the DGA address topics in multiple places and through different approaches. The 2020 DGAC had conclusion statements for individual questions and then integrated that evidence to inform their advice to the Departments. The Committee provided their advice in Part B integration chapters and in Part D chapter summaries. This advice informed the DGA writing team and peer-review process.

The DGA writing team included Federal staff who supported the DGAC. To aid in the writing process, staff extracted the Committee’s advice to the Departments from its report and provided it to the writing team to consider along with the previous edition of the DGA and public and federal agency comments as they undertook the writing process. Additionally, the draft DGA was peer-reviewed by additional Federal staff who supported the DGAC process, members of the 2020 DGAC, and additional external peer reviewers. The peer reviewers were asked to consider the scientific basis of the recommendations and were referred to the 2020 DGAC’s scientific report.

The Departments clearly documented where the DGA “omit or accept only parts” (per the 2017 NASEM report recommendation) of the DGAC’s advice on DietaryGuidelines.gov: https://www.dietaryguidelines.gov/about-dietary-guidelines/related-projects/usda-hhs-response-nationalacademies-sciences-engineering. DGA peer reviewers did not comment

on other inconsistencies between the DGAC report and DGA. Additionally, following release of the 2020–2025 DGA, three 2020 DGAC members coauthored a commentary. In this commentary, they acknowledged the differences in recommendations for added sugars and alcoholic beverages, but they also noted that: “Overall, there is strong agreement between the new DGA and 2020 DGAC report, in terms of both message and philosophy.” And “In total, the 2020–2025 DGA and the 2020 DGAC are aligned in the promotion of healthy dietary patterns across the life span, emphasizing healthy weight. The advice to “Make every bite count” is sensible and actionable.” See https://doi.org/10.1093/ajcn/nqab092.

It is also important to note that the Departments were transparent in acknowledging that one criterion for prioritizing questions for the DGAC was avoiding duplication. In the DGA, the Departments include information from (and reference) existing Federal resources that were not examined by the DGAC, such as the EPA [Environmental Protection Agency]/FDA [Food and Drug Administration] Advice about Eating Fish, MyPlate.gov, the DRIs [Dietary Reference Intakes], and numerous food safety resources (e.g., foodsafety.gov). Additionally, an important input into each edition of the DGA is the preceding edition, as noted in the 2020–2025 DGA (pg 11): “Each edition of the Dietary Guidelines builds on the preceding edition, with the scientific justification for revisions informed by the Committee’s scientific report and consideration of public and Federal agency comments.”

USDA and HHS aimed to be transparent by (1) posting information at DietaryGuidelines.gov about where the DGA “omit or accept only parts” of the advice from the DGAC and (2) citing additional reference documents in the DGA. To aid with more comparison in the future, HHS and USDA will continue to work with future DGACs to clearly document its advice to the Departments in its report. Additionally, USDA and HHS can continue to explore ways to enhance transparency in the writing process.

8. The NASEM 2017 report recommended that USDA align the NESR process with best practices when feasible by four methods: staff training, collaborating with other leading agencies, inviting experts to evaluate the process, and using technology. Previously, you indicated this was accomplished through an active continuous quality advancement program with self-directed workgroups.
   1. Please identify the topics addressed by these CQA workgroups and tell us more about the specific outcomes that resulted from their work.

Response: Systematic review science and supporting technologies evolve continuously. Progress in the field of nutrition science related to the

design, conduct, and analysis of food- and nutrition-related research is also ongoing. To ensure that NESR’s process remains state-of the-art in these ever-changing environments, NESR methodology and tools are routinely evaluated and refined through a continuous quality advancement (CQA) initiative. This CQA initiative is critical for ensuring NESR remains at the forefront of food- and nutrition-related evidence synthesis, and has the capacity to maintain, and expand, to meet the demand for high-quality, mission-critical work.

Of note, NESR takes the view that within the field of systematic review, there is not a single, consensus, “gold standard” method that all organizations should seek to adopt or emulate. This view is supported by and consistent with the recommendations of the NASEM 2017 report (“Conduct of original systematic reviews will need to be transparent and follow state-of-the-art methods.....However, this National Academies committee believes the NEL and DGSAC need to have the flexibility to align with appropriate standards or methods and does not recommend any one standard be adopted, which may be subject to change and evolve over time.”). Rather, we believe that NESR requires the flexibility to adapt to the evolving fields of systematic review and nutrition science—with a focus on aligning with best practices. It is our view that “best practices” applies to the context in which an organization conducts its reviews. Thus, “best practice” is not a one-size-fits-all approach, but rather the development and use of methods that are rigorous, transparent, and minimize bias—and that are tailored to the specific purpose and context of the work. Thus, NESR seeks to learn from and collaborate with a broad range of systematic review organizations as it develops and advances its methodology—taking care to understand the rationale underlying various methodological approaches. Then, when we undertake CQA work to advance or develop our own methods, we seek to align with others, but also tailor our methods to ensure applicability to public health nutrition decision-making in the United States.

NESR’s CQA initiative has been ongoing since our inception, and is accomplished through three phases:

• Implementation: NESR implements its methodology for conducting systematic reviews, rapid reviews, and/or evidence scans in a project.
• Evaluation:
  • Following each project, NESR conducts an evaluation to identify, prioritize, and select topic areas on which to focus CQA activities. This evaluation involves consideration of input and recommendations from our team, collaborators, stakeholders, and end users; recent methodological advancements made in

• • the fields of systematic review and nutrition science; and the emergence of new technologies to support evidence synthesis work.
    • Priority is placed on CQA topics that will enhance the rigor and transparency of NESR’s work, promote diversity of expertise and a deliberative process, while managing potential biases and conflicts of interest.
  • NESR then forms “Interest Groups (IGs)” which are small, self-directed teams of NESR staff that are tasked to accomplish a specific time-limited assignment addressing a specific topic.
    • IGs typically begin their work by reviewing the “as-is” methodological approach used by NESR, and any evolutions to NESR’s methodology that occurred previously. Then, the IG will conduct a landscape analysis, documenting and reviewing methodological approaches used by other organizations with expertise and experience in systematic review methodology, nutrition science, and/or public health. This landscape analysis involves review of methodology manuals, websites and training materials, and published literature, as well as dialogue with experts and methodologists.
    • Next, the IG will conduct a comparative analysis of NESR’s approach to others used in the field, documenting whether NESR’s approach aligns with others, and where it does not. This serves as the basis for the refinement phase.
• Refinement: Based on information gathered during the evaluation phase, NESR plans, tests, and adopts refinements to its methodology. These refinements are intended to ensure that NESR’s methodology aligns with current practices in the field of systematic review and is tailored to meet the unique requirements of public health nutrition, as well as NESR’s role within the U.S. Federal government.

CQA work is accomplished by leveraging the expertise of and collaborating with methodologists from other leading systematic review organizations, tailoring established methods for use in the field of public health nutrition, expanding technological infrastructure, and enhancing NESR team knowledge and skills through ongoing training and professional development.

Below is a table that lists the major CQA activities NESR has undertaken since working with the 2015 DGAC (2016-2021)—including CQA activities

conducted prior to working with the 2020 Committee and CQA activities undertaken since completion of the 2020 Committee’s systematic reviews. We have noted the goal of the group, outcomes that resulted from the work, and what external groups we engaged with or learned from as part of the activity. However, this is not a complete list of all advancements made by NESR, nor does it reflect advancements that occurred prior to 2017.

TBD = to be determined.

2. Please describe any other improvements or new technologies identified through the CQA program that you could not implement in the 2020 DGAC cycle. If so, what were the barriers to implementing these improvements or new technologies?

Response: In the 2017 NASEM report, Recommendation 4 specified that the Secretary of USDA should ensure that all NESR systematic reviews align with best practices by:

1. Enabling ongoing training of the NEL staff,
2. Enabling engagement with and learning from external groups on the forefront of systematic review methods,
3. Inviting external systematic review experts to periodically evaluate the NESR’s methods, and
4. Investing in technological infrastructure.

NESR has had ongoing, long-term support of our CQA initiative, which has been in place since NESR’s inception. NESR has been encouraged and sup-

ported by all levels of leadership within the agency to seek out and invest in training, collaborations with other groups in the field, opportunities to have NESR’s methods evaluated, and technology to enhance our processes.

We have not identified any improvements or technologies through our CQA program that could not be implemented. That means, whenever we have undertaken a CQA activity, we have evaluated our methodology and that of others, and adopted the necessary refinements our team determined were necessary to enhance NESR’s methodology.

For example, with regard to our investment in technological infrastructure, the major technology NESR determined it needed was the Distiller software. In 2017, NESR conducted a market analysis of screening and data-extraction technologies, including evaluation of machine-learning capabilities. The market analysis included evaluation of products from Evidence Partners/Distiller, Covidence, Eppi Reviewer 4, SWIFT/ActiveScreener, and SRDR. This analysis was used to determine that DistillerSR software best met NESR’s needs, and a contract was implemented for use in the systematic reviews conducted by the 2020 DGAC—a contract that has been in place and expanded per NESR’s licensing needs. NESR uses Distiller to screen the results of literature searches, extract data, and conduct risk of bias assessments. NESR has also invested in developing, launching, and maintaining a public-facing website, https://nesr.usda.gov. NESR has also taken advantage of freely available software that did not require a financial investment, such as SWIFT, active-screener, and R statistical software. Finally, given NESR’s position within the Federal government, we have access to USDA’s full technological infrastructure (i.e., Microsoft applications [e.g., Word, Excel, PowerPoint, OneNote], citation-management tools [i.e., Endnote], video and phone conferencing [e.g., Microsoft Teams, Webex, Zoom], and cloud-based shared drives [e.g., OneDrive]).

3. How do you determine whether your CQA program effectively aligns NESR’s practices with the best practices used by others?

Response: NESR accomplishes CQA through “Interest Groups (IGs)” which are small, self-directed teams of NESR staff that are tasked to accomplish a specific time-limited assignment addressing a specific topic.

IGs typically begin their work by reviewing the “as-is” methodological approach used by NESR, and any evolutions to NESR’s methodology that occurred previously. Then, the IG will conduct a landscape analysis, documenting and reviewing methodological approaches used by other organi-

zations with expertise and experience in systematic review methodology, nutrition science, and/or public health. This landscape analysis involves review of methodology manuals, websites, and training materials, and published literature, as well as dialogue with experts and methodologists. Next, the IG will conduct a comparative analysis of NESR’s approach to others used in the field, documenting where NESR’s approach aligns with others and where it does not. This serves as the basis for the refinement phase.

Based on information gathered during the evaluation phase, NESR plans, tests, and adopts refinements to its methodology. These refinements are intended to ensure that NESR’s methodology aligns with current practices in the field of systematic review and is tailored to meet the unique requirements of public health nutrition, as well as NESR’s role within the U.S. Federal government.

Of note, NESR takes the view that within the field of systematic review, there is not a single, consensus, “gold standard” method that all organizations should seek to adopt or emulate. Rather, we believe that NESR requires the flexibility to adapt to the evolving fields of systematic review and nutrition science—with a focus on aligning with best practices. It is our view that “best practices” applies to the context in which an organization conducts its reviews. Thus, “best practice” is not a one-size-fits-all approach, but rather the development and use of methods that are rigorous, transparent, and minimize bias—and that are tailored to the specific purpose and context of the work. Thus, NESR seeks to learn from and collaborate with a broad range of systematic review organizations as it develops and advances its methodology—taking care to understand the rationale underlying various methodological approaches. Then, when we undertake CQA work to advance or develop our own methods, we seek to align with others, but also may tailor our methods to ensure applicability to public health nutrition decision-making in the United States.

That said, we have had numerous opportunities in recent years for external systematic review, nutrition science, and public health experts to evaluate NESR’s methods. Examples include the 2017 NASEM committee, after-action activities conducted with TECs from the P/B-24 project and the 2020 DGAC, and through the current NASEM committee’s work. We have also published 23 peer-reviewed publications of our work since 2019, including 20 systematic reviews and 3 methodology papers, which afforded many opportunities for review and evaluation of our methodology. We’ve also been invited to present and discuss our methods with a variety of audiences, including many with systematic review methodol-

ogy expertise, such as the Cochrane Colloquium, the EPA Systematic Review Community of Practice, International Liaison Group (ILG) on nutrient reference values (NRVs), the Nordic Nutrition Recommendations (NNR), and the National Health and Medical Research Council (NHMRC) of Australia (who develop the Australian Dietary Guidelines). And we’ve seen our methods adopted and adapted by others in the field; for example, the NNR will be using our risk-of-bias tool, the RoB-NObs, in its forthcoming systematic reviews.

9. What work has been conducted to identify advances in food pattern modeling and related methodologies?
   1. What criteria are used by USDA to determine that changes in the modeling approach are warranted?

Response: Similar to the other scientific approaches used in the Dietary Guidelines process, USDA and HHS support ongoing continuous quality advancement (CQA) efforts for food pattern modeling. Food pattern modeling helps explain how changes to food-based dietary recommendations could potentially affect Americans’ ability to meet their nutrient needs. For the Dietary Guidelines for Americans, 2020–2025 development process, food pattern modeling was used to answer a portion of the topics and supporting scientific questions the Advisory Committee examined. These questions looked at:

• The ability to meet nutrient recommendations for each stage of life through variations in USDA Food Patterns.
• Development of two USDA Food Patterns for ages 12 through 23 months.
• The relationship between added sugars consumption and achieving nutrient and food group recommendations.

For the Dietary Guidelines for Americans, 2020–2025 development process, federal staff evaluated the analytic methods and development of data inputs and constraints for food pattern modeling and compared them to methods used in the development of guidance in other countries, as well as other modeling exercises described in scientific publications. The data inputs and constraints are the foundation for food pattern modeling, and when compared to external examples, they align with best practices and exceed standards for transparent documentation. Additionally, food pattern modeling methodology for the 2020–2025 process relied on food composition data from the USDA Food and Nutrient Database for Dietary Studies (FNDDS) and Standard Reference (SR). The FNDDS and SR provide comprehensive nutrient data on food choices of Americans. Since the 2015–2020 Dietary Guidelines for Americans, additional foods were added to FNDDS to

better reflect the food supply and the increased variability of what Americans are eating. This increased variability was leveraged during food pattern modeling exercises conducted to inform the development of patterns for the Dietary Guidelines for Americans, 2020–2025.

For the Dietary Guidelines for Americans, 2025-2030 development process, the Departments are accomplishing CQA in food pattern modeling and related methodologies through the work of the Food Pattern Modeling Interest Group (FPM IG). This self-directed work group includes staff from HHS and USDA and was tasked with determining what enhancements can be made to further advance the food pattern modeling methodology to better reflect the complex interactions involved, variability in intakes, and range of possible healthful diets.

Like the NESR CQA process, the FPM IG has implemented a CQA process that includes three phases: implementation, evaluation, and refinement. The FPM IG is currently working in the evaluation phase to determine the scope and feasibility of possible enhancements. Project management planning is also underway for the refinement phase, which will include methodological enhancements, expected to be completed prior to establishing the 2025 Dietary Guidelines Advisory Committee.

During the evaluation phase, the FPM IG has worked with the Departments to provide draft scientific questions on food pattern modeling for public comment. These questions were developed with a focus on intake variability and the range of possible healthful diets within the U.S. population and include input from external sources (i.e., previous public comments, 2017 NASEM report) and federal peer review (e.g., ICHNR Subcommittee on the Dietary Guidelines, NIH Nutrition and Health Disparities Implementation Working Group). The draft questions (shown below) will be refined with consideration of the current public comments and federal agency input.

The FPM IG has also evaluated the analytic methods and development of data inputs and constraints for food pattern modeling and are in the process of completing landscape and comparative analyses to compare methods used in the development of guidance in other countries, as well as other modeling exercises described in scientific publications and the recommendations in the 2017 NASEM report. This landscape analysis involves review of methodology manuals and food pattern modeling reports, websites, and published literature, as well as dialogue with food pattern modeling experts. The comparative analysis will document where there is alignment, and where there is not, to help inform methodological enhancements. As an example, the FPM IG is exploring the use of stochastic modeling and sensitivity analyses, as well as looking at the ways to incorporate a range of nutrient values within food group recommendations to help address the variability of intakes within the U.S.

The FPM IG has also begun project planning for possible methodological enhancements, which will first go out for peer review by the ICHNR Subcommittee on Dietary Guidelines. After the evaluation phase is complete and peer review incorporated, the FPM IG will test, adopt, and document refinements to the food pattern modeling methodology. These refinements are intended to advance food pattern modeling methodology to support analyses and subsequent guidance that reflects the diverse foodways of the U.S.

Like NESR, the FPM IG also take the view that within the field of food pattern modeling, there is not a single, consensus, “gold standard” method that all organizations should seek to adopt or emulate. The landscape analysis indicates that food pattern modeling methods have many similarities in the foundational approaches. Efforts are tailored to utilize food-composition databases and dietary intake data and to take into account dietary guidance specific to the population. The FPM IG is working to align and collaborate with others while advancing the field and tailoring our methods to ensure applicability to public health nutrition decision-making and development of dietary patterns in the United States.

Work is also underway to explore how to best integrate additional systems approaches into future Dietary Guidelines development processes. The 2017 National Academies report acknowledged that systems science for nutrition at the time was in its infancy, making it difficult to incorporate in previous editions. For future editions, USDA recently secured funding to start the process of examining best practices for adapting the complex methodology of systems approaches and applying it as effectively as possible, while ensuring that the Dietary Guidelines continue to reflect the highest standard of scientific integrity and contain information adaptable for public health and consumer use. More information on these efforts will be made available later this year.

10. How is USDA working with federal partners to address gaps identified in the development of the framework for nutrients of concern?

Response: Example activities of how USDA, HHS, and other federal partners are working together are highlighted below:

• One way that the Federal government works together to address issues related to nutrition and health is the Interagency Committee on Human Nutrition Research (ICHNR). The ICHNR was created in 1983 and is charged with improving planning, coordination, and communication among federal agencies engaged in research in nutrition and/or nutrition policies, surveillance, technologies, and related issues. The ICHNR membership includes representatives from 5 Departments (encompassing at least 11 related agencies), 4 independent agencies, and a White House executive office representing most federal diet-related efforts. To work on issues related to the DGA, the ICHNR includes a Subcommittee on Dietary Guidance. This Subcommittee is chaired by the Center for the Nutrition Policy and Promotion, USDA and the Office of Disease Prevention and Health Promotion, HHS. The purpose of this Subcommittee is to: 1) Provide a mechanism for agencies with food and nutrition programs to provide input into the development of the DGA; and 2) Receive regular updates from the lead agencies on this initiative. Membership on this subcommittee includes representatives from HHS and USDA, as well as the Department of Defense (DoD) and Veterans Affairs (VA).
• The ICHNR also has a Subcommittee on Dietary Reference Intakes. This Subcommittee aims to identify DRI needs and coordinate government sponsorship of DRI reviews, supporting systematic reviews, and other related activities. The DRIs are a major scientific input to the DGA and are used widely in federal programs and policies, nutrition research, educa

• tion, food labeling, and public health programs. The U.S. DRI subcommittee has collaborated with Canadian Health Agencies through its Joint U.S. and Canada DRI working group on several important activities related to updating DRIs, including identifying DRI needs, prioritizing nutrient systematic reviews, advancing work to resolve methodological issues related to DRI reviews, and jointly sponsoring DRI-related committees through NASEM. The U.S. and Canadian governments are collaborating on upcoming DRI updates. The ICHNR identified updating the DRIs as a top priority, and subsequently USDA and HHS leadership committed funding up to 5 years. Energy and macronutrients (protein, carbohydrates and fat) have been prioritized, since there is substantial new research on the relationship between energy balance, macronutrient intake, and health outcomes, including risk of chronic diseases. The U.S. and Canada Working group recently contracted with NASEM to support a committee to assess the human requirements for energy intake and expenditure to update the energy DRIs. The working group also recently funded a NASEM Standing Committee for the Review of the Dietary Reference Intakes to address overarching issues related to DRIs. Pending a regular source of funding, the working group will develop a continuous process to update DRIs and supporting systematic reviews. The availability of updated nutrient values is critical to inform high-priority nutrition programs. Expansion of ICHNR collaboration to reflect all federal nutrition efforts can provide an opportunity for research entities and programs to highlight nutrients of interest and for all nutrition efforts to keep abreast of ongoing work to support coordinated efforts when new DRIs are available.
• The Federal Data Consortium on Pregnancy and Birth to 24 Months (P/B-24, Data Consortium) is a Federal forum for sharing information and finding solutions to resolve crucial needs for data on P/B-24 populations to inform public health initiatives. The Data Consortium facilitates the ability for Federal agencies to gather and vet input to inform projects and research plans and share information on data availability and usage. Over a dozen Federal agencies have funded projects coordinated by the Data Consortium focused on filling critical gaps in scientific knowledge to support evidence-based programs, policies, and educational initiatives across the government and non-government sectors. The collaboration spans about 225 Federal staff and leaders from 30 agencies in five Departments (HHS, USDA, U.S. Agency for International Development [USAID], EPA, and DoD). The Data

• Consortium is co-led by representatives of ODPHP, CDC, NIH, and FDA of HHS and ARS and FNS of USDA.
• The Data Consortium arose out of work by ODPHP/Office of Assistant Secretary of Health (OASH) in 2016 to coordinate “special studies” from CDC’s National Health and Nutrition Examination Survey (NHANES) to fill crucial needs for data on P/B-24 populations to support future DGA. However, the essentiality for these data goes beyond the needs of the DGA. Initiatives include The Human Milk Composition Initiative (Casavale et al. 2019) and new B-24 questionnaire content in the NHANES Diet and Behavior Questionnaire (Casavale and Woodwell, 2018). The forum has been used extensively by several agencies in planning population studies, such as CDC’s Infant Toddler Feeding Practices III and HRSA’s National Children’s Study.
• NHANES has a well-established and robust biospecimen program that complements its comprehensive dietary intake and behaviors data (NHANES, 2022a; Ahluwalia et al., 2016). Over the years NHANES has collected various biological specimens, including blood, urine, and hair. Biospecimens are processed and analyzed for various nutrition biomarkers, disease risk, and environmental exposures, etc. in collaboration with partnering laboratories (NHANES, 2021). Data are released to the public on the NHANES website and inform nutrition research and federal nutrition programs and policies (NHANES, 2022b; Ahluwalia et al., 2016). During NHANES 2021–2022 cycle, blood and urine specimens were collected to determine several analytes that vary by age group. These include nutrition biomarkers (vitamin D, folic acid, iron, magnesium, trans-fatty acids); tests for inflammation, liver and kidney function; lipid profile; and tests related to diabetes (glucose, hemoglobin A1c and insulin), as well as COVID-19 nucleocapsid and spike protein (Paulose-Ram et al., 2021).

Going forward, NHANES plans to continue biospecimen collection and analyses for biomarkers of nutritional status and disease that are of public health concern based on feasibility and available funding.

REFERENCES FOR RESPONSES TO QUESTIONS 5-10 IN APPENDIX D-4

D-5: MEETING 34 OPEN SESSION—SELECT SLIDES

TABLE D-5 Meeting 34 Open Session USDA/HHS Responses to Questions from the Committee

NOTE: This table was adapted from a selection of slides, and minor edits were made for clarification. To see all original slides and a recording of the open session, see https://www.nationalacademies.org/event/06-23-2022/evaluating-the-process-to-develop-the-dietary-guidelines-for-americans-2020-2025-committee-meeting-34-open-session.

CNPP = Center for Nutrition Policy and Promotion; CQA = Continuous Quality Advancement; DGA = Dietary Guidelines for Americans; DGAC = Dietary Guidelines Advisory Committee; HHS = U.S. Department of Health and Human Services; ODPHP = Office of Disease Prevention and Health Promotion; SME = subject-matter expert; TEC = Technical Expert Collaborative; USDA = U.S. Department of Agriculture.

DGPCG not formed by name, but the function of the DGPCG has been accomplished and will continue to evolve. TEPs not formed by name, but pre-work occurred and included a range of expertise and will continue to evolve. TEPs are different from TECs. TEC members develop and refine systematic review protocols, and participate in evidence synthesis, conclusion statement development, and grading the strength of the evidence. DGSAC was not formed by name, but the functions of the DGAC are similar to those of the DGSAC.

¹ See Appendix D-4 for the questions to the sponsors.

SOURCE: Table adapted from a presentation given by Eve Stoody and Julie Obbagy on June 23, 2022.