National Academies Press: OpenBook

Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief (2024)

Chapter: Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief

Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×
images Proceedings of a Workshop—in Brief

Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis

Proceedings of a Workshop—in Brief


In September and October 2023, the National Academies of Sciences, Engineering, and Medicine (the National Academies) Food and Nutrition Board held a virtual workshop series to discuss best practices for conducting meta-analyses (MAs) in nutrition research and utilizing MAs to inform policy. The three-part series, entitled Use of Meta-Analyses in Nutrition Research and Policy: A Workshop Series, explored the evidence on methods for planning, conducting, interpreting, and integrating the results of MA for use in nutrition research, policy development, and regulatory decision making. The workshop series was sponsored by the U.S. Food and Drug Administration (FDA) and featured invited presentations and discussions with researchers, government officials, and other global leaders in nutrition research and policy.

The first workshop in the series, held on September 19, 2023, focused on best practices for planning systematic reviews (SRs) and MAs for nutrition research. This initial workshop included opening remarks from two speakers from FDA about the critical importance of MAs to policy making at FDA and the agency’s need for a set of best practices to inform the use of MAs in nutrition policy development. Presenters and discussants addressed the foundational aspects of planning and delivering high-quality nutrition SRs and MAs. The second workshop in the series was held on September 25, 2023, to discuss the best practices for conducting SRs and MAs in the context of nutrition. Presenters discussed data errors, publication bias, and heterogeneity. The third and final workshop in the series which featured presentations on best practices in interpretation and application of SRs and MAs to evaluate the totality of evidence was held on October 3, 2023. This workshop addressed the impact of bias and data errors on interpreting results and the process for evaluating the strength of the evidence. As stated, the objective of the workshop series was to explore issues and best practices around MA; however, because MAs and SRs are closely related processes, many presentations included broader discussions around issues related to both MAs and SRs.

This Proceedings of a Workshop—in Brief1 will present highlights from the first workshop. The workshop began with planning committee chair Katherine Tucker from the University of Massachusetts–Lowell providing opening remarks to the workshop series. Tucker then introduced two speakers from FDA, Crystal Rivers and Sarah Gebauer, who discussed the critical importance

__________________

1 Individual Proceedings of a Workshop—in Brief were prepared for each workshop in the series and are available online at https://nap/edu/27467 and https://nap.edu/27468. The full workshop proceedings is available at https://nap.edu/27481.

Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×

of MAs to policy making at FDA and explained why FDA had sponsored the workshop series. Planning committee member Amanda MacFarlane of Texas A&M University then introduced the workshop’s two main presenters, Celeste Naude of Stellenbosch University and Lee Hooper with the University of East Anglia. Naude’s and Hooper’s presentations addressed the foundational aspects of planning and delivering high-quality SRs and MAs for developing nutrition guidance. Following the presentations was a panel discussion, moderated by MacFarlane, which included Naude, Hooper, and two additional discussants, Sydne Newberry of the RAND Corporation and Christopher Schmid from Brown University School of Public Health. Planning committee member Mei Chung with Tufts University provided brief closing remarks at the end of the two-hour workshop.

The objectives of the workshop were to

  • apply criteria to select studies for inclusion in SRs and MAs, with a focus on population, intervention (including treatment, dose, duration), comparators (with consideration of diet), and outcomes (with consideration for adjustment for confounders/covariates);
  • account for subgroup and sensitivity analyses when planning an SR and MA; and
  • use an appropriate data management system for extracting data.

This Proceedings of a Workshop—in Brief highlights the presentations and discussions that occurred at the first workshop in the series and is not intended to provide a comprehensive summary of the information shared during the workshop.2 Statements, recommendations, and opinions expressed are those of individual presenters and participants and are not necessarily endorsed or verified by the National Academies of Sciences, Engineering, and Medicine; they should not be construed as reflecting any group consensus.

OPENING REMARKS FROM WORKSHOP SPONSOR

Rivers and Gebauer spoke about FDA’s investment in the workshop series, including the unique research and policy development challenges faced by the field of nutrition and how they are relevant to FDA’s work. Rivers spoke about the use of MAs in the development of nutrition regulatory decisions, nutrition policy, and dietary recommendations and how MAs can be used to identify existing gaps in nutrition research. She noted that the use of MAs with SRs in guideline and policy development is becoming more common, and that there is an increasing need for a discussion about best practices.

Gebauer emphasized FDA’s unique needs regarding the agency’s nutrition regulatory framework, including how evidence is used to substantiate health claims by showing a direct relationship between a food product and a health outcome or disease risk. She also pointed out some of FDA’s unique considerations when conducting its own reviews of the evidence related to nutrition labeling and health claims. For example, FDA typically conducts health claim reviews in response to stakeholder petitions, which identify the topic of the review based on stakeholder interests and relies on publicly available data for its health claim reviews. Gebauer highlighted the need for consistent application of tools and a set of best practices for conducting MAs and for evaluating and interpreting MAs in nutrition research and policy. Gebauer noted that FDA hoped to use the discussions of the workshop series to develop clear recommendations for evaluating the quality of existing MAs, conducting MAs, and interpretating and integrating outcomes of MAs to evaluate the strength of a body of evidence for the development of nutrition policy. Rivers added that FDA intended the workshop series to inform a set of best practices for the use of MAs in future nutrition policy development at the agency.

PLANNING FOR SYSTEMATIC REVIEWS AND META-ANALYSES

Naude and Hooper’s presentation was entitled “Systematic Reviews and Meta-Analysis for Developing Nutrition Guidance: The Core Pillars of Planning and Methods to Deliver High-Quality, Useful Synthesized Evidence.” The presentation addressed questions posed in advance by the workshop sponsor:

__________________

2 The workshop agenda, presentations, and other materials are available at https://www.nationalacademies.org/event/40373_09-2023_use-of-meta-analyses-in-nutrition-research-and-policy-planning-of-meta-analysis (accessed December 11, 2023).

Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×
  • What are best practices in identifying and avoiding extraction errors and errors in calculating mean differences and confidence intervals from the primary studies that are included in a meta-analysis, given that these errors are common in published literature?
  • How should an MA be evaluated for methodological quality when extraction or data errors are present? At what point do data errors (in kind and number) reach a level that invalidates the conclusions of the meta-analysis?
  • How do you consider risk of bias when evaluating diet and disease relationships?
  • What are the best practices for addressing publication bias?
  • What criteria should be used to determine whether individual nutrition studies have too many clinical or methodological differences (e.g., treatment, dose, population, mean BMI, duration, comparators/diets, results) to be combined into the same MA?
  • What are best practices for planning appropriate subgroups and sensitivity analysis a priori?
  • How can MAs be used to evaluate the strength of the evidence when different outcomes are reported in different studies? And how can MAs be used to evaluate the strength of the totality of evidence?

Naude’s presentation focused on the importance of the planning phase of an SR or MA and how the planning process can set a research team up for success. Naude framed her involvement in the field of nutrition research, noting that she is the codirector of Cochrane Nutrition and is involved in other Cochrane groups. She is also a founding member of the South Africa Grading of Recommendations Assessment, Development and Evaluation (GRADE) Network. GRADE is a research tool used to develop and present summaries of evidence and provide a systematic and transparent approach for assessing the certainty (or quality) of evidence and making recommendations. Naude stated that she receives no industry funding.

Systematic Reviews and Meta-Analyses: An Overview

Naude explained the basic concepts of SRs and MAs, beginning with SRs. An SR is a well-defined research method that pulls together the results of many primary studies that meet pre-specified criteria to answer well-framed research questions. They are used to minimize bias when reviewing and assessing evidence. Naude stated that SRs should be performed in rigorous, methodological ways to reduce reviewer selection bias, and should have pre-determined rules for identifying and including studies. Reviewing a body of evidence is challenging, due to the growing volume of research, variability in study designs, and haphazard and biased access to research. To this end, Naude noted that as decision makers face challenges in using vast amounts of primary research to enable informed policy development, SRs are key to the planning and execution of effective policies.

As Naude described, an MA is a statistical method that may be part of the SR process. She emphasized that all MAs should be informed by a rigorous SR. MAs combine the numerical results of studies and require transparency, documentation, and careful planning. Naude explained that setting up an MA with clear questions and methods a priori is essential for minimizing bias and improving the reliability of the results.

Naude detailed the benefits of SRs and MAs for the policy development process. When completed correctly, SRs and MAs can increase transparency and objectivity, reduce bias, capture the totality of evidence on a subject, create more precise results, capture the uncertainty of existing findings, and point to the need for additional research on a topic.

The Planning Process

Naude spoke about the planning of SRs, describing the four pillars of the planning process:

  1. gathering the necessary background information,
  2. assembling an author team,
  3. assembling adequate resources for the SR, and
  4. establishing the question that the SR seeks to answer.

When considering the first pillar, background information, Naude noted that it is important to develop

Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×

an SR that is “fit to the purpose,” which she explained as considering the purported use of the SR, the end product to be informed by the guidance produced by the SR, and the target audience. She also explained that SRs are commonly used when there is a “guidance gap,” or a lack of sufficient information to create reliable guidance on an issue or a lack of clarity on how an existing guidance should be applied.

Speaking on the second pillar, the assembly of the author team, Naude noted the importance of minimizing funding and nonfinancial conflicts of interest. She said that one should seek to assemble a “dream team” consisting of experts with expertise in both methods and subject matter expertise. The authors of the SR should not have a vested interest in the findings of the review.

Regarding the third pillar, in addition to having adequate resources in place to carry out a successful SR, Naude emphasized the importance of adhering to an established timeline and practicing good data management and quality assurance protocols to ensure the reliability of the SR. Good data management and quality assurance processes, established during the planning phase, also allow for data replicability and enhanced credibility. She suggested that new software, and more recently the use of artificial intelligence (AI) enabled programs can be used to support this process.

Referencing the fourth pillar, developing the research question, Naude explained that developing the question that guides the SR is critical, as it frames all future work during the SR process. She suggested taking time to clearly develop the question with the entire team, noting that the question might be a problem statement, and could be broad or narrow depending on the SR’s needs. To develop the question, Naude stated that the team should consider the aim of the review and use the Population, Intervention, Comparator, Outcome (PICO) or Population, Exposure, Comparator, Outcome (PECO) construct to help decide which factors are most critical in the development of their research question.

Use of PICO in Planning

Naude highlighted that a logic model and conceptual framework are key parts of the planning process for SRs. Because nutrition studies are inherently complex and multifactorial, logic models are useful for refining the systematic review’s central question and guiding the review process. The PICO process can be used at three stages of the systematic review process. For example, the review PICO, planned during the protocol stage, helps to decide on the inclusion and exclusion criteria. The PICO for synthesis, which is also planned during the protocol stage, helps the researchers understand the anticipated markers of comparison, such as the intervention or population subgroups. Finally, the PICO of included studies, determined during the review stage, considers what was examined in the primary studies that have been included in the SR, and helps the team understand the extent to which data from these studies, and the SR overall, is generalizable.

Naude concluded her presentation by noting the necessity of choosing which outcomes are included in an SR and weighting these outcomes by level of importance. When rating the importance of variables, Naude suggested considering what outcomes are of greatest importance to policy makers that will use the review to inform policy and guideline development. Finally, Naude reiterated that nutrition research is inherently complex, and multiple studies measuring the same inputs or nutrients may show different results.

METHODS FOR REDUCING THE RISK OF BIAS IN SYSTEMATIC REVIEWS AND META-ANALYSES

Hooper spoke about the methods for reducing the risk of bias in SRs and MAs. She noted that the methods and resources discussed were largely drawn from Cochrane guidance3 and training4 materials. Hooper disclosed that she had not received industry funding during the last decade, has been an editor for the Cochrane Heart Group and Oral Health Group, and is a member of the World

__________________

Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×

Health Organization Nutrition Guidance Expert Advisory Group.

Hooper introduced the seven methodological pillars for reducing risk of bias in SRs. They included:

  1. writing the protocol,
  2. searching for studies,
  3. selecting studies and collecting data,
  4. assessing risk of bias of included studies,
  5. analyzing the data,
  6. interpreting the findings, and
  7. reporting the review.

Describing the second pillar on searching for studies, Hooper said that a well-conducted search is the foundation of a good SR and explained that searches should span across years, databases, languages, and paradigms. She suggested including in the review team a research expert familiar with using a “sensitive” approach, including searching in “gray” literature, which may include unpublished work, theses, and government papers. Hooper stated that the search process should be peer-reviewed, detailed, and reproducible.

With respect to the third pillar, selecting studies and collecting data, Hooper suggested that the team consider which papers found in the search fulfill the team’s pre-specified inclusion or exclusion criteria via independent duplication of assessment. The research team should refer to the PICO or PECO framework and develop rules to resolve disagreements around which studies to include and exclude. Hooper explained that data collection should be carried out independently in duplicate, and any discrepancies in the data should be discussed within the team, and after such a discussion, criteria may be refined. Hooper said it is important for SRs to gather data including study methods, participants, interventions, comparators, flow (numbers recruited, dropping out, and analyzed) and outcomes. Nutrition SRs also need to collect data on baseline nutrition status and how fully the nutrition intervention was implemented. She suggested reaching out to a study’s authors when any outcome data is not clear. In response to questions about best practices for identifying and avoiding data errors and use of MAs with data errors, Hooper warned against using any studies, SRs, or MAs that might contain errors in their data.

Hooper detailed some tools that may be useful during the data collection and study selection phases of conducting SRs. Covidence is a tool that enables the independent duplication of assessment of inclusion criteria, data extraction, and risk of bias assessment and allows for remote collaboration with teams around the globe.5 Hooper also discussed Rayyan, a free AI tool that enables collaborative reviews conducted across a broad research platform and easily allows researchers to independently assess the data in duplication.6 Another tool, EPPI Reviewer, supports a range of reviews, including metaethnographies.7 Hooper suggested using PlotDigitizer and Microsoft Paint, which can obtain numbers from a visual plot.8

On the topic of the fourth pillar, assessing risk of bias, Hooper noted that bias is an inherent part of all studies. What is critical, she said, is understanding the bias, its level in each study, and where it may be greatest in each study. Hooper offered some methods for avoiding and addressing bias in the SR process. First, she said teams should aim to exclude studies that have a high risk for bias and tabulate risk of bias in all included studies. As part of this process, Hooper suggested that teams run sensitivity analyses within an MA, including all relevant studies in the main analysis, then removing studies at highest risk of bias, and then run the sensitivity analysis again. If sensitivity analysis results differ markedly from the main analysis, then bias should be suspected and reported. Throughout this process, Hooper suggested using tools such as GRADE to assess, analyze, and present findings on the reliability of the review findings (including bias) for each review outcome. She detailed some unique forms of bias that may be

__________________

5 For more on Covidence, see https://www.covidence.org/ (accessed January 16, 2024).

6 For more on Rayyan, see https://www.rayyan.ai/ (accessed January 16, 2024).

7 For more on EPPI Reviewer, see https://eppi.ioe.ac.uk/cms/ (accessed January 16, 2024).

8 For more on PlotDigitizer, see https://plotdigitizer.com/ (accessed January 16, 2024).

Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×

present in observational nutrition studies. For example, dietary exposures are often accompanied by numerous confounders such as socioeconomic status, physical activity levels, or smoking status.

Hooper reviewed the risks of bias and study limitations of observational studies based on information from the GRADE handbook. Hooper also shared some benefits of using cohort studies, such as the potential for a longer follow up period, the potential to assess a broader set of outcomes, and larger sample sizes than randomized controlled trials (RCTs), making it potentially easier to observe population health effects. Hooper said that both RCTs and cohort studies have their own unique strengths and weaknesses, and it is important to consider the differences between the study types when deciding if and how to include them in an SR. Ideally, if observational studies are systematically reviewed, RCTs should also be searched for and included.

Addressing a question of how to consider risk of bias when evaluating diet and disease relationships, Hooper suggested that teams should assess the risk of bias, report the risk, and use a sensitivity analysis to see if their results change when removing the data considered to be at highest risk for bias.

Hooper addressed a question about best practices for addressing publication bias, stating the importance of identifying, acknowledging, and addressing the studies that are missing from the published literature. She spoke about the role of unpublished studies and explained that SRs of observational studies have greater problems with selective reporting and publication bias than SRs of RCTs because observational studies are less likely to be published than trials if they do not show statistically significant effects, resulting in a published set of studies that is unlikely to be representative of the true evidence base. She also noted that it is easier to identify unpublished RCTs than other types of unpublished studies, as RCTs are required to register in advance of being conducted and thus are discoverable through searching of trials registers. Trials registers also provide pre-specified study methods for both published and unpublished studies, allowing research teams to check whether published studies include all planned outcomes, analytical methods, or subgroup analyses. The ability to screen for these factors in RCTs enables better assessment of risk of bias for individual trials within the SR, and ultimately for the SR itself. However, Hooper explained that a similar process is not available for identifying unpublished cohort studies or the pre-specified methods of published observational studies. Only positive associations may be published, and negative or nonexistent associations may remain unpublished. She emphasized that it is critical to analyze the study methods, registry entries, and protocols in every study being considered for inclusion in an SR.

Hooper spoke on the use of data analysis for bias reduction, the fifth pillar, discussing the importance of the protocol that guides the analysis process. She noted that teams should specify in the protocol what effect size is clinically relevant to the research topic. The protocol should also prespecify methods of data analysis including comparisons to be made, effect measures, all data analysis questions, how heterogeneity will be assessed, and what subgroup analyses and sensitivity analyses will be run.

Addressing the question of what criteria should be used to determine whether individual nutrition studies with clinical or methodological differences can be combined within the same MA, Hooper responded that data from different methodologies should not be combined into a single MA. She noted that subgrouping can be used to answer subquestions.

Hooper addressed best practices for interpretation of findings, the sixth of the seven pillars. She again recommended the use of tools, such as GRADE, to upgrade or downgrade each piece of evidence. She stated that interpretation of the results may be difficult, given incomplete information, and said GRADE assumes a higher certainty of evidence when assessing RCTs and a lower quality when assessing cohort studies.

Regarding the last of the seven pillars, reporting an SR, Hooper suggested that researchers use the Preferred Reporting Items for Systematic Reviews and Meta-

Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×

Analyses (PRISMA)9 statement to ensure that their SR is reported comprehensively. PRISMA is an evidence-based minimum set of items for reporting an SR or MA that includes a flow diagram of potential studies and a checklist of the key components to be reported.

She addressed the questions of how MAs can be used to evaluate the strength of evidence when different outcomes are reported in different studies and how MAs can be used to evaluate the strength of the totality of evidence, providing an example from the development of World Health Organization (WHO) guidance on saturated fat consumption.10 WHO commissioned three SRs to assess the impact of saturated fat intake on health outcomes. The first SR focused on RCTs of at least two years duration that assessed the impact of reducing saturated fat intake on a variety of noncommunicable diseases, such as cardiovascular disease and cancer. In parallel, WHO commissioned a second SR of prospective cohort studies that addressed any relationships that were observed between saturated fat intake and the same health outcomes. They also commissioned an update of an SR and regression analyses of saturated fat intake and lipid outcomes in highly controlled metabolic studies over short periods of time. For each key outcome within each review the evidence was assessed using GRADE. Consistency in results was found across the three reviews, which was seen as strengthening the quality of evidence.

Hooper recommended some tools for analyzing existing SRs. She noted the Risk of Bias in Systematic Reviews (ROBIS) tool, which helps quantify whether a particular SR is strong enough to be used for the development of dietary guidance. ROBIS helps assess the relevance of studies, identify concerns, and judge their overall risk of bias.

PANEL DISCUSSION

A panel discussion moderated by MacFarlane followed the presentations. Additional discussants, Newberry and Schmid, joined the conversation with Naude and Hooper. MacFarlane invited comments from the panelists and questions from audience members.

Transparency

On the importance of transparency, Hooper said that making good decisions, being fully transparent, and not adding additional bias during the SR process is challenging but critical. MacFarlane added that maintaining transparency and reporting challenges can help maintain trust in a study. Hooper highlighted the importance of teamwork and transparency in the GRADE process, through which teams assess the relative quality of the assembled evidence. Newberry stated the importance of transparency regarding the quality of the evidence, and of using high-quality evidence to inform policy and guideline development.

Planning and Team Development

Newberry said that one of the biggest challenges in planning SRs and MAs is working with stakeholders to frame research questions that are truly answerable. She referenced comments from Hooper and Naude about the importance of focusing on the clinical significance of outcomes. However, for many outcomes of interest, Newberry said, the clinically important difference may be unknown. Given that many nutrition questions have an overwhelming body of literature and studies have finite sample sizes, Newberry suggested that limits on how much evidence is gathered and assessed should be established as part of the review protocol development process.

As part of the planning process for SRs, Hooper and Newberry described the benefits of searching for existing SRs on the topic of interest prior to initiating a new study, noting that many nutrition research questions have been addressed in recent SRs.

Schmid contributed to the discussion of proper planning and reinforced that SRs are only as good as the studies they include and the sampling schemes on which they are based. Importantly, including more studies does not inherently mean the right studies will be included or a better review produced. Schmid further contributed to the discussion on research teams and protocols, saying that one could compare the SR process to that of an RCT. Protocols may be revised numerous times as issues arise,

__________________

9 For information on PRISMA, see http://prisma-statement.org/prismastatement/checklist.aspx?AspxAutoDetectCookieSupport=1 (accessed January 16, 2024).

10 For information on WHO guidance on saturated fats, see https://www.who.int/publications/i/item/9789240073630 (accessed December 31, 2023).

Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×

but changes should be documented and agreed upon by all team members prior to beginning the research. Hooper added that having teams do a trial run of study inclusion with a set of 200 titles and abstracts using the set of eligibility criteria (and later a trial run of the full data extraction and risk of bias assessment tool with 2 or 3 included studies) before the review begins can improve the overall efficiency of the process.

During the discussion about creating effective teams and organizing the review process, audience members asked about the process for building the “dream team” for conducting the review, including who should be included and how roles and responsibilities should be divided among team members. Naude replied that different reviews will require different approaches. Generally, she said, a team will include a lead or project manager and other members with different areas of expertise, including subject matter experts and data analysis and information specialists. Some review teams may delegate the data extraction process for efficiency purposes. Naude and Hooper reinforced the benefits of including subject matter and methodology experts throughout the process, including protocol development and data analysis.

Assessing and Addressing Bias

The panel discussed the topic of bias. Schmid said that teams should report as much high-quality data as possible to reduce risk of bias.

MacFarlane noted that there are many ways to assess and address bias, including through use of risk of bias tools. Numerous tools exist, and it may be difficult for researchers to determine which tool or method is best for their study. Naude suggest using the most recent Cochrane tool, Risk of Bias 2, for reducing bias in RCTs. Naude explained that when using the Risk of Bias 2 tool, reviewers have the option to change the weighting of different domains.

With study designs that are not RCTs, Naude suggested using other tools to reduce bias and urged presetting rules to manage overall judgments. Consistent application of methods and tools is key, she reiterated. Hooper added that it is critical to focus on the risk of bias most relevant to the particular area of study, or the bias that would have the greatest impact on the results of the review. Newberry added that since most nutrition studies are cohort studies, not RCTs, the Risk of Bias in Nonrandomized Studies of Interventions (ROBINS-I) or Risk of Bias in Nonrandomized Studies of Exposures (ROBINS-E) may be useful to assess risk of bias. Newberry also suggested the Newcastle-Ottawa Scale tool for assessing quality of nonrandomized studies in MAs. However, she noted that these tools may not be sensitive enough to detect all types of bias.

MacFarlane posed a question about how to address the bias that may result from the lack of clear control groups in nutrition studies. For example, she explained, drug trials often have clear treatment and placebo groups, but nutrition interventions may rely on baseline nutrition status as a control group, which can be an imprecise measurement. Naude suggested closely examining the study design of the included studies to understand if the research question or intervention accommodates the imprecision of baseline nutrition status as a control group. She added that this challenge emphasizes the benefits of reporting on the primary research and ensuring that every included study meets the basic reporting guidelines established in the review protocol. Naude noted that advances in technology promise improvements to data collection, organization, and analysis in nutrition research.

CLOSING REMARKS

Chung provided closing remarks for the workshop, thanking the workshop sponsor, planning committee, the National Academies, and presenters for planning and executing the workshop. On behalf of the planning committee, Chung also thanked the audience members for attending the workshop and for contributing to the panel discussion.

This workshop was the first of three workshops in the series Use of Meta-Analyses in Nutrition Research and Policy. Subsequent workshops focused on best practices in conducting MAs and interpretation and application of MAs in nutrition research and policy.

Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×

DISCLAIMER This Proceedings of a Workshop—in Brief has been prepared by Melissa Maitin-Shepard and Marian Flaxman as a factual summary of what occurred at the meeting. The statements made are those of the rapporteurs or individual workshop participants and do not necessarily represent the views of all workshop participants; the planning committee; or the National Academies of Sciences, Engineering, and Medicine.

*The National Academies of Sciences, Engineering, and Medicine’s planning committees are solely responsible for organizing the workshop, identifying topics, and choosing speakers. The responsibility for the published Proceedings of a Workshop—in Brief rests with the institution. The planning committee comprises Katherine Tucker (Chair), University of Massachusetts–Lowell; Mei Chung, Tufts University; Russell J. de Souza, McMaster University; Amanda MacFarlane, Texas A&M University; Chizuru Nishida, World Health Organization (retired); and Janet Tooze, Wake Forest University.

REVIEWERS To ensure that it meets institutional standards for quality and objectivity, this Proceedings of a Workshop—in Brief was reviewed by Paule Joseph, National Institutes of Health and National Institute on Alcohol Abuse and Alcoholism, and Chizuru Nishida, World Health Organization (retired). Leslie Sim, National Academies of Sciences, Engineering, and Medicine served as the review coordinator.

SPONSOR This workshop was supported by the United States Food and Drug Administration (contract no.: 75F40122D00002/75F402122F19002).

STAFF Takyera Robinson, Alice Vorosmarti, and Samuel Crawford, Food and Nutrition Board, Health and Medicine Division, National Academies of Sciences, Engineering, and Medicine.

For additional information regarding the workshop, visit https://www.nationalacademies.org/event/40373_09-2023_use-of-meta-analyses-in-nutrition-research-and-policy-planning-of-meta-analysis.

SUGGESTED CITATION National Academies of Sciences, Engineering, and Medicine. 2024. Use of meta-analyses in nutrition research and policy: Planning of meta-analysis: Proceedings of a workshop—in brief. Washington, DC: The National Academies Press. https://doi.org/10.17226/27466.

Health and Medicine Division

Copyright 2024 by the National Academy of Sciences. All rights reserved.

images
Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×
Page 1
Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×
Page 2
Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×
Page 3
Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×
Page 4
Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×
Page 5
Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×
Page 6
Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×
Page 7
Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×
Page 8
Suggested Citation:"Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2024. Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief. Washington, DC: The National Academies Press. doi: 10.17226/27466.
×
Page 9
Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief Get This Book
×
 Use of Meta-Analyses in Nutrition Research and Policy: Planning of Meta-Analysis: Proceedings of a Workshop–in Brief
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The first workshop in the series on the use of meta-analysis in nutrition research and policy, held on September 19, 2023, focused on best practices for planning systematic reviews and meta-analyses for nutrition research. This was emphasized as a critical need for the development of future nutrition policy.

READ FREE ONLINE

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!