Bodnaruc Alexandra M, Khan Hassan, Shaver Nicole, Bennett Alexandria, Wong Yiu Lin, Gracey Catherine, Ly Valentina, Shea Beverley, Little Julian, Brouwers Melissa, Bier Dennis, Moher David
School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada.
School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada.
Am J Clin Nutr. 2025 Jan;121(1):111-124. doi: 10.1016/j.ajcnut.2024.10.013. Epub 2024 Dec 12.
Although high-quality nutrition systematic reviews (SRs) are important for clinical decision making, there remains debate on their methodological quality and reporting transparency.
The objective of this study was to assess the reliability and reproducibility of a sample of SRs produced by the Nutrition Evidence Systematic Review (NESR) team to inform the 2020-2025 Dietary Guidelines for Americans (DGAs).
We evaluated a sample of 8 SRs from the DGA dietary patterns subcommittee for methodological quality using the Assessment of Multiple Systematic Reviews 2 (AMSTAR 2) tool and for reporting transparency using the PRISMA 2020 and PRISMA literature search extension (PRISMA-S) checklists. We assessed the quality and reproducibility of the original search strategy of one selected SR using the Peer Review of Electronic Search Strategies checklist. The reporting transparency of the SR's narrative data synthesis was assessed using the Synthesis Without Meta-Analysis (SWiM) checklist. Interpretation bias was evaluated using existing spin bias classifications in systematic reviews.
The AMSTAR 2 assessment identified critical methodological weaknesses, and all included SRs were judged to be of critically low quality. Overall, 74% of the PRISMA 2020 checklist items and 63% of the PRISMA-S checklist items were satisfactorily fulfilled. We identified several errors and inconsistencies in the search strategy and could not reproduce searches within a 10% margin of the original results. The SWiM assessment identified concerns regarding the reporting transparency of the narrative data synthesis, but the spin bias assessment revealed no evidence of interpretation bias.
Several methodological quality and reporting concerns were identified, which could lead to reliability and reproducibility issues should a full reproduction attempt be made. However, additional research is needed to confirm the impact of these findings on conclusions statements and their generalizability across the NESR team SRs. This study was registered in the Open Science Framework (https://osf.io/ns6a9/).
尽管高质量的营养系统评价（SRs）对临床决策很重要，但关于其方法学质量和报告透明度仍存在争议。
本研究的目的是评估营养证据系统评价（NESR）团队制作的一系列SRs样本的可靠性和可重复性，以为2020 - 2025年美国膳食指南（DGAs）提供参考。
我们使用多重系统评价评估2（AMSTAR 2）工具评估了来自DGA膳食模式小组委员会的8个SRs样本的方法学质量，并使用PRISMA 2020和PRISMA文献检索扩展（PRISMA - S）清单评估了报告透明度。我们使用电子检索策略同行评审清单评估了一个选定SRs原始检索策略的质量和可重复性。使用无Meta分析的综合（SWiM）清单评估了SRs叙述性数据综合的报告透明度。使用系统评价中现有的倾向性偏差分类评估了解释偏差。
AMSTAR 2评估发现了关键的方法学弱点，所有纳入的SRs均被判定为质量极低。总体而言，PRISMA 2020清单项目的74%和PRISMA - S清单项目的63%得到了满意的完成。我们在检索策略中发现了几个错误和不一致之处，并且无法在原始结果的10%范围内重现检索。SWiM评估发现了关于叙述性数据综合报告透明度的问题，但倾向性偏差评估未发现解释偏差的证据。
发现了几个方法学质量和报告方面的问题，如果进行全面的重现尝试，可能会导致可靠性和可重复性问题。然而，需要更多的研究来确认这些发现对结论陈述的影响及其在NESR团队SRs中的普遍性。本研究已在开放科学框架（https://osf.io/ns6a9/）中注册。
