Andersen Mikkel Schou, Kofoed Mikkel Seremet, Paludan-Müller Asger Sand, Pedersen Christian Bonde, Mathiesen Tiit, Mawrin Christian, Olsen Birgitte Brinkmann, Halle Bo, Poulsen Frantz Rom
Department of Neurosurgery, Odense University Hospital, Odense, Denmark.
Department of Clinical Research, University of Southern Denmark, Odense, Denmark.
BMC Med Res Methodol. 2024 Dec 19;24(1):306. doi: 10.1186/s12874-024-02413-0.
Systematic reviews within the field of animal research are becoming more common. However, in animal translational research, issues related to methodological quality and quality of reporting continue to arise, potentially leading to underestimation or overestimation of the effects of interventions or prevent studies from being replicated. The various tools and checklists available to ensure good-quality studies and proper reporting include both unique and/or overlapping items and/or simply lack necessary elements or are too situational to certain conditions or diseases. Currently, there is no tool available, which covers all aspects of animal models, from bench-top activities to animal facilities, hence a new tool is needed. This tool should be designed to be able to assess all kinds of animal studies such as old, new, low quality, high quality, interventional and noninterventional on. It should do this on multiple levels through items on quality of reporting, methodological (technical) quality, and risk of bias, for use in assessing the overall quality of studies involving animal research.
During a systematic review of meningioma models in animals, we developed a novel unifying tool that can assess all types of animal studies from multiple perspectives. The tool was inspired by the Collaborative Approach to Meta Analysis and Review of Animal Data from Experimental Studies (CAMARADES) checklist, the ARRIVE 2.0 guidelines, and SYRCLE's risk of bias tool, while also incorporating unique items. We used the interrater agreement percentage and Cohen's kappa index to test the interrater agreement between two independent reviewers for the items in the tool.
There was high interrater agreement across all items (92.9%, 95% CI 91.0-94.8). Cohen's kappa index showed quality of reporting had the best mean index of 0.86 (95%-CI 0.78-0.94), methodological quality had a mean index of 0.83 (95%-CI 0.78-0.94) and finally the items from SYRCLE's risk of bias had a mean kappa index of 0.68 (95%-CI 0.57-0.79).
The Critical Appraisal of Methodological (technical) Quality, Quality of Reporting and Risk of Bias in Animal Research (CRIME-Q) tool unifies a broad spectrum of information (both unique items and items inspired by other methods) about the quality of reporting and methodological (technical) quality, and contains items from SYRCLE's risk of bias. The tool is intended for use in assessing overall study quality across multiple domains and items and is not, unlike other tools, restricted to any particular model or study design (whether interventional or noninterventional). It is also easy to apply when designing and conducting animal experiments to ensure proper reporting and design in terms of replicability, transparency, and validity.
动物研究领域的系统评价越来越普遍。然而，在动物转化研究中，与方法学质量和报告质量相关的问题仍不断出现，这可能导致对干预效果的低估或高估，或使研究无法被重复。现有的各种确保高质量研究和恰当报告的工具和清单，包含独特和/或重叠的条目，和/或仅仅缺少必要元素，或过于针对特定条件或疾病。目前，尚无涵盖从实验台活动到动物设施等动物模型所有方面的工具，因此需要一种新工具。该工具应设计成能够评估各种动物研究，如新旧研究、低质量和高质量研究、干预性和非干预性研究等。它应通过报告质量、方法学（技术）质量和偏倚风险等条目在多个层面进行评估，以用于评估涉及动物研究的整体研究质量。
在对动物脑膜瘤模型的系统评价过程中，我们开发了一种新颖的统一工具，该工具可从多个角度评估所有类型的动物研究。该工具的灵感来自于实验研究动物数据的荟萃分析和综述协作方法（CAMARADES）清单、ARRIVE 2.0指南以及SYRCLE的偏倚风险工具，同时还纳入了独特的条目。我们使用评分者间一致性百分比和科恩kappa指数来测试工具中两个独立评审员对条目的评分者间一致性。
所有条目间的评分者间一致性较高（92.9%，95%CI 91.0 - 94.8）。科恩kappa指数显示，报告质量的平均指数最佳，为0.86（95% - CI 0.78 - 0.94），方法学质量的平均指数为0.83（95% - CI 0.78 - 0.94），最后SYRCLE偏倚风险条目的平均kappa指数为0.68（95% - CI 0.57 - 0.79）。
动物研究中方法学（技术）质量、报告质量和偏倚风险的批判性评价（CRIME - Q）工具整合了关于报告质量和方法学（技术）质量的广泛信息（既有独特条目，也有受其他方法启发的条目），并包含SYRCLE偏倚风险的条目。该工具旨在用于评估多个领域和条目的整体研究质量，与其他工具不同，它不限于任何特定模型或研究设计（无论是干预性还是非干预性）。在设计和进行动物实验时也易于应用，以确保在可重复性、透明度和有效性方面的恰当报告和设计。
