Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Istanbul, Turkey.
Institute of Radiology and Nuclear Medicine, Cantonal Hospital Baselland, Liestal, Switzerland.
Eur Radiol. 2024 Apr;34(4):2805-2815. doi: 10.1007/s00330-023-10243-9. Epub 2023 Sep 22.
To evaluate the usage of a well-known and widely adopted checklist, Checklist for Artificial Intelligence in Medical imaging (CLAIM), for self-reporting through a systematic analysis of its citations.
Google Scholar, Web of Science, and Scopus were used to search for citations (date, 29 April 2023). CLAIM's use for self-reporting with proof (i.e., filled-out checklist) and other potential use cases were systematically assessed in research papers. Eligible papers were evaluated independently by two readers, with the help of automatic annotation. Item-by-item confirmation analysis on papers with checklist proof was subsequently performed.
A total of 391 unique citations were identified from three databases. Of the 118 papers included in this study, 12 (10%) provided a proof of self-reported CLAIM checklist. More than half (70; 59%) only mentioned some sort of adherence to CLAIM without providing any proof in the form of a checklist. Approximately one-third (36; 31%) cited the CLAIM for reasons unrelated to their reporting or methodological adherence. Overall, the claims on 57 to 93% of the items per publication were confirmed in the item-by-item analysis, with a mean and standard deviation of 81% and 10%, respectively.
Only a small proportion of the publications used CLAIM as checklist and supplied filled-out documentation; however, the self-reported checklists may contain errors and should be approached cautiously. We hope that this systematic citation analysis would motivate artificial intelligence community about the importance of proper self-reporting, and encourage researchers, journals, editors, and reviewers to take action to ensure the proper usage of checklists.
Only a small percentage of the publications used CLAIM for self-reporting with proof (i.e., filled-out checklist). However, the filled-out checklist proofs may contain errors, e.g., false claims of adherence, and should be approached cautiously. These may indicate inappropriate usage of checklists and necessitate further action by authorities.
• Of 118 eligible papers, only 12 (10%) followed the CLAIM checklist for self-reporting with proof (i.e., filled-out checklist). More than half (70; 59%) only mentioned some kind of adherence without providing any proof. • Overall, claims on 57 to 93% of the items were valid in item-by-item confirmation analysis, with a mean and standard deviation of 81% and 10%, respectively. • Even with the checklist proof, the items declared may contain errors and should be approached cautiously.
通过系统分析其引用情况，评估一个广为人知且广泛应用的清单——医学影像人工智能清单（CLAIM）在自我报告中的使用情况。
使用 Google Scholar、Web of Science 和 Scopus 搜索引文（截至 2023 年 4 月 29 日）。系统评估研究论文中 CLAIM 用于自我报告的证明（即填写的清单）和其他潜在用途。由两名读者独立评估符合条件的论文，并在自动标注的帮助下进行评估。随后对具有清单证明的论文进行逐项确认分析。
从三个数据库中总共确定了 391 个唯一的引文。在纳入本研究的 118 篇论文中，有 12 篇（10%）提供了 CLAIM 自我报告清单的证明。超过一半（70 篇；59%）仅提到某种程度上遵守 CLAIM，但没有以清单形式提供任何证明。大约三分之一（36 篇；31%）引用 CLAIM 是出于与其报告或方法遵守无关的原因。总体而言，在逐项分析中，每篇论文有 57%至 93%的项目得到确认，平均值和标准差分别为 81%和 10%。
只有一小部分出版物将 CLAIM 用作清单并提供了填写好的文件；然而，自我报告的清单可能包含错误，因此应谨慎对待。我们希望这项系统的引文分析能促使人工智能社区认识到正确自我报告的重要性，并鼓励研究人员、期刊、编辑和审稿人采取行动，确保清单的正确使用。
只有一小部分出版物（10%）使用 CLAIM 进行自我报告证明（即填写好的清单）。然而，填写好的清单证明可能包含错误，例如虚假声称的遵守情况，因此应谨慎对待。这可能表明清单使用不当，需要当局采取进一步行动。
在 118 篇符合条件的论文中，只有 12 篇（10%）按照 CLAIM 清单进行了自我报告证明（即填写了清单）。超过一半（70 篇；59%）仅提到某种程度上的遵守，而没有提供任何证明。
在逐项确认分析中，整体上有 57%至 93%的项目的声明有效，平均值和标准差分别为 81%和 10%。
即使有清单证明，所声明的项目也可能包含错误，因此应谨慎对待。
