Si Liping, Zhong Jingyu, Huo Jiayu, Xuan Kai, Zhuang Zixu, Hu Yangfan, Wang Qian, Zhang Huan, Yao Weiwu
Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, No. 1111 Xianxia Road, Changning District, Shanghai, 200336, China.
Institute for Medical Imaging Technology, School of Biomedical Engineering, Shanghai Jiao Tong University, Huashan Road #1954, Shanghai, 200030, China.
Eur Radiol. 2022 Feb;32(2):1353-1361. doi: 10.1007/s00330-021-08190-4. Epub 2021 Aug 4.
Our purposes were (1) to explore the methodologic quality of the studies on the deep learning in knee imaging with CLAIM criterion and (2) to offer our vision for the development of CLAIM to assure high-quality reports about the application of AI to medical imaging in knee joint.
A Checklist for Artificial Intelligence in Medical Imaging systematic review was conducted from January 1, 2015, to June 1, 2020, using PubMed, EMBASE, and Web of Science databases. A total of 36 articles discussing deep learning applications in knee joint imaging were identified, divided by imaging modality, and characterized by imaging task, data source, algorithm type, and outcome metrics.
A total of 36 studies were identified and divided into: X-ray (44.44%) and MRI (55.56%). The mean CLAIM score of the 36 studies was 27.94 (standard deviation, 4.26), which was 66.53% of the ideal score of 42.00. The CLAIM items achieved an average good inter-rater agreement (ICC 0.815, 95% CI 0.660-0.902). In total, 32 studies performed internal cross-validation on the data set, while only 4 studies conducted external validation of the data set.
The overall scientific quality of deep learning in knee imaging is insufficient; however, deep learning remains a promising technology for diagnostic or predictive purpose. Improvements in study design, validation, and open science need to be made to demonstrate the generalizability of findings and to achieve clinical applications. Widespread application, pre-trained scoring procedure, and modification of CLAIM in response to clinical needs are necessary in the future.
• Limited deep learning studies were established in knee imaging with mean score of 27.94, which was 66.53% of the ideal score of 42.00, commonly due to invalidated results, retrospective study design, and absence of a clear definition of the CLAIM items in detail. • A previous trained data extraction instrument allowed reaching moderate inter-rater agreement in the application of the CLAIM, while CLAIM still needs improvement in scoring items and result reporting to become a wide adaptive tool in reviews of deep learning studies.
我们的目的是（1）使用CLAIM标准探讨膝关节成像深度学习研究的方法学质量，以及（2）提出我们对CLAIM发展的展望，以确保关于人工智能在膝关节医学成像中应用的高质量报告。
于2015年1月1日至2020年6月1日，使用PubMed、EMBASE和Web of Science数据库进行了一项医学成像人工智能系统评价清单研究。共识别出36篇讨论深度学习在膝关节成像中应用的文章，按成像模态进行划分，并根据成像任务、数据源、算法类型和结果指标进行特征描述。
共识别出36项研究，分为：X线（44.44%）和MRI（55.56%）。36项研究的平均CLAIM评分为27.94（标准差，4.26），占理想分数42.00的66.53%。CLAIM项目在评分者间达成了平均良好的一致性（ICC 0.815，95%CI 0.660 - 0.902）。总共32项研究对数据集进行了内部交叉验证，而只有4项研究对数据集进行了外部验证。
膝关节成像深度学习的整体科学质量不足；然而，深度学习仍然是一种用于诊断或预测目的的有前景的技术。需要在研究设计、验证和开放科学方面进行改进，以证明研究结果的可推广性并实现临床应用。未来有必要进行广泛应用、预训练评分程序以及根据临床需求对CLAIM进行修改。
• 在膝关节成像中开展的深度学习研究有限，平均评分为27.94，占理想分数42.00的66.53%，这通常是由于结果无效、回顾性研究设计以及未详细明确CLAIM项目的定义所致。• 先前训练的数据提取工具在CLAIM的应用中实现了评分者间适度的一致性，而CLAIM在评分项目和结果报告方面仍需改进，以成为深度学习研究综述中广泛适用的工具。
