Ningbo Eye Hospital, Wenzhou Medical University, Ningbo, 315000, China; School of Ophthalmology and Optometry and Eye Hospital, Wenzhou Medical University, Wenzhou, 325027, China.
School of Electronics Engineering, Xi'an University of Posts and Telecommunications, Xi'an, 710121, China.
Int J Med Inform. 2021 Mar;147:104363. doi: 10.1016/j.ijmedinf.2020.104363. Epub 2020 Dec 13.
Recent advances in artificial intelligence (AI) have shown great promise in detecting some diseases based on medical images. Most studies developed AI diagnostic systems only using eligible images. However, in real-world settings, ineligible images (including poor-quality and poor-location images) that can compromise downstream analysis are inevitable, leading to uncertainty about the performance of these AI systems. This study aims to develop a deep learning-based image eligibility verification system (DLIEVS) for detecting and filtering out ineligible fundus images.
A total of 18,031 fundus images (9,188 subjects) collected from 4 clinical centres were used to develop and evaluate the DLIEVS for detecting eligible, poor-location, and poor-quality fundus images. Four deep learning algorithms (AlexNet, DenseNet121, Inception V3, and ResNet50) were leveraged to train models to obtain the best model for the DLIEVS. The performance of the DLIEVS was evaluated using the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity, as compared with a reference standard determined by retina experts.
In the internal test dataset, the best algorithm (DenseNet121) achieved AUCs of 1.000, 0.999, and 1.000 for the classification of eligible, poor-location, and poor-quality images, respectively. In the external test datasets, the AUCs of the best algorithm (DenseNet121) for detecting eligible, poor-location, and poor-quality images were ranged from 0.999-1.000, 0.997-1.000, and 0.997-0.999, respectively.
Our DLIEVS can accurately discriminate poor-quality and poor-location images from eligible images. This system has the potential to serve as a pre-screening technique to filter out ineligible images obtained from real-world settings, ensuring only eligible images will be applied in the subsequent image-based AI diagnostic analyses.
人工智能（AI）的最新进展显示，基于医学图像检测某些疾病具有很大的潜力。大多数研究仅使用合格的图像来开发 AI 诊断系统。然而，在实际环境中，不可靠的图像（包括质量差和位置不佳的图像）不可避免地会影响下游分析，这导致对这些 AI 系统性能的不确定性。本研究旨在开发一种基于深度学习的图像合格性验证系统（DLIEVS），用于检测和筛选不合格的眼底图像。
共使用来自 4 个临床中心的 18031 张眼底图像（9188 名患者）来开发和评估 DLIEVS 以检测合格、位置不佳和质量差的眼底图像。利用 4 种深度学习算法（AlexNet、DenseNet121、Inception V3 和 ResNet50）来训练模型，以获得 DLIEVS 的最佳模型。通过与视网膜专家确定的参考标准相比，使用受试者工作特征曲线下的面积（AUC）、敏感性和特异性来评估 DLIEVS 的性能。
在内部测试数据集，最佳算法（DenseNet121）对合格、位置不佳和质量差图像的分类的 AUC 分别为 1.000、0.999 和 1.000。在外部测试数据集，最佳算法（DenseNet121）对检测合格、位置不佳和质量差图像的 AUC 范围分别为 0.999-1.000、0.997-1.000 和 0.997-0.999。
我们的 DLIEVS 可以准确地区分合格图像与质量差和位置不佳的图像。该系统有可能作为一种预筛选技术，从实际环境中筛选出不合格的图像，以确保只有合格的图像应用于随后的基于图像的 AI 诊断分析中。
