Peng Yuanyuan, Lin Aidi, Wang Meng, Lin Tian, Liu Linna, Wu Jianhua, Zou Ke, Shi Tingkun, Feng Lixia, Liang Zhen, Li Tao, Liang Dan, Yu Shanshan, Sun Dawei, Luo Jing, Gao Ling, Chen Xinjian, Cheng Ching-Yu, Fu Huazhu, Chen Haoyu
School of Biomedical Engineering, Anhui Medical University, Hefei, Anhui 230032, China.
Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong Kong, Shantou, Guangdong 515041, China.
Cell Rep Med. 2025 Jan 21;6(1):101876. doi: 10.1016/j.xcrm.2024.101876. Epub 2024 Dec 19.
Inability to express the confidence level and detect unseen disease classes limits the clinical implementation of artificial intelligence in the real world. We develop a foundation model with uncertainty estimation (FMUE) to detect 16 retinal conditions on optical coherence tomography (OCT). In the internal test set, FMUE achieves a higher F1 score of 95.74% than other state-of-the-art algorithms (92.03%-93.66%) and improves to 97.44% with threshold strategy. The model achieves similar excellent performance on two external test sets from the same and different OCT machines. In human-model comparison, FMUE achieves a higher F1 score of 96.30% than retinal experts (86.95%, p = 0.004), senior doctors (82.71%, p < 0.001), junior doctors (66.55%, p < 0.001), and generative pretrained transformer 4 with vision (GPT-4V) (32.39%, p < 0.001). Besides, FMUE predicts high uncertainty scores for >85% images of non-target-category diseases or with low quality to prompt manual checks and prevent misdiagnosis. Our FMUE provides a trustworthy method for automatic retinal anomaly detection in a clinical open-set environment.
无法表达置信度以及检测未见过的疾病类别限制了人工智能在现实世界中的临床应用。我们开发了一种具有不确定性估计的基础模型（FMUE），用于在光学相干断层扫描（OCT）上检测16种视网膜疾病。在内部测试集中，FMUE实现了95.74%的更高F1分数，高于其他先进算法（92.03%-93.66%），并通过阈值策略提高到97.44%。该模型在来自相同和不同OCT机器的两个外部测试集上也取得了类似的优异性能。在人机模型比较中，FMUE实现了96.30%的更高F1分数，高于视网膜专家（86.95%，p = 0.004）、高级医生（82.71%，p < 0.001）、初级医生（66.55%，p < 0.001）以及具有视觉的生成式预训练变压器4（GPT-4V）（32.39%，p < 0.001）。此外，FMUE对超过85%的非目标类别疾病图像或低质量图像预测出高不确定性分数，以提示人工检查并防止误诊。我们的FMUE为临床开放集环境中的视网膜异常自动检测提供了一种可靠的方法。
