Beijing Tongren Eye Center, Beijing Key Laboratory of Intraocular Tumor Diagnosis and Treatment, Beijing Tongren Hospital, Capital Medical University, Beijing, China; Beijing Ophthalmology and Visual Sciences Key Laboratory, Beijing Tongren Hospital, Capital Medical University, Beijing, China; Medical Artificial Intelligence Research and Verification Key Laboratory of the Ministry of Industry and Information Technology, Beijing Tongren Hospital, Capital Medical University, Beijing, China.
Beijing Airdoc Technology Co., Ltd., Beijing, China; Augmented Intelligence and Multimodal Analytics (AIM) for Health Lab, Faculty of Information Technology, Monash University, Clayton, Australia; Faculty of Engineering, Monash University, Clayton, Australia.
Ophthalmol Retina. 2024 Jul;8(7):666-677. doi: 10.1016/j.oret.2024.01.019. Epub 2024 Jan 26.
We aimed to develop a deep learning system capable of identifying subjects with cognitive impairment quickly and easily based on multimodal ocular images.
Cross sectional study.
Participants of Beijing Eye Study 2011 and patients attending Beijing Tongren Eye Center and Beijing Tongren Hospital Physical Examination Center.
We trained and validated a deep learning algorithm to assess cognitive impairment using retrospectively collected data from the Beijing Eye Study 2011. Cognitive impairment was defined as a Mini-Mental State Examination score < 24. Based on fundus photographs and OCT images, we developed 5 models based on the following sets of images: macula-centered fundus photographs, optic disc-centered fundus photographs, fundus photographs of both fields, OCT images, and fundus photographs of both fields with OCT (multimodal). The performance of the models was evaluated and compared in an external validation data set, which was collected from patients attending Beijing Tongren Eye Center and Beijing Tongren Hospital Physical Examination Center.
Area under the curve (AUC).
A total of 9424 retinal photographs and 4712 OCT images were used to develop the model. The external validation sets from each center included 1180 fundus photographs and 590 OCT images. Model comparison revealed that the multimodal performed best, achieving an AUC of 0.820 in the internal validation set, 0.786 in external validation set 1, and 0.784 in external validation set 2. We evaluated the performance of the multi-model in different sexes and different age groups; there were no significant differences. The heatmap analysis showed that signals around the optic disc in fundus photographs and the retina and choroid around the macular and optic disc regions in OCT images were used by the multimodal to identify participants with cognitive impairment.
Fundus photographs and OCT can provide valuable information on cognitive function. Multimodal models provide richer information compared with single-mode models. Deep learning algorithms based on multimodal retinal images may be capable of screening cognitive impairment. This technique has potential value for broader implementation in community-based screening or clinic settings.
FINANCIAL DISCLOSURE(S): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
我们旨在开发一种深度学习系统，能够基于多模态眼部图像快速、轻松地识别认知障碍患者。
横断面研究。
北京眼研究 2011 年的参与者和北京同仁眼中心及北京同仁医院体检中心就诊的患者。
我们使用北京眼研究 2011 年回顾性收集的数据训练和验证了一种深度学习算法，以评估认知障碍。认知障碍定义为简易精神状态检查评分<24。基于眼底照片和 OCT 图像，我们基于以下图像集开发了 5 种模型：黄斑中心眼底照片、视盘中心眼底照片、双眼眼底照片、OCT 图像和双眼眼底加 OCT（多模态）。在来自北京同仁眼中心和北京同仁医院体检中心就诊的患者的外部验证数据集中评估和比较了模型的性能。
曲线下面积（AUC）。
共使用 9424 张视网膜照片和 4712 张 OCT 图像来开发模型。来自每个中心的外部验证集包括 1180 张眼底照片和 590 张 OCT 图像。模型比较显示，多模态在内部验证集中的 AUC 为 0.820，在外部验证集 1 中的 AUC 为 0.786，在外部验证集 2 中的 AUC 为 0.784，表现最佳。我们评估了多模型在不同性别和不同年龄组中的性能；没有显著差异。热图分析显示，眼底照片中视盘周围的信号和 OCT 图像中黄斑和视盘区域周围的视网膜和脉络膜被多模态用于识别认知障碍患者。
眼底照片和 OCT 可以提供有价值的认知功能信息。与单模态模型相比，多模态模型提供了更丰富的信息。基于多模态视网膜图像的深度学习算法可能能够筛查认知障碍。这项技术在社区筛查或临床环境中具有广泛实施的潜在价值。
本文末尾的脚注和披露中可能包含专有或商业披露。
