Christian Doppler Laboratory for Artificial Intelligence in Retina, Department of Ophthalmology and Optometry, Medical University of Vienna, Vienna, Austria.
Sci Rep. 2023 Sep 27;13(1):16231. doi: 10.1038/s41598-023-43018-9.
Deep neural networks have been increasingly proposed for automated screening and diagnosis of retinal diseases from optical coherence tomography (OCT), but often provide high-confidence predictions on out-of-distribution (OOD) cases, compromising their clinical usage. With this in mind, we performed an in-depth comparative analysis of the state-of-the-art uncertainty estimation methods for OOD detection in retinal OCT imaging. The analysis was performed within the use-case of automated screening and staging of age-related macular degeneration (AMD), one of the leading causes of blindness worldwide, where we achieved a macro-average area under the curve (AUC) of 0.981 for AMD classification. We focus on a few-shot Outlier Exposure (OE) method and the detection of near-OOD cases that share pathomorphological characteristics with the inlier AMD classes. Scoring the OOD case based on the Cosine distance in the feature space from the penultimate network layer proved to be a robust approach for OOD detection, especially in combination with the OE. Using Cosine distance and only 8 outliers exposed per class, we were able to improve the near-OOD detection performance of the OE with Reject Bucket method by [Formula: see text] 10% compared to without OE, reaching an AUC of 0.937. The Cosine distance served as a robust metric for OOD detection of both known and unknown classes and should thus be considered as an alternative to the reject bucket class probability in OE approaches, especially in the few-shot scenario. The inclusion of these methodologies did not come at the expense of classification performance, and can substantially improve the reliability and trustworthiness of the resulting deep learning-based diagnostic systems in the context of retinal OCT.
深度神经网络已被越来越多地用于从光学相干断层扫描（OCT）中自动筛选和诊断视网膜疾病，但它们经常对离群（OOD）病例提供高度置信的预测，从而影响其临床应用。考虑到这一点，我们对用于视网膜 OCT 成像的 OOD 检测的最新不确定性估计方法进行了深入的比较分析。该分析是在自动筛选和分期年龄相关性黄斑变性（AMD）的用例中进行的，AMD 是全球导致失明的主要原因之一，我们在 AMD 分类方面实现了宏观平均曲线下面积（AUC）为 0.981。我们专注于少数样本外暴露（OE）方法和检测与内联 AMD 类具有相似病理形态特征的近 OOD 病例。基于倒数第二层网络层的特征空间中的余弦距离对 OOD 病例进行评分被证明是一种用于 OOD 检测的稳健方法，尤其是与 OE 结合使用时。使用余弦距离和每个类仅暴露 8 个异常值，我们能够将具有 Reject Bucket 方法的 OE 的近 OOD 检测性能提高 [Formula: see text] 与没有 OE 相比，提高了 10%，达到 AUC 为 0.937。余弦距离是用于检测已知和未知类别的稳健指标，因此应被视为 OE 方法中拒绝桶类概率的替代方法，尤其是在少数样本情况下。包含这些方法并不会影响分类性能，并且可以大大提高基于深度学习的诊断系统在视网膜 OCT 背景下的可靠性和可信度。
