Department of Science and Technology, Linköping University, Linköping, Sweden.
Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden.
Sci Rep. 2022 May 18;12(1):8329. doi: 10.1038/s41598-022-11826-0.
Deep learning (DL) has shown great potential in digital pathology applications. The robustness of a diagnostic DL-based solution is essential for safe clinical deployment. In this work we evaluate if adding uncertainty estimates for DL predictions in digital pathology could result in increased value for the clinical applications, by boosting the general predictive performance or by detecting mispredictions. We compare the effectiveness of model-integrated methods (MC dropout and Deep ensembles) with a model-agnostic approach (Test time augmentation, TTA). Moreover, four uncertainty metrics are compared. Our experiments focus on two domain shift scenarios: a shift to a different medical center and to an underrepresented subtype of cancer. Our results show that uncertainty estimates increase reliability by reducing a model's sensitivity to classification threshold selection as well as by detecting between 70 and 90% of the mispredictions done by the model. Overall, the deep ensembles method achieved the best performance closely followed by TTA.
深度学习（DL）在数字病理学应用中显示出巨大的潜力。基于 DL 的诊断解决方案的稳健性对于安全的临床部署至关重要。在这项工作中，我们评估在数字病理学中为 DL 预测添加不确定性估计是否可以通过提高一般预测性能或通过检测误报来为临床应用带来更多价值。我们将模型集成方法（MC 丢弃和 Deep ensembles）与模型不可知方法（Test time augmentation，TTA）进行比较。此外，还比较了四种不确定性指标。我们的实验集中在两个领域转移场景上：转移到不同的医疗中心和代表性不足的癌症亚型。我们的结果表明，不确定性估计通过降低模型对分类阈值选择的敏感性以及通过检测模型造成的 70%至 90%的误报，从而提高了可靠性。总体而言，Deep ensembles 方法的性能最好，紧随其后的是 TTA。
