School of Future Technology, South China University of Technology, Guangzhou 511442, China; Pazhou Lab, Guangzhou 510320, China; The University of Oxford, Oxford OX14AL, UK.
School of Future Technology, South China University of Technology, Guangzhou 511442, China; Pazhou Lab, Guangzhou 510320, China; Cardiovascular Disease Center, Xiyuan Hospital of China Academy of Chinese Medical Sciences, Beijing 100091, China.
Sci Bull (Beijing). 2024 Sep 30;69(18):2906-2919. doi: 10.1016/j.scib.2024.06.037. Epub 2024 Jul 23.
In medical image segmentation, it is often necessary to collect opinions from multiple experts to make the final decision. This clinical routine helps to mitigate individual bias. However, when data is annotated by multiple experts, standard deep learning models are often not applicable. In this paper, we propose a novel neural network framework called Multi-rater Prism (MrPrism) to learn medical image segmentation from multiple labels. Inspired by iterative half-quadratic optimization, MrPrism combines the task of assigning multi-rater confidences and calibrated segmentation in a recurrent manner. During this process, MrPrism learns inter-observer variability while taking into account the image's semantic properties and finally converges to a self-calibrated segmentation result reflecting inter-observer agreement. Specifically, we propose Converging Prism (ConP) and Diverging Prism (DivP) to iteratively process the two tasks. ConP learns calibrated segmentation based on multi-rater confidence maps estimated by DivP, and DivP generates multi-rater confidence maps based on segmentation masks estimated by ConP. Experimental results show that the two tasks can mutually improve each other through this recurrent process. The final converged segmentation result of MrPrism outperforms state-of-the-art (SOTA) methods for a wide range of medical image segmentation tasks. The code is available at https://github.com/WuJunde/MrPrism.
在医学图像分割中，通常需要收集多个专家的意见来做出最终决策。这种临床常规有助于减轻个体偏见。然而，当数据由多个专家进行注释时，标准的深度学习模型通常不适用。在本文中，我们提出了一种名为多评分者棱镜（MrPrism）的新型神经网络框架，用于从多个标签中学习医学图像分割。受迭代半二次优化的启发，MrPrism 以递归的方式结合了为多评分者分配置信度和校准分割的任务。在这个过程中，MrPrism 在考虑图像语义属性的同时学习观察者间的可变性，最终收敛到反映观察者间一致性的自我校准分割结果。具体来说，我们提出了收敛棱镜（ConP）和发散棱镜（DivP）来迭代处理这两个任务。ConP 根据由 DivP 估计的多评分者置信度图学习校准分割，而 DivP 根据由 ConP 估计的分割掩模生成多评分者置信度图。实验结果表明，这两个任务可以通过这个递归过程相互促进。MrPrism 的最终收敛分割结果在广泛的医学图像分割任务中优于最先进的（SOTA）方法。代码可在 https://github.com/WuJunde/MrPrism 上获得。
