David Gabriel, Faure Emmanuel
Laboratoire d'Informatique, de Robotique et de Micro-électronique de Montpellier, Centre National de la Recherche Scientifique, Université Montpellier, Montpellier, France.
Front Bioinform. 2025 Jan 29;4:1497539. doi: 10.3389/fbinf.2024.1497539. eCollection 2024.
In recent years, the exploitation of three-dimensional (3D) data in deep learning has gained momentum despite its inherent challenges. The necessity of 3D approaches arises from the limitations of two-dimensional (2D) techniques when applied to 3D data due to the lack of global context. A critical task in medical and microscopy 3D image analysis is instance segmentation, which is inherently complex due to the need for accurately identifying and segmenting multiple object instances in an image. Here, we introduce a 3D adaptation of the Mask R-CNN, a powerful end-to-end network designed for instance segmentation. Our implementation adapts a widely used 2D TensorFlow Mask R-CNN by developing custom TensorFlow operations for 3D Non-Max Suppression and 3D Crop And Resize, facilitating efficient training and inference on 3D data. We validate our 3D Mask R-CNN on two experiences. The first experience uses a controlled environment of synthetic data with instances exhibiting a wide range of anisotropy and noise. Our model achieves good results while illustrating the limit of the 3D Mask R-CNN for the noisiest objects. Second, applying it to real-world data involving cell instance segmentation during the morphogenesis of the ascidian embryo , we show that our 3D Mask R-CNN outperforms the state-of-the-art method, achieving high recall and precision scores. The model preserves cell connectivity, which is crucial for applications in quantitative study. Our implementation is open source, ensuring reproducibility and facilitating further research in 3D deep learning.
近年来,尽管存在固有挑战,但深度学习中三维(3D)数据的利用仍在加速发展。由于缺乏全局上下文,二维(2D)技术在应用于3D数据时存在局限性,因此3D方法成为必要。医学和显微镜3D图像分析中的一项关键任务是实例分割,由于需要在图像中准确识别和分割多个对象实例,这一任务本身就很复杂。在此,我们介绍了Mask R-CNN的3D改编版,这是一种专为实例分割设计的强大端到端网络。我们的实现通过开发用于3D非极大值抑制和3D裁剪与调整大小的自定义TensorFlow操作,对广泛使用的2D TensorFlow Mask R-CNN进行了改编,便于在3D数据上进行高效训练和推理。我们在两项实验中验证了我们的3D Mask R-CNN。第一项实验使用了合成数据的受控环境,其中实例表现出广泛的各向异性和噪声。我们的模型取得了良好的结果,同时也说明了3D Mask R-CNN对噪声最大的对象的局限性。其次,将其应用于涉及海鞘胚胎形态发生过程中细胞实例分割的真实世界数据,我们表明我们的3D Mask R-CNN优于现有方法,实现了高召回率和精确率得分。该模型保留了细胞连通性,这对于定量研究中的应用至关重要。我们的实现是开源的,确保了可重复性,并便于在3D深度学习中进行进一步研究。