Suppr超能文献

使用3D Mask R-CNN对合成数据和胚胎显微镜图像进行端到端3D实例分割。

End-to-end 3D instance segmentation of synthetic data and embryo microscopy images with a 3D Mask R-CNN.

作者信息

David Gabriel, Faure Emmanuel

机构信息

Laboratoire d'Informatique, de Robotique et de Micro-électronique de Montpellier, Centre National de la Recherche Scientifique, Université Montpellier, Montpellier, France.

出版信息

Front Bioinform. 2025 Jan 29;4:1497539. doi: 10.3389/fbinf.2024.1497539. eCollection 2024.

Abstract

In recent years, the exploitation of three-dimensional (3D) data in deep learning has gained momentum despite its inherent challenges. The necessity of 3D approaches arises from the limitations of two-dimensional (2D) techniques when applied to 3D data due to the lack of global context. A critical task in medical and microscopy 3D image analysis is instance segmentation, which is inherently complex due to the need for accurately identifying and segmenting multiple object instances in an image. Here, we introduce a 3D adaptation of the Mask R-CNN, a powerful end-to-end network designed for instance segmentation. Our implementation adapts a widely used 2D TensorFlow Mask R-CNN by developing custom TensorFlow operations for 3D Non-Max Suppression and 3D Crop And Resize, facilitating efficient training and inference on 3D data. We validate our 3D Mask R-CNN on two experiences. The first experience uses a controlled environment of synthetic data with instances exhibiting a wide range of anisotropy and noise. Our model achieves good results while illustrating the limit of the 3D Mask R-CNN for the noisiest objects. Second, applying it to real-world data involving cell instance segmentation during the morphogenesis of the ascidian embryo , we show that our 3D Mask R-CNN outperforms the state-of-the-art method, achieving high recall and precision scores. The model preserves cell connectivity, which is crucial for applications in quantitative study. Our implementation is open source, ensuring reproducibility and facilitating further research in 3D deep learning.

摘要

近年来,尽管存在固有挑战,但深度学习中三维(3D)数据的利用仍在加速发展。由于缺乏全局上下文,二维(2D)技术在应用于3D数据时存在局限性,因此3D方法成为必要。医学和显微镜3D图像分析中的一项关键任务是实例分割,由于需要在图像中准确识别和分割多个对象实例,这一任务本身就很复杂。在此,我们介绍了Mask R-CNN的3D改编版,这是一种专为实例分割设计的强大端到端网络。我们的实现通过开发用于3D非极大值抑制和3D裁剪与调整大小的自定义TensorFlow操作,对广泛使用的2D TensorFlow Mask R-CNN进行了改编,便于在3D数据上进行高效训练和推理。我们在两项实验中验证了我们的3D Mask R-CNN。第一项实验使用了合成数据的受控环境,其中实例表现出广泛的各向异性和噪声。我们的模型取得了良好的结果,同时也说明了3D Mask R-CNN对噪声最大的对象的局限性。其次,将其应用于涉及海鞘胚胎形态发生过程中细胞实例分割的真实世界数据,我们表明我们的3D Mask R-CNN优于现有方法,实现了高召回率和精确率得分。该模型保留了细胞连通性,这对于定量研究中的应用至关重要。我们的实现是开源的,确保了可重复性,并便于在3D深度学习中进行进一步研究。

https://cdn.ncbi.nlm.nih.gov/pmc/blobs/b6a0/11814465/ef82fe93891d/fbinf-04-1497539-g001.jpg

相似文献

1
End-to-end 3D instance segmentation of synthetic data and embryo microscopy images with a 3D Mask R-CNN.
Front Bioinform. 2025 Jan 29;4:1497539. doi: 10.3389/fbinf.2024.1497539. eCollection 2024.
2
Mask-Refined R-CNN: A Network for Refining Object Details in Instance Segmentation.
Sensors (Basel). 2020 Feb 13;20(4):1010. doi: 10.3390/s20041010.
3
Fusing attention mechanism with Mask R-CNN for instance segmentation of grape cluster in the field.
Front Plant Sci. 2022 Jul 22;13:934450. doi: 10.3389/fpls.2022.934450. eCollection 2022.
4
Improved-Mask R-CNN: Towards an accurate generic MSK MRI instance segmentation platform (data from the Osteoarthritis Initiative).
Comput Med Imaging Graph. 2022 Apr;97:102056. doi: 10.1016/j.compmedimag.2022.102056. Epub 2022 Mar 19.
5
SCG: Saliency and Contour Guided Salient Instance Segmentation.
IEEE Trans Image Process. 2021;30:5862-5874. doi: 10.1109/TIP.2021.3088282. Epub 2021 Jun 28.
6
Pan-cancer image segmentation based on feature pyramids and Mask R-CNN framework.
Med Phys. 2024 Aug;51(8):5427-5440. doi: 10.1002/mp.17014. Epub 2024 Mar 4.
7
Object Detection Based on the GrabCut Method for Automatic Mask Generation.
Micromachines (Basel). 2022 Nov 28;13(12):2095. doi: 10.3390/mi13122095.
9
Co-Mask R-CNN: collaborative learning-based method for tooth instance segmentation.
J Clin Pediatr Dent. 2024 Nov;48(6):161-172. doi: 10.22514/jocpd.2024.136. Epub 2024 Nov 3.
10

本文引用的文献

1
Cellpose 2.0: how to train your own model.
Nat Methods. 2022 Dec;19(12):1634-1641. doi: 10.1038/s41592-022-01663-4. Epub 2022 Nov 7.
2
A survey on applications of deep learning in microscopy image analysis.
Comput Biol Med. 2021 Jul;134:104523. doi: 10.1016/j.compbiomed.2021.104523. Epub 2021 May 29.
3
Democratising deep learning for microscopy with ZeroCostDL4Mic.
Nat Commun. 2021 Apr 15;12(1):2276. doi: 10.1038/s41467-021-22518-0.
4
Cellpose: a generalist algorithm for cellular segmentation.
Nat Methods. 2021 Jan;18(1):100-106. doi: 10.1038/s41592-020-01018-x. Epub 2020 Dec 14.
5
3D Deep Learning on Medical Images: A Review.
Sensors (Basel). 2020 Sep 7;20(18):5097. doi: 10.3390/s20185097.
6
Accurate and versatile 3D segmentation of plant tissues at cellular resolution.
Elife. 2020 Jul 29;9:e57613. doi: 10.7554/eLife.57613.
8
MorphoNet: an interactive online morphological browser to explore complex multi-scale data.
Nat Commun. 2019 Jun 27;10(1):2812. doi: 10.1038/s41467-019-10668-1.
9
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.
IEEE Trans Pattern Anal Mach Intell. 2017 Jun;39(6):1137-1149. doi: 10.1109/TPAMI.2016.2577031. Epub 2016 Jun 6.

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验