Suppr超能文献

O-Net:一种将卷积神经网络(CNN)与Transformer深度融合以实现同步分割和分类的新型框架。

O-Net: A Novel Framework With Deep Fusion of CNN and Transformer for Simultaneous Segmentation and Classification.

作者信息

Wang Tao, Lan Junlin, Han Zixin, Hu Ziwei, Huang Yuxiu, Deng Yanglin, Zhang Hejun, Wang Jianchao, Chen Musheng, Jiang Haiyan, Lee Ren-Guey, Gao Qinquan, Du Ming, Tong Tong, Chen Gang

机构信息

College of Physics and Information Engineering, Fuzhou University, Fuzhou, China.

Fujian Key Lab of Medical Instrumentation and Pharmaceutical Technology, Fuzhou University, Fuzhou, China.

出版信息

Front Neurosci. 2022 Jun 2;16:876065. doi: 10.3389/fnins.2022.876065. eCollection 2022.

Abstract

The application of deep learning in the medical field has continuously made huge breakthroughs in recent years. Based on convolutional neural network (CNN), the U-Net framework has become the benchmark of the medical image segmentation task. However, this framework cannot fully learn global information and remote semantic information. The transformer structure has been demonstrated to capture global information relatively better than the U-Net, but the ability to learn local information is not as good as CNN. Therefore, we propose a novel network referred to as the O-Net, which combines the advantages of CNN and transformer to fully use both the global and the local information for improving medical image segmentation and classification. In the encoder part of our proposed O-Net framework, we combine the CNN and the Swin Transformer to acquire both global and local contextual features. In the decoder part, the results of the Swin Transformer and the CNN blocks are fused to get the final results. We have evaluated the proposed network on the synapse multi-organ CT dataset and the ISIC 2017 challenge dataset for the segmentation task. The classification network is simultaneously trained by using the encoder weights of the segmentation network. The experimental results show that our proposed O-Net achieves superior segmentation performance than state-of-the-art approaches, and the segmentation results are beneficial for improving the accuracy of the classification task. The codes and models of this study are available at https://github.com/ortonwang/O-Net.

摘要

近年来,深度学习在医学领域的应用不断取得巨大突破。基于卷积神经网络(CNN)的U-Net框架已成为医学图像分割任务的基准。然而,该框架无法充分学习全局信息和远程语义信息。已证明变压器结构在捕获全局信息方面比U-Net相对更好,但学习局部信息的能力不如CNN。因此,我们提出了一种新颖的网络,称为O-Net,它结合了CNN和变压器的优点,以充分利用全局和局部信息来改进医学图像分割和分类。在我们提出的O-Net框架的编码器部分,我们将CNN和Swin Transformer结合起来,以获取全局和局部上下文特征。在解码器部分,将Swin Transformer和CNN块的结果融合以获得最终结果。我们在突触多器官CT数据集和ISIC 2017挑战数据集上对提出的网络进行了分割任务评估。分类网络通过使用分割网络的编码器权重同时进行训练。实验结果表明,我们提出的O-Net比现有方法具有更好的分割性能,并且分割结果有利于提高分类任务的准确性。本研究的代码和模型可在https://github.com/ortonwang/O-Net上获取。

https://cdn.ncbi.nlm.nih.gov/pmc/blobs/868c/9201625/b8b58cefbf9f/fnins-16-876065-g0001.jpg

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验