Suppr超能文献

基于融合 Transformer 与并行 CNN 的微观高光谱图像分类。

Microscopic Hyperspectral Image Classification Based on Fusion Transformer With Parallel CNN.

出版信息

IEEE J Biomed Health Inform. 2023 Jun;27(6):2910-2921. doi: 10.1109/JBHI.2023.3253722. Epub 2023 Jun 5.

Abstract

Microscopic hyperspectral image (MHSI) has received considerable attention in the medical field. The wealthy spectral information provides potentially powerful identification ability when combining with advanced convolutional neural network (CNN). However, for high-dimensional MHSI, the local connection of CNN makes it difficult to extract the long-range dependencies of spectral bands. Transformer overcomes this problem well because of its self-attention mechanism. Nevertheless, transformer is inferior to CNN in extracting spatial detailed features. Therefore, a classification framework integrating transformer and CNN in parallel, named as Fusion Transformer (FUST), is proposed for MHSI classification tasks. Specifically, the transformer branch is employed to extract the overall semantics and capture the long-range dependencies of spectral bands to highlight the key spectral information. The parallel CNN branch is designed to extract significant multiscale spatial features. Furthermore, the feature fusion module is developed to effectively fuse and process the features extracted by the two branches. Experimental results on three MHSI datasets demonstrate that the proposed FUST achieves superior performance when compared with state-of-the-art methods.

摘要

显微高光谱图像 (MHSI) 在医学领域受到了相当多的关注。当与先进的卷积神经网络 (CNN) 结合使用时,其丰富的光谱信息提供了潜在的强大识别能力。然而,对于高维 MHSI,CNN 的局部连接使得很难提取光谱带的远程依赖关系。Transformer 由于其自注意力机制很好地克服了这个问题。然而,Transformer 在提取空间细节特征方面逊于 CNN。因此,提出了一种将 Transformer 和 CNN 并行集成的分类框架,称为融合 Transformer (FUST),用于 MHSI 分类任务。具体来说,使用 Transformer 分支提取整体语义并捕获光谱带的远程依赖关系,以突出关键的光谱信息。并行的 CNN 分支用于提取重要的多尺度空间特征。此外,开发了特征融合模块来有效地融合和处理两个分支提取的特征。在三个 MHSI 数据集上的实验结果表明,与最先进的方法相比,所提出的 FUST 具有优越的性能。

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验