Liu Wenjie, Wu Guoqing, Wang Han, Ren Fuji
School of Transportation and Civil Engineering, Nantong University, Nantong, 226019, China.
Nantong Institute of Technology, Nantong, 226002, China.
Sci Rep. 2025 Jul 2;15(1):22710. doi: 10.1038/s41598-025-09346-8.
The attention mechanism plays a crucial role in enhancing model performance by guiding the model to focus on important features. However, existing attention mechanism methods primarily concentrate on learning attention features within individual modules while ignoring interactions among overall attention features. To overcome this limitation, we propose a dense skip-attention method for convolutional networks - a simple but effective approach to boost performance. Our method establishes dense skip-attention connections that interconnect all attention modules, forcing the model to learn interactive attention features within the network architecture. We conduct extensive experiments on the ImageNet 2012 and Microsoft COCO (MS COCO) 2017 datasets to validate the effectiveness of our approach. The experimental results demonstrate that our method improves the performance of existing attention mechanism methods, such as Squeeze-and-Excitation Networks, Efficient Channel Attention Networks and Convolutional Block Attention Module, in tasks like image classification, object detection, and instance segmentation. Notably, it achieves these improvements without significantly increasing model parameters or computational cost, maintaining minimal impact on both aspects.
注意力机制在引导模型关注重要特征以提升模型性能方面发挥着关键作用。然而，现有的注意力机制方法主要集中在单个模块内学习注意力特征，而忽略了整体注意力特征之间的交互。为克服这一局限性，我们提出了一种用于卷积网络的密集跳跃注意力方法——一种简单但有效的提升性能的方法。我们的方法建立了连接所有注意力模块的密集跳跃注意力连接，迫使模型在网络架构内学习交互式注意力特征。我们在ImageNet 2012和微软COCO（MS COCO）2017数据集上进行了广泛实验，以验证我们方法的有效性。实验结果表明，我们的方法在图像分类、目标检测和实例分割等任务中，提高了诸如挤压激励网络、高效通道注意力网络和卷积块注意力模块等现有注意力机制方法的性能。值得注意的是，它在不显著增加模型参数或计算成本的情况下实现了这些改进，在这两方面保持了最小影响。
