Liu Sheng-Lan, Ding Yu-Ning, Zhang Jin-Rong, Liu Kai-Yuan, Zhang Si-Fan, Wang Fei-Long, Huang Gao
IEEE Trans Neural Netw Learn Syst. 2025 Apr;36(4):7615-7626. doi: 10.1109/TNNLS.2024.3384770. Epub 2025 Apr 4.
Graph convolutional networks (GCNs) have been widely used in skeleton-based action recognition. However, existing approaches are limited in fine-grained action recognition due to the similarity of interclass data. Moreover, the noisy data from pose extraction increase the challenge of fine-grained recognition. In this work, we propose a flexible attention block called channel-variable spatial-temporal attention (CVSTA) to enhance the discriminative power of spatial-temporal joints and obtain a more compact intraclass feature distribution. Based on CVSTA, we construct a multidimensional refinement GCN (MDR-GCN) that can improve the discrimination among channel-, joint-, and frame-level features for fine-grained actions. Furthermore, we propose a robust decouple loss (RDL) that significantly boosts the effect of the CVSTA and reduces the impact of noise. The proposed method combining MDR-GCN with RDL outperforms the known state-of-the-art skeleton-based approaches on fine-grained datasets, FineGym99 and FSD-10, and also on the coarse NTU-RGB + D 120 dataset and NTU-RGB + D X-view version. Our code is publicly available at https://github.com/dingyn-Reno/MDR-GCN.
图卷积网络（GCN）已被广泛应用于基于骨架的动作识别。然而，由于类间数据的相似性，现有的方法在细粒度动作识别方面存在局限性。此外，姿态提取过程中产生的噪声数据增加了细粒度识别的挑战。在这项工作中，我们提出了一种灵活的注意力模块，称为通道可变时空注意力（CVSTA），以增强时空关节的判别能力，并获得更紧凑的类内特征分布。基于CVSTA，我们构建了一个多维细化GCN（MDR-GCN），它可以提高细粒度动作在通道、关节和帧级特征之间的辨别能力。此外，我们还提出了一种鲁棒解耦损失（RDL），它显著增强了CVSTA的效果，并降低了噪声的影响。将MDR-GCN与RDL相结合的方法在细粒度数据集FineGym99和FSD-10以及粗糙的NTU-RGB + D 120数据集和NTU-RGB + D X-view版本上优于已知的基于骨架的先进方法。我们的代码可在https://github.com/dingyn-Reno/MDR-GCN上公开获取。
