School of Computing and Engineering, University of Huddersfield, Huddersfield HD1 3DH, UK.
School of Cyberspace Security, Xi'an University of Posts and Telecommunications, Xi'an 710061, China.
Sensors (Basel). 2023 Feb 22;23(5):2452. doi: 10.3390/s23052452.
Human action recognition has drawn significant attention because of its importance in computer vision-based applications. Action recognition based on skeleton sequences has rapidly advanced in the last decade. Conventional deep learning-based approaches are based on extracting skeleton sequences through convolutional operations. Most of these architectures are implemented by learning spatial and temporal features through multiple streams. These studies have enlightened the action recognition endeavor from various algorithmic angles. However, three common issues are observed: (1) The models are usually complicated; therefore, they have a correspondingly higher computational complexity. (2) For supervised learning models, the reliance on labels during training is always a drawback. (3) Implementing large models is not beneficial to real-time applications. To address the above issues, in this paper, we propose a multi-layer perceptron (MLP)-based self-supervised learning framework with a contrastive learning loss function (ConMLP). ConMLP does not require a massive computational setup; it can effectively reduce the consumption of computational resources. Compared with supervised learning frameworks, ConMLP is friendly to the huge amount of unlabeled training data. In addition, it has low requirements for system configuration and is more conducive to being embedded in real-world applications. Extensive experiments show that ConMLP achieves the top one inference result of 96.9% on the NTU RGB+D dataset. This accuracy is higher than the state-of-the-art self-supervised learning method. Meanwhile, ConMLP is also evaluated in a supervised learning manner, which has achieved comparable performance to the state of the art of recognition accuracy.
人体动作识别因其在基于计算机视觉的应用中的重要性而受到广泛关注。基于骨骼序列的动作识别在过去十年中得到了快速发展。传统的基于深度学习的方法基于通过卷积操作提取骨骼序列。这些体系结构中的大多数都是通过通过多个流学习空间和时间特征来实现的。这些研究从各种算法角度启发了动作识别的努力。然而，观察到三个常见问题：（1）模型通常很复杂；因此，它们具有相应更高的计算复杂性。（2）对于监督学习模型，训练期间对标签的依赖始终是一个缺点。（3）实施大型模型不利于实时应用。为了解决上述问题，在本文中，我们提出了一种基于多层感知机（MLP）的自监督学习框架，具有对比学习损失函数（ConMLP）。ConMLP 不需要大量的计算设置；它可以有效地减少计算资源的消耗。与监督学习框架相比，ConMLP 对大量未标记的训练数据更友好。此外，它对系统配置的要求较低，更有利于嵌入到实际应用中。大量实验表明，ConMLP 在 NTU RGB+D 数据集上的推断结果达到了 96.9%的最高水平。这一精度高于最先进的自监督学习方法。同时，ConMLP 也以监督学习的方式进行了评估，其识别精度达到了与最先进方法相当的水平。
