Mixed Reality and Interaction Lab, Department of Software, Sejong University, Seoul 05006, Republic of Korea.
Sensors (Basel). 2024 Mar 16;24(6):1908. doi: 10.3390/s24061908.
Physical exercise affects many facets of life, including mental health, social interaction, physical fitness, and illness prevention, among many others. Therefore, several AI-driven techniques have been developed in the literature to recognize human physical activities. However, these techniques fail to adequately learn the temporal and spatial features of the data patterns. Additionally, these techniques are unable to fully comprehend complex activity patterns over different periods, emphasizing the need for enhanced architectures to further increase accuracy by learning spatiotemporal dependencies in the data individually. Therefore, in this work, we develop an attention-enhanced dual-stream network (PAR-Net) for physical activity recognition with the ability to extract both spatial and temporal features simultaneously. The PAR-Net integrates convolutional neural networks (CNNs) and echo state networks (ESNs), followed by a self-attention mechanism for optimal feature selection. The dual-stream feature extraction mechanism enables the PAR-Net to learn spatiotemporal dependencies from actual data. Furthermore, the incorporation of a self-attention mechanism makes a substantial contribution by facilitating targeted attention on significant features, hence enhancing the identification of nuanced activity patterns. The PAR-Net was evaluated on two benchmark physical activity recognition datasets and achieved higher performance by surpassing the baselines comparatively. Additionally, a thorough ablation study was conducted to determine the best optimal model for human physical activity recognition.
体育锻炼影响着生活的方方面面，包括心理健康、社会交往、身体健康和疾病预防等。因此，文献中已经开发了几种基于人工智能的技术来识别人体活动。然而，这些技术无法充分学习数据模式的时间和空间特征。此外，这些技术无法完全理解不同时期复杂的活动模式，这强调了需要增强架构，通过单独学习数据中的时空依赖性来进一步提高准确性。因此，在这项工作中，我们开发了一种具有注意力增强功能的双流网络（PAR-Net），用于进行身体活动识别，能够同时提取空间和时间特征。PAR-Net 集成了卷积神经网络（CNNs）和回声状态网络（ESNs），然后采用自注意力机制进行最优特征选择。双流特征提取机制使 PAR-Net 能够从实际数据中学习时空依赖性。此外，引入自注意力机制通过对重要特征进行有针对性的关注，从而增强对细微活动模式的识别，做出了实质性的贡献。PAR-Net 在两个基准身体活动识别数据集上进行了评估，通过与基线相比，性能得到了显著提高。此外，还进行了彻底的消融研究，以确定最适合人体活动识别的最佳模型。
