Department of Electrical and Computer Engineering, Inha University, Incheon 22212, Korea.
Department of Otorhinolaryngology, Inha University Hospital, Incheon 22332, Korea.
Sensors (Basel). 2022 May 23;22(10):3932. doi: 10.3390/s22103932.
Inertial-measurement-unit (IMU)-based human activity recognition (HAR) studies have improved their performance owing to the latest classification model. In this study, the conformer, which is a state-of-the-art (SOTA) model in the field of speech recognition, is introduced in HAR to improve the performance of the transformer-based HAR model. The transformer model has a multi-head self-attention structure that can extract temporal dependency well, similar to the recurrent neural network (RNN) series while having higher computational efficiency than the RNN series. However, recent HAR studies have shown good performance by combining an RNN-series and convolutional neural network (CNN) model. Therefore, the performance of the transformer-based HAR study can be improved by adding a CNN layer that extracts local features well. The model that improved these points is the conformer-based-model model. To evaluate the proposed model, WISDM, UCI-HAR, and PAMAP2 datasets were used. A synthetic minority oversampling technique was used for the data augmentation algorithm to improve the dataset. From the experiment, the conformer-based HAR model showed better performance than baseline models: the transformer-based-model and the 1D-CNN HAR models. Moreover, the performance of the proposed algorithm was superior to that of algorithms proposed in recent similar studies which do not use RNN-series.
基于惯性测量单元（IMU）的人体活动识别（HAR）研究由于最新的分类模型而提高了性能。在这项研究中，引入了在语音识别领域处于领先地位的转换器(conformer)，以提高基于转换器的 HAR 模型的性能。转换器模型具有多头自注意力结构，能够很好地提取时间依赖性，类似于递归神经网络（RNN）系列，而计算效率高于 RNN 系列。然而，最近的 HAR 研究表明，通过结合 RNN 系列和卷积神经网络（CNN）模型可以取得很好的性能。因此，通过添加能够很好地提取局部特征的 CNN 层，可以提高基于转换器的 HAR 研究的性能。改进这些点的模型是基于转换器的模型。为了评估所提出的模型，使用了 WISDM、UCI-HAR 和 PAMAP2 数据集。为了数据增强算法，使用了合成少数过采样技术来改进数据集。从实验中可以看出，基于转换器的 HAR 模型的性能优于基线模型：基于转换器的模型和 1D-CNN HAR 模型。此外，所提出算法的性能优于不使用 RNN 系列的最近类似研究中提出的算法。
