School of Mechanical Engineering and Hubei Modern Manufacturing Quality Engineering Key Laboratory, Hubei University of Technology, Wuhan 430068, China.
School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China.
Sensors (Basel). 2023 Nov 10;23(22):9101. doi: 10.3390/s23229101.
Global aging leads to a surge in neurological diseases. Quantitative gait analysis for the early detection of neurological diseases can effectively reduce the impact of the diseases. Recently, extensive research has focused on gait-abnormality-recognition algorithms using a single type of portable sensor. However, these studies are limited by the sensor's type and the task specificity, constraining the widespread application of quantitative gait recognition. In this study, we propose a multimodal gait-abnormality-recognition framework based on a Convolutional Neural Network-Bidirectional Long Short-Term Memory (CNN-BiLSTM) network. The as-established framework effectively addresses the challenges arising from smooth data interference and lengthy time series by employing an adaptive sliding window technique. Then, we convert the time series into time-frequency plots to capture the characteristic variations in different abnormality gaits and achieve a unified representation of the multiple data types. This makes our signal processing method adaptable to several types of sensors. Additionally, we use a pre-trained Deep Convolutional Neural Network (DCNN) for feature extraction, and the consequently established CNN-BiLSTM network can achieve high-accuracy recognition by fusing and classifying the multi-sensor input data. To validate the proposed method, we conducted diversified experiments to recognize the gait abnormalities caused by different neuropathic diseases, such as amyotrophic lateral sclerosis (ALS), Parkinson's disease (PD), and Huntington's disease (HD). In the PDgait dataset, the framework achieved an accuracy of 98.89% in the classification of Parkinson's disease severity, surpassing DCLSTM's 96.71%. Moreover, the recognition accuracy of ALS, PD, and HD on the PDgait dataset was 100%, 96.97%, and 95.43% respectively, surpassing the majority of previously reported methods. These experimental results strongly demonstrate the potential of the proposed multimodal framework for gait abnormality identification. Due to the advantages of the framework, such as its suitability for different types of sensors and fewer training parameters, it is more suitable for gait monitoring in daily life and the customization of medical rehabilitation schedules, which will help more patients alleviate the harm caused by their diseases.
全球老龄化导致神经退行性疾病的发病率呈指数级增长。使用单一类型的便携式传感器进行定量步态分析，可有效降低此类疾病的影响。最近，广泛的研究集中在使用单类传感器的步态异常识别算法上。然而，这些研究受到传感器类型和任务特异性的限制，限制了定量步态识别的广泛应用。在本研究中，我们提出了一种基于卷积神经网络-双向长短期记忆网络（CNN-BiLSTM）的多模态步态异常识别框架。所建立的框架通过自适应滑动窗口技术，有效解决了平滑数据干扰和长序列时间的挑战。然后，我们将时间序列转换为时频图，以捕获不同异常步态的特征变化，并实现多种数据类型的统一表示。这使得我们的信号处理方法适应于多种类型的传感器。此外，我们使用预训练的深度卷积神经网络（DCNN）进行特征提取，所建立的 CNN-BiLSTM 网络可以通过融合和分类多传感器输入数据实现高精度识别。为了验证所提出的方法，我们进行了多样化的实验来识别不同神经退行性疾病引起的步态异常，如肌萎缩侧索硬化症（ALS）、帕金森病（PD）和亨廷顿病（HD）。在 PDgait 数据集上，该框架在帕金森病严重程度的分类中达到了 98.89%的准确率，优于 DCLSTM 的 96.71%。此外，在 PDgait 数据集上，ALS、PD 和 HD 的识别准确率分别为 100%、96.97%和 95.43%，超过了大多数之前报道的方法。这些实验结果有力地证明了所提出的多模态框架用于步态异常识别的潜力。由于该框架具有适用于不同类型传感器和较少训练参数等优点，因此更适用于日常生活中的步态监测和医疗康复计划的定制，这将有助于更多的患者减轻疾病带来的伤害。
