Center for Orthopaedic Biomechanics, University of Denver, Denver, CO 80208, USA.
Computer Vision and Social Robotics Laboratory, University of Denver, Denver, CO 80208, USA.
Sensors (Basel). 2023 Jun 21;23(13):5778. doi: 10.3390/s23135778.
Through wearable sensors and deep learning techniques, biomechanical analysis can reach beyond the lab for clinical and sporting applications. Transformers, a class of recent deep learning models, have become widely used in state-of-the-art artificial intelligence research due to their superior performance in various natural language processing and computer vision tasks. The performance of transformer models has not yet been investigated in biomechanics applications. In this study, we introduce a Biomechanical Multi-activity Transformer-based model, BioMAT, for the estimation of joint kinematics from streaming signals of multiple inertia measurement units (IMUs) using a publicly available dataset. This dataset includes IMU signals and the corresponding sagittal plane kinematics of the hip, knee, and ankle joints during multiple activities of daily living. We evaluated the model's performance and generalizability and compared it against a convolutional neural network long short-term model, a bidirectional long short-term model, and multi-linear regression across different ambulation tasks including level ground walking (LW), ramp ascent (RA), ramp descent (RD), stair ascent (SA), and stair descent (SD). To investigate the effect of different activity datasets on prediction accuracy, we compared the performance of a universal model trained on all activities against task-specific models trained on individual tasks. When the models were tested on three unseen subjects' data, BioMAT outperformed the benchmark models with an average root mean square error (RMSE) of 5.5 ± 0.5°, and normalized RMSE of 6.8 ± 0.3° across all three joints and all activities. A unified BioMAT model demonstrated superior performance compared to individual task-specific models across four of five activities. The RMSE values from the universal model for LW, RA, RD, SA, and SD activities were 5.0 ± 1.5°, 6.2 ± 1.1°, 5.8 ± 1.1°, 5.3 ± 1.6°, and 5.2 ± 0.7° while these values for task-specific models were, 5.3 ± 2.1°, 6.7 ± 2.0°, 6.9 ± 2.2°, 4.9 ± 1.4°, and 5.6 ± 1.3°, respectively. Overall, BioMAT accurately estimated joint kinematics relative to previous machine learning algorithms across different activities directly from the sequence of IMUs signals instead of time-normalized gait cycle data.
通过可穿戴传感器和深度学习技术，生物力学分析可以超越实验室，应用于临床和运动领域。由于在各种自然语言处理和计算机视觉任务中表现出色，Transformer 是一类最新的深度学习模型，已在最先进的人工智能研究中得到广泛应用。在生物力学应用中，Transformer 模型的性能尚未得到研究。在这项研究中，我们引入了一种基于生物力学多活动 Transformer 的模型（BioMAT），用于使用公开可用的数据集从多个惯性测量单元（IMU）的流信号中估计关节运动学。该数据集包括 IMU 信号以及在日常生活中进行多项活动时髋关节、膝关节和踝关节的矢状面运动学。我们评估了模型的性能和泛化能力，并将其与卷积神经网络长短期记忆模型、双向长短期记忆模型和多线性回归进行了比较，这些模型涵盖了不同的步行任务，包括平地行走（LW）、斜坡上升（RA）、斜坡下降（RD）、楼梯上升（SA）和楼梯下降（SD）。为了研究不同活动数据集对预测精度的影响，我们比较了在所有活动中训练的通用模型与在单个任务中训练的特定任务模型的性能。当模型在三个未见主体的数据上进行测试时，BioMAT 的表现优于基准模型，平均均方根误差（RMSE）为 5.5 ± 0.5°，所有三个关节和所有活动的归一化 RMSE 为 6.8 ± 0.3°。与五个活动中的四个活动相比，统一的 BioMAT 模型表现优于特定任务模型。通用模型在 LW、RA、RD、SA 和 SD 活动中的 RMSE 值分别为 5.0 ± 1.5°、6.2 ± 1.1°、5.8 ± 1.1°、5.3 ± 1.6°和 5.2 ± 0.7°，而特定任务模型的 RMSE 值分别为 5.3 ± 2.1°、6.7 ± 2.0°、6.9 ± 2.2°、4.9 ± 1.4°和 5.6 ± 1.3°。总的来说，BioMAT 直接从 IMU 信号的序列而不是时间归一化的步态周期数据中，相对于以前的机器学习算法，在不同的活动中准确地估计了关节运动学。
