Chen Lujing, Liu Rui, Yang Xin, Zhou Dongsheng, Zhang Qiang, Wei Xiaopeng
National and Local Joint Engineering Laboratory of Computer Aided Design, School of Software Engineering, Dalian University, Dalian, 116622, China.
School of Computer Science and Technology, Dalian University of Technology, Dalian, 116024, China.
Vis Comput Ind Biomed Art. 2022 Jul 29;5(1):19. doi: 10.1186/s42492-022-00112-5.
In recent years, human motion prediction has become an active research topic in computer vision. However, owing to the complexity and stochastic nature of human motion, it remains a challenging problem. In previous works, human motion prediction has always been treated as a typical inter-sequence problem, and most works have aimed to capture the temporal dependence between successive frames. However, although these approaches focused on the effects of the temporal dimension, they rarely considered the correlation between different joints in space. Thus, the spatio-temporal coupling of human joints is considered, to propose a novel spatio-temporal network based on a transformer and a gragh convolutional network (GCN) (STTG-Net). The temporal transformer is used to capture the global temporal dependencies, and the spatial GCN module is used to establish local spatial correlations between the joints for each frame. To overcome the problems of error accumulation and discontinuity in the motion prediction, a revision method based on fusion strategy is also proposed, in which the current prediction frame is fused with the previous frame. The experimental results show that the proposed prediction method has less prediction error and the prediction motion is smoother than previous prediction methods. The effectiveness of the proposed method is also demonstrated comparing it with the state-of-the-art method on the Human3.6 M dataset.
近年来，人体运动预测已成为计算机视觉领域一个活跃的研究课题。然而，由于人体运动的复杂性和随机性，它仍然是一个具有挑战性的问题。在以往的工作中，人体运动预测一直被视为一个典型的序列间问题，大多数工作旨在捕捉连续帧之间的时间依赖性。然而，尽管这些方法关注时间维度的影响，但它们很少考虑不同关节在空间上的相关性。因此，考虑人体关节的时空耦合，提出了一种基于Transformer和图卷积网络（GCN）的新型时空网络（STTG-Net）。时间Transformer用于捕捉全局时间依赖性，空间GCN模块用于为每一帧建立关节之间的局部空间相关性。为了克服运动预测中的误差累积和不连续性问题，还提出了一种基于融合策略的修正方法，即将当前预测帧与前一帧进行融合。实验结果表明，所提出的预测方法具有较小的预测误差，且预测运动比以往的预测方法更平滑。通过在Human3.6 M数据集上与现有最先进方法进行比较，也证明了所提方法的有效性。
