Frame topology fusion-based hierarchical graph convolution for automatic assessment of physical rehabilitation exercises.

Stroke rehabilitation movements are significantly influenced by patient subjectivity, leading to challenges in capturing subtle differences and temporal characteristics of patient motions. Existing methods typically focus on adjacent joint movements, overlooking the intricate interdependencies among body joints. Moreover, they lack the capacity to assess motion quality based on diverse temporal characteristics. To address these challenges, we propose a Frame Topology Fusion Hierarchical Graph Convolution Network (FTF-HGCN). This method aims to provide a more precise assessment of rehabilitation movement quality by effectively modeling both spatial and temporal features. First, this method combines nearby and distant keypoints to construct a fused topology structure for obtaining the enhanced motion representation. This allows the network to focus on joints with larger motion amplitudes. Second, based on the fused topology structure, a learnable topological matrix is established for each action frame to capture subtle differences between patient movements. Finally, a hierarchical temporal convolution attention module is employed to integrate motion feature information across different time sequences. Subsequently, a fully connected layer is used to output the predicted quality score of rehabilitation movements. Extensive experiments were conducted on KIMORE and UI-PRMD datasets, achieving best performance on relevant evaluation metrics (MAD: 13.4[Formula: see text], RMSE: 39.8[Formula: see text], MAPE: 7.6[Formula: see text]). This shows that the proposed FTF-HGCN method is capable of delivering accurate evaluations and offering superior support for the home-based rehabilitation of stroke patients.