Bioengineering College of Chongqing University, Chongqing, China; School of Transportation and Logistics, East China Jiaotong University, Nanchang, China.
Department of Rehabilitation Medicine, Jiangxi Provincial Peoples Hospital, Nanchang, China.
Comput Biol Med. 2021 Apr;131:104282. doi: 10.1016/j.compbiomed.2021.104282. Epub 2021 Feb 17.
Finger mobility plays a crucial role in everyday living and is a leading indicator during hand rehabilitation and assistance tasks. Depth-based hand pose estimation is a potentially low-cost solution for the clinical and home-based measurement of symptoms of limited human finger motion.
The purpose of this study was to achieve the contactless measurement of finger motion based on depth-based hand pose estimation using Azure Kinect depth cameras and transfer learning, and to evaluate the accuracy in comparison with a three-dimensional motion analysis (3DMA) system.
Thirty participants performed a series of tasks during which their hand motions were measured concurrently using the Azure Kinect and 3DMA systems. We propose a simple and effective approach to achieving real-time hand pose estimations from single depth images using ensemble convolutional neural networks trained by a transfer learning strategy. Algorithms to calculate the finger joint motion angles are presented by tracking the locations of the 24 hand joints. To demonstrate their potential, the Azure-Kinect-based 3D finger motion measurement system and algorithms are experimentally verified through comparison with a camera-based 3DMA system, which is the gold standard.
Our results revealed that the Azure-Kinect-based hand pose estimation system produced highly correlated measurements of hand joint coordinates. Our method achieved excellent performance in terms of the fraction of good frames ( >80%) when the error thresholds were larger than approximately 2 cm, and the range of mean error distance was 0.23--1.05 cm. For joint angles, the Azure Kinect and 3DMA systems had comparable inter-trial reliability (ICC ranging from 0.89 to 0.97) and excellent concurrent validity, with Pearsons r-values >0.90 for most measurements (range: 0.88--0.97). The 95% BlandAltman limits of agreement were narrow enough for the Azure Kinect to be considered a valid tool for the measurement of all reported joint angles of the index finger and thumb in pinching. Moreover, our method runs in real time at over 45 fps.
The results of this study suggest that the proposed method has the capacity to measure the performance of fine motor skills.
手指的灵活性在日常生活中起着至关重要的作用，是手部康复和辅助任务中重要的指标。基于深度的手位估计是一种潜在的低成本解决方案，可以在临床和家庭环境中测量手指运动受限的症状。
本研究旨在使用 Azure Kinect 深度摄像机和迁移学习实现基于深度的手位估计的非接触式手指运动测量，并与三维运动分析（3DMA）系统进行比较，评估其准确性。
30 名参与者在进行一系列任务时，同时使用 Azure Kinect 和 3DMA 系统测量手部运动。我们提出了一种简单而有效的方法，使用基于迁移学习策略训练的集成卷积神经网络，从单个深度图像实时估计手位。通过跟踪 24 个手部关节的位置，提出了计算手指关节运动角度的算法。为了展示其潜力，通过与基于相机的 3DMA 系统（黄金标准）进行比较，实验验证了基于 Azure-Kinect 的 3D 手指运动测量系统和算法。
我们的结果表明，基于 Azure Kinect 的手位估计系统对手部关节坐标的测量具有高度相关性。当误差阈值大于约 2cm 时，我们的方法在良好帧数（>80%）的比例方面表现出色，平均误差距离范围为 0.23--1.05cm。对于关节角度，Azure Kinect 和 3DMA 系统具有相似的组内相关性（ICC 范围为 0.89 到 0.97）和极好的同时有效性，大多数测量的 Pearson r 值>0.90（范围：0.88--0.97）。Azure Kinect 的 95% BlandAltman 一致性界限足够窄，可以认为其是一种有效的工具，可用于测量捏合时食指和拇指的所有报告关节角度。此外，我们的方法实时运行速度超过 45fps。
本研究结果表明，该方法具有测量精细运动技能表现的能力。
