IEEE Trans Neural Syst Rehabil Eng. 2020 Mar;28(3):748-755. doi: 10.1109/TNSRE.2020.2968912. Epub 2020 Jan 23.
Individuals with spinal cord injury (SCI) report upper limb function as their top recovery priority. To accurately represent the true impact of new interventions on patient function, evaluation should occur in a natural setting. Wearable cameras can be used to monitor hand function at home, using computer vision to automatically analyze the resulting egocentric videos. A key step in this process, hand detection, is difficult to accomplish robustly and reliably, hindering the deployment of a complete monitoring system in the home and community. We propose an accurate and efficient hand detection method that uses a simple combination of existing detection and tracking algorithms, evaluated on a new hand detection dataset, consisting of 167,622 frames of egocentric videos collected from 17 individuals with SCI in a home simulation laboratory. The F1-scores for the best detector and tracker alone (SSD and Median Flow) were 0.90±0.07 and 0.42±0.18, respectively. The best combination method, in which a detector was used to initialize and reset a tracker, resulted in an F1-score of 0.87±0.07 while being two times faster than the fastest detector. The method proposed here, in combination with wearable cameras, will help clinicians directly measure hand function in a patient's daily life at home.
脊髓损伤 (SCI) 患者报告上肢功能是他们最优先的恢复目标。为了准确地反映新干预措施对患者功能的真实影响，评估应该在自然环境中进行。可穿戴相机可用于在家中监测手部功能，使用计算机视觉自动分析产生的自我中心视频。在这个过程中，手检测是一个关键步骤，但很难实现稳健和可靠，这阻碍了完整的监测系统在家中和社区中的部署。我们提出了一种准确高效的手检测方法，该方法使用简单的现有检测和跟踪算法组合，在一个新的手检测数据集上进行评估，该数据集由 17 名 SCI 患者在家庭模拟实验室中采集的 167622 帧自我中心视频组成。最佳检测器和跟踪器（SSD 和 Median Flow）的 F1 分数分别为 0.90±0.07 和 0.42±0.18。最佳组合方法是使用检测器初始化和重置跟踪器，F1 分数为 0.87±0.07，同时比最快的检测器快两倍。这里提出的方法与可穿戴相机相结合，将帮助临床医生直接测量患者在家中的日常生活中的手部功能。
