Huang Guan, Tran Son N, Bai Quan, Alty Jane
College of Sciences and Engineering, University of Tasmania, Sandy Bay, TAS 7005 Australia.
Wicking Dementia Research and Education Centre, University of Tasmania, Hobart, TAS 7000 Australia.
Neural Comput Appl. 2023;35(11):8143-8156. doi: 10.1007/s00521-022-08090-8. Epub 2022 Dec 10.
There is an urgent need, accelerated by the COVID-19 pandemic, for methods that allow clinicians and neuroscientists to remotely evaluate hand movements. This would help detect and monitor degenerative brain disorders that are particularly prevalent in older adults. With the wide accessibility of computer cameras, a vision-based real-time hand gesture detection method would facilitate online assessments in home and clinical settings. However, motion blur is one of the most challenging problems in the fast-moving hands data collection. The objective of this study was to develop a computer vision-based method that accurately detects older adults' hand gestures using video data collected in real-life settings. We invited adults over 50 years old to complete validated hand movement tests (fast finger tapping and hand opening-closing) at home or in clinic. Data were collected without researcher supervision via a website programme using standard laptop and desktop cameras. We processed and labelled images, split the data into training, validation and testing, respectively, and then analysed how well different network structures detected hand gestures. We recruited 1,900 adults (age range 50-90 years) as part of the TAS Test project and developed UTAS7k-a new dataset of 7071 hand gesture images, split 4:1 into clear: motion-blurred images. Our new network, RGRNet, achieved 0.782 mean average precision (mAP) on clear images, outperforming the state-of-the-art network structure (YOLOV5-P6, mAP 0.776), and mAP 0.771 on blurred images. A new robust real-time automated network that detects static gestures from a single camera, RGRNet, and a new database comprising the largest range of individual hands, UTAS7k, both show strong potential for medical and research applications.
The online version contains supplementary material available at 10.1007/s00521-022-08090-8.
受新冠疫情加速影响，迫切需要能让临床医生和神经科学家远程评估手部运动的方法。这将有助于检测和监测在老年人中特别普遍的退行性脑部疾病。鉴于电脑摄像头广泛普及，基于视觉的实时手势检测方法将便于在家庭和临床环境中进行在线评估。然而，运动模糊是快速移动的手部数据采集中最具挑战性的问题之一。本研究的目的是开发一种基于计算机视觉的方法，该方法能使用在现实生活环境中收集的视频数据准确检测老年人的手势。我们邀请50岁以上的成年人在家中或诊所完成经过验证的手部运动测试（快速手指敲击和手部开合）。数据通过网站程序，使用标准笔记本电脑和台式机摄像头在无研究人员监督的情况下收集。我们对图像进行处理和标注，分别将数据拆分为训练集、验证集和测试集，然后分析不同网络结构对手势的检测效果。作为TAS测试项目的一部分，我们招募了1900名成年人（年龄范围50 - 90岁），并开发了UTAS7k——一个包含7071张手势图像的新数据集，按4:1分为清晰图像与运动模糊图像。我们的新网络RGRNet在清晰图像上的平均精度均值（mAP）达到0.782，优于最先进的网络结构（YOLOV5 - P6，mAP为0.776），在模糊图像上的mAP为0.771。一种能从单个摄像头检测静态手势的新型鲁棒实时自动化网络RGRNet，以及一个包含最大范围个体手部的新数据库UTAS7k，在医学和研究应用中均显示出强大潜力。
在线版本包含可在10.1007/s00521 - 022 - 08090 - 8获取的补充材料。
