Department of Convergence Medicine, University of Ulsan College of Medicine, Asan Medical Center, Seoul, South Korea.
Department of Convergence Medicine, University of Ulsan College of Medicine, Asan Medical Center, Seoul, South Korea; Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Korea.
Comput Methods Programs Biomed. 2020 Jul;190:105385. doi: 10.1016/j.cmpb.2020.105385. Epub 2020 Feb 6.
Hand gesture recognition systems in operating rooms (ORs) are crucial for browsing and controlling computer-aided devices, which have been developed to decrease the risk of contamination during surgical procedures.
We proposed the use of hand gesture recognition to enhance accuracies and recognition areas with the capsule network (CapsNet) of deep neural network and Leap Motionâ Our method includes the i) extraction and preprocessing of infrared (IR) images (60 frames per second) from Leap Motion™, ii) training of various types of networks, and iii) gesture recognition evaluation in the OR. We trained the images of training dataset (N=903) and tested images (N=100) using five types of surgical hand gestures including hovering, grab, click, one peak, and two peaks by 10 subjects with various types of augmentation methods including rotate (0, 90, 180), scale, translation, illumination, and resize.
CapsNet achieved a classification accuracy of 86.46% (around 10% improvement) compared with 73.67% for the baseline convolutional neural network (CNN) and 76.4% for VGG16.
In conclusion, the accuracy of hand gesture recognition with CapsNet was better than that of conventional CNNs, which could be used to navigate and manipulate various types of computer-aided devices and applications through contactless gesture interaction.
手术室中的手势识别系统对于浏览和控制计算机辅助设备至关重要，这些设备的开发旨在降低手术过程中的污染风险。
我们提出使用手势识别技术，通过深度神经网络的胶囊网络（CapsNet）和 Leap Motion 来提高准确性和识别区域。我们的方法包括：i）从 Leap Motion™ 提取和预处理红外（IR）图像（每秒 60 帧），ii）训练各种类型的网络，以及 iii）在手术室中进行手势识别评估。我们使用五种类型的手术手势（包括悬停、抓取、点击、单峰和双峰）对 903 张训练数据集图像和 100 张测试图像进行了训练，其中包括旋转（0、90、180）、缩放、平移、光照和调整大小等多种增强方法，由 10 位具有不同类型的受试者进行。
CapsNet 的分类准确率为 86.46%（比基线卷积神经网络（CNN）的 73.67%提高了约 10%），比 VGG16 高 76.4%。
总之，CapsNet 的手势识别准确性优于传统的 CNN，可以通过非接触式手势交互来导航和操作各种类型的计算机辅助设备和应用程序。
