Wang Li, Wang Changyuan, Zhang Yu, Gao Lina
School of Optoelectronic Engineering, Xi'an Technological University, Xi'an 710000, China.
School of Computer Science, Xi'an Technological University, Xi'an 710000, China.
Math Biosci Eng. 2023 Jun 21;20(8):13974-13988. doi: 10.3934/mbe.2023622.
Improving the efficiency of human-computer interaction is one of the critical goals of intelligent aircraft cockpit research. The gaze interaction control method can vastly reduce the manual operation of operators and improve the intellectual level of human-computer interaction. Eye-tracking is the basis of sight interaction, so the performance of eye-tracking will directly affect the outcome of gaze interaction. This paper presents an eye-tracking method suitable for human-computer interaction in an aircraft cockpit, which can now estimate the gaze position of operators on multiple screens based on face images. We use a multi-camera system to capture facial images, so that operators are not limited by the angle of head rotation. To improve the accuracy of gaze estimation, we have constructed a hybrid network. One branch uses the transformer framework to extract the global features of the face images; the other branch uses a convolutional neural network structure to extract the local features of the face images. Finally, the extracted features of the two branches are fused for eye-tracking. The experimental results show that the proposed method not only solves the problem of limited head movement for operators but also improves the accuracy of gaze estimation. In addition, our method has a capture rate of more than 80% for targets of different sizes, which is better than the other compared models.
提高人机交互效率是智能飞机驾驶舱研究的关键目标之一。注视交互控制方法可以大幅减少操作人员的手动操作，提高人机交互的智能水平。眼动追踪是视觉交互的基础，因此眼动追踪的性能将直接影响注视交互的结果。本文提出了一种适用于飞机驾驶舱人机交互的眼动追踪方法，该方法目前可以基于面部图像估计操作人员在多个屏幕上的注视位置。我们使用多摄像头系统捕捉面部图像，使操作人员不受头部旋转角度的限制。为了提高注视估计的准确性，我们构建了一个混合网络。一个分支使用Transformer框架提取面部图像的全局特征；另一个分支使用卷积神经网络结构提取面部图像的局部特征。最后，将两个分支提取的特征融合用于眼动追踪。实验结果表明，该方法不仅解决了操作人员头部运动受限的问题，还提高了注视估计的准确性。此外，我们的方法对不同大小目标的捕获率超过80%，优于其他对比模型。
