Department of Electrical Engineering and Computer Science, University of Kassel, 34127 Kassel, Germany.
Sensors (Basel). 2019 Mar 5;19(5):1121. doi: 10.3390/s19051121.
The aim of the study is to develop a real-time eyeblink detection algorithm that can detect eyeblinks during the closing phase for a virtual reality headset (VR headset) and accordingly classify the eye's current state (open or closed). The proposed method utilises analysis of a motion vector for detecting eyelid closure, and a Haar cascade classifier (HCC) for localising the eye in the captured frame. When the downward motion vector (DMV) is detected, a cross-correlation between the current region of interest (eye in the current frame) and a template image for an open eye is used for verifying eyelid closure. A finite state machine is used for decision making regarding eyeblink occurrence and tracking the eye state in a real-time video stream. The main contributions of this study are, first, the ability of the proposed algorithm to detect eyeblinks during the closing or the pause phases before the occurrence of the reopening phase of the eyeblink. Second, realising the proposed approach by implementing a valid real-time eyeblink detection sensor for a VR headset based on a real case scenario. The sensor is used in the ongoing study that we are conducting. The performance of the proposed method was 83.9% for accuracy, 91.8% for precision and 90.40% for the recall. The processing time for each frame took approximately 11 milliseconds. Additionally, we present a new dataset for non-frontal eye monitoring configuration for eyeblink tracking inside a VR headset. The data annotations are also included, such that the dataset can be used for method validation and performance evaluation in future studies.
本研究的目的是开发一种实时眨眼检测算法，该算法可以在虚拟现实耳机（VR 耳机）的关闭阶段检测眨眼，并相应地对眼睛的当前状态（打开或关闭）进行分类。该方法利用运动矢量分析来检测眼睑闭合，并使用 Haar 级联分类器（HCC）来定位捕获帧中的眼睛。当检测到向下运动矢量（DMV）时，使用当前感兴趣区域（当前帧中的眼睛）与开眼模板图像之间的互相关来验证眼睑闭合。使用有限状态机来做出关于眨眼发生的决策，并在实时视频流中跟踪眼睛状态。本研究的主要贡献有：第一，提出的算法能够在眨眼的关闭或暂停阶段检测眨眼，而无需等待眨眼的重新打开阶段。第二，通过实现一种基于实际案例场景的 VR 耳机有效实时眨眼检测传感器来实现所提出的方法。该传感器正在我们正在进行的研究中使用。所提出方法的性能为：准确性为 83.9%，精度为 91.8%，召回率为 90.40%。每个帧的处理时间约为 11 毫秒。此外，我们还提出了一个新的数据集，用于 VR 耳机内部的非正面眼部监测配置的眨眼跟踪。数据集还包括注释，以便在未来的研究中可以用于方法验证和性能评估。
