Kong Youngsun, Posada-Quintero Hugo F, Daley Matthew S, Chon Ki H, Bolkhovsky Jeffrey
Department of Biomedical Engineering, University of Connecticut, 407 Engineering and Science Building, 67 N Eagleville, Storrs, CT 06269, USA.
Naval Submarine Medical Research Laboratory, Groton, CT, USA.
Atten Percept Psychophys. 2021 Jan;83(1):525-540. doi: 10.3758/s13414-020-02199-5. Epub 2020 Nov 17.
We have performed a direct comparison between facial features obtained from a webcam and vigilance-task performance during prolonged wakefulness. Prolonged wakefulness deteriorates working performance due to changes in cognition, emotion, and by delayed response. Facial features can be potentially collected everywhere using webcams located in the workplace. If this type of device can obtain relevant information to predict performance deterioration, this technology can potentially reduce serious accidents and fatality. We extracted 34 facial indices, including head movements, facial expressions, and perceived facial emotions from 20 participants undergoing the psychomotor vigilance task (PVT) over 25 hours. We studied the correlation between facial indices and the performance indices derived from PVT, and evaluated the feasibility of facial indices as detectors of diminished reaction time during the PVT. Furthermore, we tested the feasibility of classifying performance as normal or impaired using several machine learning algorithms with correlated facial indices. Twenty-one indices were found significantly correlated with PVT indices. Pitch, from the head movement indices, and four perceived facial emotions-anger, surprise, sadness, and disgust-exhibited significant correlations with indices of performance. The eye-related facial expression indices showed especially strong correlation and higher feasibility of facial indices as classifiers. Significantly correlated indices were shown to explain more variance than the other indices for most of the classifiers. The facial indices obtained from a webcam strongly correlate with working performance during 25 hours of prolonged wakefulness.
我们对通过网络摄像头获取的面部特征与长时间清醒期间的警觉任务表现进行了直接比较。长时间清醒会因认知、情绪变化以及反应延迟而导致工作表现下降。利用工作场所中的网络摄像头可以在任何地方潜在地收集面部特征。如果这种设备能够获取相关信息来预测表现下降，这项技术就有可能减少严重事故和死亡。我们从20名参与者在25小时内进行精神运动警觉任务（PVT）时提取了34个面部指标，包括头部动作、面部表情和感知到的面部情绪。我们研究了面部指标与PVT得出的表现指标之间的相关性，并评估了面部指标作为PVT期间反应时间缩短检测器的可行性。此外，我们使用几种与面部指标相关的机器学习算法测试了将表现分类为正常或受损的可行性。发现21个指标与PVT指标显著相关。头部动作指标中的俯仰，以及四种感知到的面部情绪——愤怒、惊讶、悲伤和厌恶——与表现指标呈现出显著相关性。与眼睛相关的面部表情指标显示出特别强的相关性，并且面部指标作为分类器的可行性更高。对于大多数分类器而言，显著相关的指标比其他指标解释了更多的方差。从网络摄像头获得的面部指标与长时间清醒25小时期间的工作表现密切相关。
