Khan Zohaib Ahmad, Xia Yuanqing, Aurangzeb Khursheed, Khaliq Fiza, Alam Mahmood, Khan Javed Ali, Anwar Muhammad Shahid
School of Automation, Beijing Institute of Technology, Beijing, China.
Department of Computer Engineering, College of Computer and Information Sciences, King Saud University, Riyadh, Saudi Arabia.
PeerJ Comput Sci. 2024 Mar 29;10:e1887. doi: 10.7717/peerj-cs.1887. eCollection 2024.
Emotion detection (ED) involves the identification and understanding of an individual's emotional state through various cues such as facial expressions, voice tones, physiological changes, and behavioral patterns. In this context, behavioral analysis is employed to observe actions and behaviors for emotional interpretation. This work specifically employs behavioral metrics like drawing and handwriting to determine a person's emotional state, recognizing these actions as physical functions integrating motor and cognitive processes. The study proposes an attention-based transformer model as an innovative approach to identify emotions from handwriting and drawing samples, thereby advancing the capabilities of ED into the domains of fine motor skills and artistic expression. The initial data obtained provides a set of points that correspond to the handwriting or drawing strokes. Each stroke point is subsequently delivered to the attention-based transformer model, which embeds it into a high-dimensional vector space. The model builds a prediction about the emotional state of the person who generated the sample by integrating the most important components and patterns in the input sequence using self-attentional processes. The proposed approach possesses a distinct advantage in its enhanced capacity to capture long-range correlations compared to conventional recurrent neural networks (RNN). This characteristic makes it particularly well-suited for the precise identification of emotions from samples of handwriting and drawings, signifying a notable advancement in the field of emotion detection. The proposed method produced cutting-edge outcomes of 92.64% on the benchmark dataset known as EMOTHAW (Emotion Recognition Handwriting and Drawing).
情绪检测（ED）涉及通过面部表情、语音语调、生理变化和行为模式等各种线索来识别和理解个体的情绪状态。在此背景下，行为分析被用于观察行为和动作以进行情绪解读。这项工作特别采用绘画和笔迹等行为指标来确定一个人的情绪状态，将这些行为视为整合运动和认知过程的身体功能。该研究提出了一种基于注意力的变压器模型，作为从笔迹和绘画样本中识别情绪的创新方法，从而将情绪检测的能力扩展到精细运动技能和艺术表达领域。获得的初始数据提供了一组与笔迹或绘画笔触相对应的点。随后，每个笔触点被输入到基于注意力的变压器模型中，该模型将其嵌入到高维向量空间中。该模型通过使用自注意力过程整合输入序列中最重要的成分和模式，对生成样本的人的情绪状态进行预测。与传统的循环神经网络（RNN）相比，所提出的方法在捕捉长距离相关性方面具有明显优势。这一特性使其特别适合从笔迹和绘画样本中精确识别情绪，标志着情绪检测领域的显著进步。所提出的方法在名为EMOTHAW（情绪识别 笔迹和绘画）的基准数据集上产生了92.64%的前沿成果。
