Takemoto Ayumi, Aispuriete Inese, Niedra Laima, Dreimane Lana Franceska
Faculty of Computing, University of Latvia, Riga, Latvia.
Bioinformatics Laboratory, Riga Stradins University, Riga, Latvia.
Front Digit Health. 2023 Mar 10;5:1080023. doi: 10.3389/fdgth.2023.1080023. eCollection 2023.
Depression has a major effect on the quality of life. Thus, identifying an effective way to detect depression is important in the field of human-machine interaction. To examine whether a combination of a virtual avatar communication system and facial expression monitoring potentially classifies people as being with or without depression, this study consists of three research aims; 1) to understand the effect of different types of interviewers such as human and virtual avatars, on people with depression symptoms, 2) to clarify the effect of neutral conversation topics on facial expressions and emotions in people with depression symptoms, and 3) to compare verbal and non-verbal information between people with or without depression. In this study, twenty-seven participants-fifteen in the control group and twelve in the depression symptoms group-were recruited. They were asked to talk to a virtual avatar and human interviewers on both neutral and negative conversation topics and to score PANAS; meanwhile, facial expressions were recorded by a web camera. Facial expressions were analyzed by both manual and automatic analyses. In the manual analysis, three annotators counted gaze directions and reacting behaviors. On the other hand, automatic facial expression detection was conducted using OpenFace. The results of PANAS suggested that there was no significance between different interviewers' types. Furthermore, in the control group, the frequency of look-downward was larger in negative conversation topics than in neutral conversation topics. The intensity of Dimpler was larger in the control group than in the depression symptoms group. Moreover, the intensity of Chin Raiser was larger in neutral conversation topics than in negative conversation topics in the depression symptoms group. However, in the control groups, there was no significance in the types of conversation topics. In conclusion, 1) there was no significance between human and virtual avatar interviewers in emotions, facial expressions, and eye gaze patterns, 2) neutral conversation topics induced less negative emotion in both the control and depression symptoms group, and 3) different facial expressions' patterns between people with, or without depression, were observed in the virtual avatar communication system.
抑郁症对生活质量有重大影响。因此，在人机交互领域中，找到一种有效的抑郁症检测方法至关重要。为了研究虚拟化身通信系统与面部表情监测相结合是否有可能将人们分类为患有或未患有抑郁症，本研究包含三个研究目标：1）了解不同类型的访谈者（如人类和虚拟化身）对有抑郁症状的人的影响；2）阐明中性对话主题对有抑郁症状的人的面部表情和情绪的影响；3）比较有或没有抑郁症的人之间的言语和非言语信息。在本研究中，招募了27名参与者——15名在对照组，12名在抑郁症状组。他们被要求就中性和负面对话主题与虚拟化身和人类访谈者交谈，并对积极和消极情绪量表（PANAS）进行评分；与此同时，用网络摄像头记录面部表情。面部表情通过人工分析和自动分析两种方式进行分析。在人工分析中，三名注释者计算注视方向和反应行为。另一方面，使用OpenFace进行自动面部表情检测。积极和消极情绪量表的结果表明，不同访谈者类型之间没有显著差异。此外，在对照组中，负面对话主题中向下看的频率高于中性对话主题。对照组中酒窝肌的强度大于抑郁症状组。此外，在抑郁症状组中，中性对话主题中抬下巴肌的强度大于负面对话主题。然而，在对照组中，对话主题类型没有显著差异。总之，1）在情绪、面部表情和目光注视模式方面，人类和虚拟化身访谈者之间没有显著差异；2）中性对话主题在对照组和抑郁症状组中引发的负面情绪较少；3）在虚拟化身通信系统中，观察到了有或没有抑郁症的人之间不同的面部表情模式。
