Fiorini Laura, D'Onofrio Grazia, Sorrentino Alessandra, Cornacchia Loizzo Federica Gabriella, Russo Sergio, Ciccone Filomena, Giuliani Francesco, Sancarlo Daniele, Cavallo Filippo
Department of Industrial Engineering, University of Florence, Firenze, Italy.
The BioRobotics Institute, Scuola Superiore Sant'Anna, Pontedera (Pisa), Italy.
JMIR Hum Factors. 2024 Jan 26;11:e45494. doi: 10.2196/45494.
Social robots are becoming increasingly important as companions in our daily lives. Consequently, humans expect to interact with them using the same mental models applied to human-human interactions, including the use of cospeech gestures. Research efforts have been devoted to understanding users' needs and developing robot's behavioral models that can perceive the user state and properly plan a reaction. Despite the efforts made, some challenges regarding the effect of robot embodiment and behavior in the perception of emotions remain open.
The aim of this study is dual. First, it aims to assess the role of the robot's cospeech gestures and embodiment in the user's perceived emotions in terms of valence (stimulus pleasantness), arousal (intensity of evoked emotion), and dominance (degree of control exerted by the stimulus). Second, it aims to evaluate the robot's accuracy in identifying positive, negative, and neutral emotions displayed by interacting humans using 3 supervised machine learning algorithms: support vector machine, random forest, and K-nearest neighbor.
Pepper robot was used to elicit the 3 emotions in humans using a set of 60 images retrieved from a standardized database. In particular, 2 experimental conditions for emotion elicitation were performed with Pepper robot: with a static behavior or with a robot that expresses coherent (COH) cospeech behavior. Furthermore, to evaluate the role of the robot embodiment, the third elicitation was performed by asking the participant to interact with a PC, where a graphical interface showed the same images. Each participant was requested to undergo only 1 of the 3 experimental conditions.
A total of 60 participants were recruited for this study, 20 for each experimental condition for a total of 3600 interactions. The results showed significant differences (P<.05) in valence, arousal, and dominance when stimulated with the Pepper robot behaving COH with respect to the PC condition, thus underlying the importance of the robot's nonverbal communication and embodiment. A higher valence score was obtained for the elicitation of the robot (COH and robot with static behavior) with respect to the PC. For emotion recognition, the K-nearest neighbor classifiers achieved the best accuracy results. In particular, the COH modality achieved the highest level of accuracy (0.97) when compared with the static behavior and PC elicitations (0.88 and 0.94, respectively).
The results suggest that the use of multimodal communication channels, such as cospeech and visual channels, as in the COH modality, may improve the recognition accuracy of the user's emotional state and can reinforce the perceived emotion. Future studies should investigate the effect of age, culture, and cognitive profile on the emotion perception and recognition going beyond the limitation of this work.
社交机器人在我们的日常生活中作为陪伴者正变得越来越重要。因此，人类期望使用与人际互动相同的心理模型与它们进行交互，包括使用伴随言语的手势。研究工作致力于理解用户需求并开发能够感知用户状态并正确规划反应的机器人行为模型。尽管已经做出了努力，但关于机器人的实体化和行为在情感感知方面的影响仍存在一些挑战。
本研究有双重目的。首先，旨在评估机器人伴随言语的手势和实体化在用户感知情绪的效价（刺激的愉悦度）、唤醒度（诱发情绪的强度）和支配度（刺激施加的控制程度）方面所起的作用。其次，旨在使用三种监督机器学习算法（支持向量机、随机森林和K近邻）评估机器人识别互动人类所展示的积极、消极和中性情绪的准确性。
使用Pepper机器人，通过从标准化数据库中检索的一组60张图像来诱发人类的三种情绪。具体而言，使用Pepper机器人进行了两种情绪诱发实验条件：静态行为或表现出连贯（COH）伴随言语行为的机器人。此外，为了评估机器人实体化的作用，通过要求参与者与一台电脑进行交互来进行第三种诱发实验，电脑的图形界面显示相同的图像。每位参与者仅被要求经历三种实验条件中的一种。
本研究共招募了60名参与者，每种实验条件20名，总共进行了3600次交互。结果显示，与电脑条件相比，当用表现出COH行为的Pepper机器人刺激时，在效价、唤醒度和支配度方面存在显著差异（P<0.05），从而突出了机器人非语言交流和实体化的重要性。与电脑相比，机器人（COH和静态行为机器人）诱发的效价值更高。对于情绪识别，K近邻分类器取得了最佳的准确性结果。特别是，与静态行为和电脑诱发实验（分别为0.88和0.94）相比，COH模式达到了最高的准确性水平（0.97）。
结果表明，如COH模式中那样使用多模态交流渠道，如伴随言语和视觉渠道，可能会提高用户情绪状态的识别准确性，并能增强感知到的情绪。未来的研究应超越本研究的局限性，调查年龄、文化和认知特征对情绪感知和识别的影响。
