Guilford Glazer Faculty of Business and Management, Ben-Gurion University of the Negev, Be'er-Sheva, Israel.
Department of Industrial Engineering and Management, Ben-Gurion University of the Negev, Be'er-Sheva, Israel.
PLoS One. 2023 Sep 13;18(9):e0290564. doi: 10.1371/journal.pone.0290564. eCollection 2023.
Emotion recognition is key to interpersonal communication and to human-machine interaction. Body expression may contribute to emotion recognition, but most past studies focused on a few motions, limiting accurate recognition. Moreover, emotions in most previous research were acted out, resulting in non-natural motion, which is unapplicable in reality. We present an approach for emotion recognition based on body motion in naturalistic settings, examining authentic emotions, natural movement, and a broad collection of motion parameters. A lab experiment using 24 participants manipulated participants' emotions using pretested movies into five conditions: happiness, relaxation, fear, sadness, and emotionally-neutral. Emotion was manipulated within subjects, with fillers in between and a counterbalanced order. A motion capture system measured posture and motion during standing and walking; a force plate measured center of pressure location. Traditional statistics revealed nonsignificant effects of emotions on most motion parameters; only 7 of 229 parameters demonstrate significant effects. Most significant effects are in parameters representing postural control during standing, which is consistent with past studies. Yet, the few significant effects suggest that it is impossible to recognize emotions based on a single motion parameter. We therefore developed machine learning models to classify emotions using a collection of parameters, and examined six models: k-nearest neighbors, decision tree, logistic regression, and the support vector machine with radial base function and linear and polynomial functions. The decision tree using 25 parameters provided the highest average accuracy (45.8%), more than twice the random guess for five conditions, which advances past studies demonstrating comparable accuracies, due to our naturalistic setting. This research suggests that machine learning models are valuable for emotion recognition in reality and lays the foundation for further progress in emotion recognition models, informing the development of recognition devices (e.g., depth camera), to be used in home-setting human-machine interactions.
情感识别是人际交流和人机交互的关键。身体表达可能有助于情感识别，但过去的大多数研究都集中在少数动作上，限制了准确识别。此外，大多数先前研究中的情绪都是通过表演产生的，导致运动不自然，在现实中无法应用。我们提出了一种基于自然环境中身体运动的情感识别方法，研究真实的情感、自然的运动和广泛的运动参数。使用 24 名参与者的实验室实验使用经过预先测试的电影在五种条件下操纵参与者的情绪：快乐、放松、恐惧、悲伤和中性。情绪是在被试内进行操纵的，填充剂在中间，顺序是平衡的。运动捕捉系统测量站立和行走时的姿势和运动；压力板测量中心压力位置。传统统计学显示，情绪对大多数运动参数没有显著影响；只有 229 个参数中的 7 个显示出显著影响。大多数显著影响存在于代表站立时姿势控制的参数中，这与过去的研究一致。然而，少数显著影响表明，不可能仅基于单个运动参数识别情绪。因此，我们开发了机器学习模型，使用一系列参数对情绪进行分类，并检查了六个模型：k-最近邻、决策树、逻辑回归以及具有径向基函数和线性和多项式函数的支持向量机。使用 25 个参数的决策树提供了最高的平均准确性（45.8%），比五个条件的随机猜测高出两倍，这优于过去的研究，因为我们的自然环境。这项研究表明，机器学习模型在现实中的情感识别中很有价值，并为情感识别模型的进一步发展奠定了基础，为识别设备（例如深度相机）的开发提供了信息，以用于家庭环境中的人机交互。
