Uluer Pinar, Kose Hatice, Gumuslu Elif, Barkana Duygun Erol
Department of Computer Engineering, Galatasaray University, Istanbul, Turkey.
Department of AI and Data Engineering, Istanbul Technical University, Istanbul, Turkey.
Int J Soc Robot. 2023;15(4):643-660. doi: 10.1007/s12369-021-00830-5. Epub 2021 Nov 16.
This study presents an assistive robotic system enhanced with emotion recognition capabilities for children with hearing disabilities. The system is designed and developed for the audiometry tests and rehabilitation of children in a clinical setting and includes a social humanoid robot (Pepper), an interactive interface, gamified audiometry tests, sensory setup and a machine/deep learning based emotion recognition module. Three scenarios involving conventional setup, tablet setup and setup with the robot+tablet are evaluated with 16 children having cochlear implant or hearing aid. Several machine learning techniques and deep learning models are used for the classification of the three test setups and for the classification of the emotions (pleasant, neutral, unpleasant) of children using the recorded physiological signals by E4 wristband. The results show that the collected signals during the tests can be separated successfully and the positive and negative emotions of children can be better distinguished when they interact with the robot than in the other two setups. In addition, the children's objective and subjective evaluations as well as their impressions about the robot and its emotional behaviors are analyzed and discussed extensively.
本研究提出了一种为听障儿童增强了情感识别能力的辅助机器人系统。该系统是为临床环境中儿童的听力测试和康复而设计和开发的，包括一个社交人形机器人（Pepper）、一个交互式界面、游戏化听力测试、传感装置以及一个基于机器学习/深度学习的情感识别模块。对16名佩戴人工耳蜗或助听器的儿童进行了三种场景的评估，分别是传统设置、平板电脑设置以及机器人+平板电脑设置。使用了几种机器学习技术和深度学习模型对这三种测试设置进行分类，并使用E4腕带记录的生理信号对儿童的情绪（愉悦、中性、不愉快）进行分类。结果表明，测试过程中收集到的信号能够成功分离，并且与其他两种设置相比，儿童在与机器人互动时，其积极和消极情绪能够得到更好的区分。此外，还对儿童的客观和主观评价以及他们对机器人及其情感行为的印象进行了广泛的分析和讨论。
