Feng Huanghao, Mahoor Mohammad H, Dino Francesca
Changshu Institute of Technology, Suzhou, China.
Computer Vision and Social Robotics Labarotory, Department of Electrical and Computer Engineering, University of Denver, Denver, CO, United States.
Front Robot AI. 2022 May 23;9:855819. doi: 10.3389/frobt.2022.855819. eCollection 2022.
Children with Autism Spectrum Disorder (ASD) experience deficits in verbal and nonverbal communication skills including motor control, turn-taking, and emotion recognition. Innovative technology, such as socially assistive robots, has shown to be a viable method for Autism therapy. This paper presents a novel robot-based music-therapy platform for modeling and improving the social responses and behaviors of children with ASD. Our autonomous social interactive system consists of three modules. Module one provides an autonomous initiative positioning system for the robot, NAO, to properly localize and play the instrument (Xylophone) using the robot's arms. Module two allows NAO to play customized songs composed by individuals. Module three provides a real-life music therapy experience to the users. We adopted Short-time Fourier Transform and Levenshtein distance to fulfill the design requirements: 1) "music detection" and 2) "smart scoring and feedback", which allows NAO to understand music and provide additional practice and oral feedback to the users as applicable. We designed and implemented six Human-Robot-Interaction (HRI) sessions including four intervention sessions. Nine children with ASD and seven Typically Developing participated in a total of fifty HRI experimental sessions. Using our platform, we collected and analyzed data on social behavioral changes and emotion recognition using Electrodermal Activity (EDA) signals. The results of our experiments demonstrate most of the participants were able to complete motor control tasks with 70% accuracy. Six out of the nine ASD participants showed stable turn-taking behavior when playing music. The results of automated emotion classification using Support Vector Machines illustrates that emotional arousal in the ASD group can be detected and well recognized EDA bio-signals. In summary, the results of our data analyses, including emotion classification using EDA signals, indicate that the proposed robot-music based therapy platform is an attractive and promising assistive tool to facilitate the improvement of fine motor control and turn-taking skills in children with ASD.
患有自闭症谱系障碍（ASD）的儿童在言语和非言语沟通技能方面存在缺陷，包括运动控制、轮流互动和情感识别。创新技术，如社交辅助机器人，已被证明是一种可行的自闭症治疗方法。本文提出了一种基于机器人的新型音乐治疗平台，用于模拟和改善ASD儿童的社交反应和行为。我们的自主社交互动系统由三个模块组成。模块一为机器人NAO提供自主主动定位系统，以便使用机器人的手臂正确定位并演奏乐器（木琴）。模块二允许NAO演奏由个人创作的定制歌曲。模块三为用户提供真实的音乐治疗体验。我们采用短时傅里叶变换和莱文斯坦距离来满足设计要求：1）“音乐检测”和2）“智能评分与反馈”，这使得NAO能够理解音乐并在适用时为用户提供额外的练习和口头反馈。我们设计并实施了六次人机交互（HRI）会话，包括四次干预会话。九名ASD儿童和七名发育正常的儿童总共参加了五十次HRI实验会话。使用我们的平台，我们收集并分析了关于社交行为变化和使用皮肤电活动（EDA）信号进行情感识别的数据。我们的实验结果表明，大多数参与者能够以70%的准确率完成运动控制任务。九名ASD参与者中有六名在演奏音乐时表现出稳定的轮流互动行为。使用支持向量机进行自动情感分类的结果表明，ASD组中的情绪唤醒可以通过EDA生物信号被检测并很好地识别。总之，我们的数据分析结果，包括使用EDA信号进行情感分类，表明所提出的基于机器人音乐的治疗平台是一种有吸引力且有前景的辅助工具，有助于改善ASD儿童的精细运动控制和轮流互动技能。
