Javed Hifza, Park Chung Hyuk
Department of Biomedical Engineering, School of Engineering and Applied Science, George Washington University, Washington, DC, United States.
Front Robot AI. 2022 Jun 20;9:880691. doi: 10.3389/frobt.2022.880691. eCollection 2022.
This work describes the design of real-time dance-based interaction with a humanoid robot, where the robot seeks to promote physical activity in children by taking on multiple roles as a dance partner. It acts as a leader by initiating dances but can also act as a follower by mimicking a child's dance movements. Dances in the leader role are produced by a sequence-to-sequence (S2S) Long Short-Term Memory (LSTM) network trained on children's music videos taken from YouTube. On the other hand, a music orchestration platform is implemented to generate background music in the follower mode as the robot mimics the child's poses. In doing so, we also incorporated the largely unexplored paradigm of learning-by-teaching by including multiple robot roles that allow the child to both learn from and teach to the robot. Our work is among the first to implement a largely autonomous, real-time full-body dance interaction with a bipedal humanoid robot that also explores the impact of the robot roles on child engagement. Importantly, we also incorporated in our design formal constructs taken from autism therapy, such as the least-to-most prompting hierarchy, reinforcements for positive behaviors, and a time delay to make behavioral observations. We implemented a multimodal child engagement model that encompasses both affective engagement (displayed through eye gaze focus and facial expressions) as well as task engagement (determined by the level of physical activity) to determine child engagement states. We then conducted a virtual exploratory user study to evaluate the impact of mixed robot roles on user engagement and found no statistically significant difference in the children's engagement in single-role and multiple-role interactions. While the children were observed to respond positively to both robot behaviors, they preferred the music-driven leader role over the movement-driven follower role, a result that can partly be attributed to the virtual nature of the study. Our findings support the utility of such a platform in practicing physical activity but indicate that further research is necessary to fully explore the impact of each robot role.
这项工作描述了与人形机器人进行基于舞蹈的实时交互设计，其中机器人试图通过扮演多种舞蹈伙伴角色来促进儿童的身体活动。它通过发起舞蹈来扮演领导者角色，但也可以通过模仿儿童的舞蹈动作来扮演跟随者角色。领导者角色的舞蹈由一个序列到序列（S2S）长短期记忆（LSTM）网络生成，该网络基于从YouTube上获取的儿童音乐视频进行训练。另一方面，实现了一个音乐编排平台，以便在跟随者模式下生成背景音乐，因为机器人模仿儿童的姿势。在此过程中，我们还纳入了一种在很大程度上未被探索的通过教学学习的范式，即包括多个机器人角色，使儿童既能向机器人学习，也能教机器人。我们的工作是首批实现与双足人形机器人进行基本自主、实时全身舞蹈交互的工作之一，同时也探索了机器人角色对儿童参与度的影响。重要的是，我们还在设计中纳入了来自自闭症治疗的形式结构，如从最少到最多的提示层次结构、对积极行为的强化以及进行行为观察的时间延迟。我们实现了一个多模态儿童参与模型，该模型既包括情感参与（通过目光聚焦和面部表情表现）以及任务参与（由身体活动水平决定），以确定儿童的参与状态。然后，我们进行了一项虚拟探索性用户研究，以评估混合机器人角色对用户参与度的影响，结果发现儿童在单角色和多角色交互中的参与度没有统计学上的显著差异。虽然观察到儿童对机器人的两种行为都有积极反应，但他们更喜欢音乐驱动的领导者角色，而不是动作驱动的跟随者角色，这一结果部分可归因于研究的虚拟性质。我们的研究结果支持了这样一个平台在身体活动实践中的效用，但表明需要进一步研究以充分探索每个机器人角色的影响。
