Department of Psychology, Sapienza University of Rome Rome, Italy ; IRCCS, Fondazione Santa Lucia Rome, Italy.
CNRS-AIST Joint Robotics Laboratory, UMI3218/CRT Tsukuba, Japan ; UM2-CNRS LIRMM UMR5506 Montpellier, France.
Front Neurorobot. 2014 Jun 17;8:20. doi: 10.3389/fnbot.2014.00020. eCollection 2014.
Advancement in brain computer interfaces (BCI) technology allows people to actively interact in the world through surrogates. Controlling real humanoid robots using BCI as intuitively as we control our body represents a challenge for current research in robotics and neuroscience. In order to successfully interact with the environment the brain integrates multiple sensory cues to form a coherent representation of the world. Cognitive neuroscience studies demonstrate that multisensory integration may imply a gain with respect to a single modality and ultimately improve the overall sensorimotor performance. For example, reactivity to simultaneous visual and auditory stimuli may be higher than to the sum of the same stimuli delivered in isolation or in temporal sequence. Yet, knowledge about whether audio-visual integration may improve the control of a surrogate is meager. To explore this issue, we provided human footstep sounds as audio feedback to BCI users while controlling a humanoid robot. Participants were asked to steer their robot surrogate and perform a pick-and-place task through BCI-SSVEPs. We found that audio-visual synchrony between footsteps sound and actual humanoid's walk reduces the time required for steering the robot. Thus, auditory feedback congruent with the humanoid actions may improve motor decisions of the BCI's user and help in the feeling of control over it. Our results shed light on the possibility to increase robot's control through the combination of multisensory feedback to a BCI user.
脑机接口 (BCI) 技术的进步使人们能够通过替身主动与世界互动。使用 BCI 像我们控制身体一样直观地控制真实的人形机器人，这代表了机器人技术和神经科学领域当前研究的一个挑战。为了成功地与环境互动，大脑整合了多种感官线索，形成了对世界的连贯表示。认知神经科学研究表明，多感觉整合可能意味着相对于单一感觉模式的增益，最终会提高整体感觉运动性能。例如，对同时的视觉和听觉刺激的反应可能高于单独或按时间顺序提供的相同刺激的总和。然而，关于音频视觉整合是否可以改善替身控制的知识却很少。为了探索这个问题，我们在控制人形机器人的同时，为 BCI 用户提供了脚步声作为音频反馈。参与者被要求通过 BCI-SSVEPs 来操纵他们的机器人替身并执行一个取放任务。我们发现，脚步声与实际人形机器人行走之间的视听同步会减少操纵机器人所需的时间。因此，与人形机器人动作一致的听觉反馈可以改善 BCI 用户的运动决策，并帮助他们获得对机器人的控制感。我们的结果表明，通过将多感觉反馈与 BCI 用户相结合，有可能增加机器人的控制。
