Chan Vincent Yue-Sek, Jin Craig T, van Schaik André
School of Electrical and Information Engineering, The University of Sydney Sydney, NSW, Australia.
Front Neurosci. 2012 Feb 8;6:21. doi: 10.3389/fnins.2012.00021. eCollection 2012.
This paper presents the first robotic system featuring audio-visual (AV) sensor fusion with neuromorphic sensors. We combine a pair of silicon cochleae and a silicon retina on a robotic platform to allow the robot to learn sound localization through self motion and visual feedback, using an adaptive ITD-based sound localization algorithm. After training, the robot can localize sound sources (white or pink noise) in a reverberant environment with an RMS error of 4-5° in azimuth. We also investigate the AV source binding problem and an experiment is conducted to test the effectiveness of matching an audio event with a corresponding visual event based on their onset time. Despite the simplicity of this method and a large number of false visual events in the background, a correct match can be made 75% of the time during the experiment.
本文介绍了首个具有视听（AV）传感器与神经形态传感器融合功能的机器人系统。我们在一个机器人平台上组合了一对硅耳蜗和一个硅视网膜，以使机器人能够通过自身运动和视觉反馈，利用基于自适应耳间时间差（ITD）的声音定位算法来学习声音定位。经过训练后，该机器人能够在混响环境中定位声源（白噪声或粉红噪声），方位角的均方根误差为4 - 5°。我们还研究了视听源绑定问题，并进行了一项实验，以测试基于音频事件和相应视觉事件的起始时间来匹配二者的有效性。尽管此方法简单，且背景中有大量虚假视觉事件，但在实验过程中仍有75%的时间能够做出正确匹配。
