Event Driven Perception for Robotics, Istituto Italiano di Tecnologia, 16163, Genoa, Italy.
Electrical Engineering and Computer Science, Technische Universität Berlin, 10623, Berlin, Germany.
Sci Rep. 2022 May 10;12(1):7645. doi: 10.1038/s41598-022-11723-6.
To interact with its environment, a robot working in 3D space needs to organise its visual input in terms of objects or their perceptual precursors, proto-objects. Among other visual cues, depth is a submodality used to direct attention to visual features and objects. Current depth-based proto-object attention models have been implemented for standard RGB-D cameras that produce synchronous frames. In contrast, event cameras are neuromorphic sensors that loosely mimic the function of the human retina by asynchronously encoding per-pixel brightness changes at very high temporal resolution, thereby providing advantages like high dynamic range, efficiency (thanks to their high degree of signal compression), and low latency. We propose a bio-inspired bottom-up attention model that exploits event-driven sensing to generate depth-based saliency maps that allow a robot to interact with complex visual input. We use event-cameras mounted in the eyes of the iCub humanoid robot to directly extract edge, disparity and motion information. Real-world experiments demonstrate that our system robustly selects salient objects near the robot in the presence of clutter and dynamic scene changes, for the benefit of downstream applications like object segmentation, tracking and robot interaction with external objects.
为了与 3D 空间中的环境交互，在该空间中工作的机器人需要根据对象或其感知前体（原对象）组织其视觉输入。在其他视觉线索中，深度是一种用于将注意力引导到视觉特征和对象的子模态。当前基于深度的原对象注意模型已针对生成同步帧的标准 RGB-D 相机实现。相比之下，事件相机是一种神经形态传感器，通过以非常高的时间分辨率异步地对每个像素的亮度变化进行编码，从而松散地模拟人眼的功能，从而提供了高动态范围、效率（由于其高度的信号压缩）和低延迟等优势。我们提出了一种受生物启发的自下而上的注意模型，该模型利用事件驱动的传感来生成基于深度的显着性图，使机器人能够与复杂的视觉输入进行交互。我们使用安装在 iCub 人形机器人眼睛中的事件相机来直接提取边缘、视差和运动信息。实际实验表明，我们的系统在存在杂乱和动态场景变化的情况下，能够稳健地选择机器人附近的显着对象，从而有利于下游应用，如对象分割、跟踪和机器人与外部对象的交互。
