Department of Computer Science, University of Maryland, College Park, MD 20742, USA.
Sci Robot. 2019 May 15;4(30). doi: 10.1126/scirobotics.aaw6736.
The hallmark of modern robotics is the ability to directly fuse the platform's perception with its motoric ability-the concept often referred to as "active perception." Nevertheless, we find that action and perception are often kept in separated spaces, which is a consequence of traditional vision being frame based and only existing in the moment and motion being a continuous entity. This bridge is crossed by the dynamic vision sensor (DVS), a neuromorphic camera that can see the motion. We propose a method of encoding actions and perceptions together into a single space that is meaningful, semantically informed, and consistent by using hyperdimensional binary vectors (HBVs). We used DVS for visual perception and showed that the visual component can be bound with the system velocity to enable dynamic world perception, which creates an opportunity for real-time navigation and obstacle avoidance. Actions performed by an agent are directly bound to the perceptions experienced to form its own "memory." Furthermore, because HBVs can encode entire histories of actions and perceptions-from atomic to arbitrary sequences-as constant-sized vectors, autoassociative memory was combined with deep learning paradigms for controls. We demonstrate these properties on a quadcopter drone ego-motion inference task and the MVSEC (multivehicle stereo event camera) dataset.
现代机器人学的标志是能够将平台的感知直接与运动能力融合在一起——这一概念通常被称为“主动感知”。然而，我们发现动作和感知经常被保持在分离的空间中，这是传统视觉基于帧且仅存在于瞬间，而运动是连续实体的结果。动态视觉传感器 (DVS) 跨越了这一鸿沟，DVS 是一种能够感知运动的神经形态相机。我们提出了一种方法，通过使用超维二进制向量 (HBV) 将动作和感知一起编码到一个有意义的、语义上有信息的、一致的单一空间中。我们使用 DVS 进行视觉感知，并展示了视觉组件可以与系统速度绑定，从而实现动态世界感知，这为实时导航和避障创造了机会。代理执行的动作直接与所经历的感知绑定，从而形成其自身的“记忆”。此外，由于 HBV 可以将动作和感知的整个历史——从原子序列到任意序列——编码为固定大小的向量，因此自联想记忆与深度学习范例相结合用于控制。我们在四旋翼无人机自身运动推断任务和 MVSEC（多车辆立体事件相机）数据集上演示了这些特性。
