Equipe de Vision et Calcul Naturel, UMR S968 Institut National de la Santé et de la Recherche Médicale, Centre National de la Recherche Scientifique UMR 7210, Centre Hospitalier National d' Ophtalmologie des Quinze-Vingts, Université Pierre et Marie Curie Paris, France.
Institute of Neuroinformatics, University of Zürich and Eidgenössische Technische Hochschule (ETH) Zürich Zürich, Switzerland.
Front Neurosci. 2015 Feb 24;9:46. doi: 10.3389/fnins.2015.00046. eCollection 2015.
Bio-inspired asynchronous event-based vision sensors are currently introducing a paradigm shift in visual information processing. These new sensors rely on a stimulus-driven principle of light acquisition similar to biological retinas. They are event-driven and fully asynchronous, thereby reducing redundancy and encoding exact times of input signal changes, leading to a very precise temporal resolution. Approaches for higher-level computer vision often rely on the reliable detection of features in visual frames, but similar definitions of features for the novel dynamic and event-based visual input representation of silicon retinas have so far been lacking. This article addresses the problem of learning and recognizing features for event-based vision sensors, which capture properties of truly spatiotemporal volumes of sparse visual event information. A novel computational architecture for learning and encoding spatiotemporal features is introduced based on a set of predictive recurrent reservoir networks, competing via winner-take-all selection. Features are learned in an unsupervised manner from real-world input recorded with event-based vision sensors. It is shown that the networks in the architecture learn distinct and task-specific dynamic visual features, and can predict their trajectories over time.
受生物启发的异步事件驱动视觉传感器目前正在视觉信息处理领域引发一场范式转变。这些新型传感器基于类似于生物视网膜的刺激驱动式的光采集原理。它们是事件驱动和完全异步的，从而减少了冗余并对输入信号变化的确切时间进行了编码，实现了非常精确的时间分辨率。用于高级计算机视觉的方法通常依赖于在视觉帧中可靠地检测特征，但对于新型动态和基于事件的硅视网膜视觉输入表示形式，类似的特征定义迄今一直缺乏。本文针对基于事件的视觉传感器的特征学习和识别问题展开研究，该传感器可以捕获稀疏视觉事件信息的真正时空体积的属性。本文提出了一种基于一组预测递归储层网络的新型计算架构，该架构通过竞争获胜者选择来实现特征学习和编码。从基于事件的视觉传感器记录的真实世界输入中以无监督的方式学习特征。结果表明，该架构中的网络可以学习到独特且特定于任务的动态视觉特征，并可以随着时间的推移预测它们的轨迹。
