Paulun Lukas, Wendt Anne, Kasabov Nikola
Knowledge Engineering and Discovery Research Institute, Auckland University of Technology, Auckland, New Zealand.
Mathematical Institute, Albert Ludwigs University of Freiburg, Freiburg im Breisgau, Germany.
Front Comput Neurosci. 2018 Jun 12;12:42. doi: 10.3389/fncom.2018.00042. eCollection 2018.
This paper introduces a new system for dynamic visual recognition that combines bio-inspired hardware with a brain-like spiking neural network. The system is designed to take data from a dynamic vision sensor (DVS) that simulates the functioning of the human retina by producing an address event output (spike trains) based on the movement of objects. The system then convolutes the spike trains and feeds them into a brain-like spiking neural network, called NeuCube, which is organized in a three-dimensional manner, representing the organization of the primary visual cortex. Spatio-temporal patterns of the data are learned during a deep unsupervised learning stage, using spike-timing-dependent plasticity. In a second stage, supervised learning is performed to train the network for classification tasks. The convolution algorithm and the mapping into the network mimic the function of retinal ganglion cells and the retinotopic organization of the visual cortex. The NeuCube architecture can be used to visualize the deep connectivity inside the network before, during, and after training and thereby allows for a better understanding of the learning processes. The method was tested on the benchmark MNIST-DVS dataset and achieved a classification accuracy of 92.90%. The paper discusses advantages and limitations of the new method and concludes that it is worth exploring further on different datasets, aiming for advances in dynamic computer vision and multimodal systems that integrate visual, aural, tactile, and other kinds of information in a biologically plausible way.
本文介绍了一种用于动态视觉识别的新系统，该系统将受生物启发的硬件与类脑脉冲神经网络相结合。该系统旨在从动态视觉传感器（DVS）获取数据，该传感器通过基于物体的运动产生地址事件输出（脉冲序列）来模拟人类视网膜的功能。然后，该系统对脉冲序列进行卷积，并将其输入到一个名为NeuCube的类脑脉冲神经网络中，该网络以三维方式组织，代表初级视觉皮层的组织方式。在深度无监督学习阶段，利用脉冲时间依赖可塑性学习数据的时空模式。在第二阶段，进行监督学习以训练网络执行分类任务。卷积算法和网络映射模仿了视网膜神经节细胞的功能以及视觉皮层的视网膜拓扑组织。NeuCube架构可用于在训练前、训练期间和训练后可视化网络内部的深度连接，从而更好地理解学习过程。该方法在基准MNIST-DVS数据集上进行了测试，分类准确率达到了92.90%。本文讨论了新方法的优点和局限性，并得出结论，值得在不同数据集上进一步探索，以期在动态计算机视觉和以生物学上合理的方式整合视觉、听觉、触觉和其他类型信息的多模态系统方面取得进展。
