Müller-Cleve Simon F, Fra Vittorio, Khacef Lyes, Pequeño-Zurro Alejandro, Klepatsch Daniel, Forno Evelina, Ivanovich Diego G, Rastogi Shavika, Urgese Gianvito, Zenke Friedemann, Bartolozzi Chiara
Istituto Italiano di Tecnologia, Event-Driven Perception in Robotics, Genoa, Italy.
Politecnico di Torino, Electronic Design Automation (EDA) Group, Torino, Italy.
Front Neurosci. 2022 Nov 11;16:951164. doi: 10.3389/fnins.2022.951164. eCollection 2022.
Spatio-temporal pattern recognition is a fundamental ability of the brain which is required for numerous real-world activities. Recent deep learning approaches have reached outstanding accuracies in such tasks, but their implementation on conventional embedded solutions is still very computationally and energy expensive. Tactile sensing in robotic applications is a representative example where real-time processing and energy efficiency are required. Following a brain-inspired computing approach, we propose a new benchmark for spatio-temporal tactile pattern recognition at the edge through Braille letter reading. We recorded a new Braille letters dataset based on the capacitive tactile sensors of the iCub robot's fingertip. We then investigated the importance of spatial and temporal information as well as the impact of event-based encoding on spike-based computation. Afterward, we trained and compared feedforward and recurrent Spiking Neural Networks (SNNs) offline using Backpropagation Through Time (BPTT) with surrogate gradients, then we deployed them on the Intel Loihi neuromorphic chip for fast and efficient inference. We compared our approach to standard classifiers, in particular to the Long Short-Term Memory (LSTM) deployed on the embedded NVIDIA Jetson GPU, in terms of classification accuracy, power, and energy consumption together with computational delay. Our results show that the LSTM reaches ~97% of accuracy, outperforming the recurrent SNN by ~17% when using continuous frame-based data instead of event-based inputs. However, the recurrent SNN on Loihi with event-based inputs is ~500 times more energy-efficient than the LSTM on Jetson, requiring a total power of only ~30 mW. This work proposes a new benchmark for tactile sensing and highlights the challenges and opportunities of event-based encoding, neuromorphic hardware, and spike-based computing for spatio-temporal pattern recognition at the edge.
时空模式识别是大脑的一项基本能力，许多现实世界的活动都需要它。最近的深度学习方法在这类任务中取得了出色的准确率，但其在传统嵌入式解决方案上的实现仍然在计算和能源方面成本高昂。机器人应用中的触觉传感就是一个需要实时处理和能源效率的典型例子。遵循受大脑启发的计算方法，我们通过盲文阅读提出了一种用于边缘时空触觉模式识别的新基准。我们基于iCub机器人指尖的电容式触觉传感器记录了一个新的盲文字母数据集。然后，我们研究了空间和时间信息的重要性以及基于事件的编码对基于脉冲的计算的影响。之后，我们使用带替代梯度的时间反向传播（BPTT）离线训练并比较了前馈和递归脉冲神经网络（SNN），然后将它们部署在英特尔Loihi神经形态芯片上进行快速高效的推理。我们在分类准确率、功率、能耗以及计算延迟方面，将我们的方法与标准分类器，特别是部署在嵌入式NVIDIA Jetson GPU上的长短期记忆（LSTM）进行了比较。我们的结果表明，当使用基于连续帧的数据而非基于事件的输入时，LSTM的准确率达到了约97%，比递归SNN高出约17%。然而，在Loihi上使用基于事件输入的递归SNN比Jetson上的LSTM节能约500倍，总功率仅需约30 mW。这项工作提出了一种新的触觉传感基准，并突出了基于事件的编码、神经形态硬件以及基于脉冲的计算在边缘时空模式识别方面的挑战和机遇。
