Layher Georg, Brosch Tobias, Neumann Heiko
Institute of Neural Information Processing, Ulm University Ulm, Germany.
Front Neurorobot. 2017 Mar 22;11:13. doi: 10.3389/fnbot.2017.00013. eCollection 2017.
Intelligent agents, such as robots, have to serve a multitude of autonomous functions. Examples are, e.g., collision avoidance, navigation and route planning, active sensing of its environment, or the interaction and non-verbal communication with people in the extended reach space. Here, we focus on the recognition of the action of a human agent based on a biologically inspired visual architecture of analyzing articulated movements. The proposed processing architecture builds upon coarsely segregated streams of sensory processing along different pathways which separately process form and motion information (Layher et al., 2014). Action recognition is performed in an event-based scheme by identifying representations of characteristic pose configurations (key poses) in an image sequence. In line with perceptual studies, key poses are selected unsupervised utilizing a feature-driven criterion which combines extrema in the motion energy with the horizontal and the vertical extendedness of a body shape. Per class representations of key pose frames are learned using a deep convolutional neural network consisting of 15 convolutional layers. The network is trained using the () framework (Esser et al., 2016), which realizes the mapping of the trained synaptic weights onto the platform (Merolla et al., 2014). After the mapping, the trained network achieves real-time capabilities for processing input streams and classify input images at about 1,000 frames per second while the computational stages only consume about 70 mW of energy (without spike transduction). Particularly regarding mobile robotic systems, a low energy profile might be crucial in a variety of application scenarios. Cross-validation results are reported for two different datasets and compared to state-of-the-art action recognition approaches. The results demonstrate, that (I) the presented approach is on par with other key pose based methods described in the literature, which select key pose frames by optimizing classification accuracy, (II) compared to the training on the full set of frames, representations trained on key pose frames result in a higher confidence in class assignments, and (III) key pose representations show promising generalization capabilities in a cross-dataset evaluation.
智能体，如机器人，必须执行多种自主功能。例如，避撞、导航与路线规划、对其环境的主动感知，或在扩展可达空间内与人员的交互和非语言通信。在此，我们专注于基于一种受生物启发的用于分析关节运动的视觉架构来识别人类智能体的动作。所提出的处理架构基于沿不同路径粗略分离的感官处理流构建，这些路径分别处理形状和运动信息（莱赫尔等人，2014年）。动作识别通过基于事件的方案进行，即识别图像序列中特征姿势配置（关键姿势）的表示。与感知研究一致，关键姿势是利用一种特征驱动标准无监督选择的，该标准将运动能量中的极值与身体形状的水平和垂直延伸相结合。使用由15个卷积层组成的深度卷积神经网络来学习关键姿势帧的每类表示。该网络使用（）框架（埃塞尔等人，2016年）进行训练，该框架实现了将训练后的突触权重映射到平台上（梅罗拉等人，2014年）。映射之后，训练后的网络实现了处理输入流的实时能力，并以每秒约1000帧的速度对输入图像进行分类，而计算阶段仅消耗约70毫瓦的能量（无尖峰转换）。特别是对于移动机器人系统，低能耗在各种应用场景中可能至关重要。报告了针对两个不同数据集的交叉验证结果，并与当前最先进的动作识别方法进行了比较。结果表明，（I）所提出的方法与文献中描述的其他基于关键姿势的方法相当，这些方法通过优化分类准确率来选择关键姿势帧；（II）与在完整帧集上进行训练相比，在关键姿势帧上训练的表示在类别分配中具有更高的置信度；（III）关键姿势表示在跨数据集评估中显示出有前景的泛化能力。
