Institute for Neural Information Processing, Ulm University, James-Franck-Ring, Ulm, 89081, Germany.
Biol Cybern. 2023 Oct;117(4-5):299-329. doi: 10.1007/s00422-023-00966-9. Epub 2023 Jun 12.
Advanced computer vision mechanisms have been inspired by neuroscientific findings. However, with the focus on improving benchmark achievements, technical solutions have been shaped by application and engineering constraints. This includes the training of neural networks which led to the development of feature detectors optimally suited to the application domain. However, the limitations of such approaches motivate the need to identify computational principles, or motifs, in biological vision that can enable further foundational advances in machine vision. We propose to utilize structural and functional principles of neural systems that have been largely overlooked. They potentially provide new inspirations for computer vision mechanisms and models. Recurrent feedforward, lateral, and feedback interactions characterize general principles underlying processing in mammals. We derive a formal specification of core computational motifs that utilize these principles. These are combined to define model mechanisms for visual shape and motion processing. We demonstrate how such a framework can be adopted to run on neuromorphic brain-inspired hardware platforms and can be extended to automatically adapt to environment statistics. We argue that the identified principles and their formalization inspires sophisticated computational mechanisms with improved explanatory scope. These and other elaborated, biologically inspired models can be employed to design computer vision solutions for different tasks and they can be used to advance neural network architectures of learning.
先进的计算机视觉机制受到了神经科学发现的启发。然而,由于关注于提高基准测试成绩,技术解决方案受到了应用和工程限制的影响。这包括神经网络的训练,这导致了最适合应用领域的特征检测器的发展。然而,这些方法的局限性促使人们需要确定生物视觉中的计算原理或模式,从而能够在机器视觉中取得进一步的基础进展。我们建议利用在很大程度上被忽视的神经系统的结构和功能原理。它们为计算机视觉机制和模型提供了新的灵感。在哺乳动物中,递归前馈、侧部和反馈相互作用是处理的基本原理。我们推导出了利用这些原理的核心计算模式的形式规范。这些模式被组合起来定义用于视觉形状和运动处理的模型机制。我们展示了如何在神经形态脑启发硬件平台上采用这样的框架,并可以扩展到自动适应环境统计。我们认为,所确定的原理及其形式化激发了具有改进解释范围的复杂计算机制。这些和其他经过精心设计的、受生物启发的模型可用于为不同任务设计计算机视觉解决方案,并可用于推进学习的神经网络架构。