Cao Chunshui, Huang Yongzhen, Yang Yi, Wang Liang, Wang Zilei, Tan Tieniu
IEEE Trans Pattern Anal Mach Intell. 2019 Jul;41(7):1627-1640. doi: 10.1109/TPAMI.2018.2843329. Epub 2018 Jun 4.
Feedback is a fundamental mechanism existing in the human visual system, but has not been explored deeply in designing computer vision algorithms. In this paper, we claim that feedback plays a critical role in understanding convolutional neural networks (CNNs), e.g., how a neuron in CNNs describes an object's pattern, and how a collection of neurons form comprehensive perception to an object. To model the feedback in CNNs, we propose a novel model named Feedback CNN and develop two new processing algorithms, i.e., neural pathway pruning and pattern recovering. We mathematically prove that the proposed method can reach local optimum. Note that Feedback CNN belongs to weakly supervised methods and can be trained only using category-level labels. But it possesses a powerful capability to accurately localize and segment category-specific objects. We conduct extensive visualization analysis, and the results reveal the close relationship between neurons and object parts in Feedback CNN. Finally, we evaluate the proposed Feedback CNN over the tasks of weakly supervised object localization and segmentation, and the experimental results on ImageNet and Pascal VOC show that our method remarkably outperforms the state-of-the-art ones.
反馈是人类视觉系统中存在的一种基本机制，但在设计计算机视觉算法方面尚未得到深入探索。在本文中，我们认为反馈在理解卷积神经网络（CNN）中起着关键作用，例如，CNN中的神经元如何描述物体的模式，以及神经元集合如何对物体形成综合感知。为了对CNN中的反馈进行建模，我们提出了一种名为反馈CNN的新型模型，并开发了两种新的处理算法，即神经通路修剪和模式恢复。我们从数学上证明了所提出的方法可以达到局部最优。请注意，反馈CNN属于弱监督方法，只能使用类别级标签进行训练。但它具有强大的能力，能够准确地定位和分割特定类别的物体。我们进行了广泛的可视化分析，结果揭示了反馈CNN中神经元与物体部分之间的密切关系。最后，我们在弱监督目标定位和分割任务上评估了所提出的反馈CNN，在ImageNet和Pascal VOC上的实验结果表明，我们的方法显著优于现有方法。
