Faculty of Information Technology and Communication Sciences, Tampere University, Tampere, Finland.
Department of Electrical Engineering, Qatar University, Doha, Qatar.
Neural Netw. 2021 Mar;135:201-211. doi: 10.1016/j.neunet.2020.12.014. Epub 2020 Dec 23.
Discriminative learning based on convolutional neural networks (CNNs) aims to perform image restoration by learning from training examples of noisy-clean image pairs. It has become the go-to methodology for tackling image restoration and has outperformed the traditional non-local class of methods. However, the top-performing networks are generally composed of many convolutional layers and hundreds of neurons, with trainable parameters in excess of several million. We claim that this is due to the inherently linear nature of convolution-based transformation, which is inadequate for handling severe restoration problems. Recently, a non-linear generalization of CNNs, called the operational neural networks (ONN), has been shown to outperform CNN on AWGN denoising. However, its formulation is burdened by a fixed collection of well-known non-linear operators and an exhaustive search to find the best possible configuration for a given architecture, whose efficacy is further limited by a fixed output layer operator assignment. In this study, we leverage the Taylor series-based function approximation to propose a self-organizing variant of ONNs, Self-ONNs, for image restoration, which synthesizes novel nodal transformations on-the-fly as part of the learning process, thus eliminating the need for redundant training runs for operator search. In addition, it enables a finer level of operator heterogeneity by diversifying individual connections of the receptive fields and weights. We perform a series of extensive ablation experiments across three severe image restoration tasks. Even when a strict equivalence of learnable parameters is imposed, Self-ONNs surpass CNNs by a considerable margin across all problems, improving the generalization performance by up to 3 dB in terms of PSNR.
基于卷积神经网络 (CNN) 的判别式学习旨在通过从噪声-清洁图像对的训练示例中学习来执行图像恢复。它已成为解决图像恢复问题的首选方法，并且优于传统的非局部类方法。然而，性能最佳的网络通常由许多卷积层和数百个神经元组成，可训练参数超过数百万个。我们声称，这是由于基于卷积的变换的固有线性性质，这对于处理严重的恢复问题是不够的。最近，称为操作神经网络 (ONN) 的 CNN 的非线性推广已经在加性高斯白噪声去噪方面优于 CNN。然而，它的公式受到一组固定的知名非线性运算符和为给定架构找到最佳可能配置的详尽搜索的限制，其有效性进一步受到固定输出层运算符分配的限制。在这项研究中，我们利用基于泰勒级数的函数逼近来提出一种用于图像恢复的自组织 ONN 变体，Self-ONNs，它在学习过程中即时合成新的节点变换，从而消除了冗余的训练运行以进行运算符搜索的需要。此外，它通过使感受野和权重的各个连接多样化，实现了更细粒度的运算符异质性。我们在三个严重的图像恢复任务中进行了一系列广泛的消融实验。即使在严格等同的可学习参数的情况下，Self-ONNs 在所有问题上都以相当大的优势超过了 CNN，在 PSNR 方面提高了 3dB 的泛化性能。
