IEEE Trans Pattern Anal Mach Intell. 2023 Apr;45(4):4537-4551. doi: 10.1109/TPAMI.2022.3196652. Epub 2023 Mar 7.
It has been shown that equivariant convolution is very helpful for many types of computer vision tasks. Recently, the 2D filter parametrization technique has played an important role for designing equivariant convolutions, and has achieved success in making use of rotation symmetry of images. However, the current filter parametrization strategy still has its evident drawbacks, where the most critical one lies in the accuracy problem of filter representation. To address this issue, in this paper we explore an ameliorated Fourier series expansion for 2D filters, and propose a new filter parametrization method based on it. The proposed filter parametrization method not only finely represents 2D filters with zero error when the filter is not rotated (similar as the classical Fourier series expansion), but also substantially alleviates the aliasing-effect-caused quality degradation when the filter is rotated (which usually arises in classical Fourier series expansion method). Accordingly, we construct a new equivariant convolution method based on the proposed filter parametrization method, named F-Conv. We prove that the equivariance of the proposed F-Conv is exact in the continuous domain, which becomes approximate only after discretization. Moreover, we provide theoretical error analysis for the case when the equivariance is approximate, showing that the approximation error is related to the mesh size and filter size. Extensive experiments show the superiority of the proposed method. Particularly, we adopt rotation equivariant convolution methods to a typical low-level image processing task, image super-resolution. It can be substantiated that the proposed F-Conv based method evidently outperforms classical convolution based methods. Compared with pervious filter parametrization based methods, the F-Conv performs more accurately on this low-level image processing task, reflecting its intrinsic capability of faithfully preserving rotation symmetries in local image features.
已证明等变卷积对于许多类型的计算机视觉任务非常有帮助。最近，二维滤波器参数化技术在设计等变卷积方面发挥了重要作用，并成功利用了图像的旋转对称性。然而，当前的滤波器参数化策略仍然存在明显的缺陷，其中最关键的问题在于滤波器表示的准确性。为了解决这个问题，本文探索了一种改进的二维滤波器傅里叶级数展开，并提出了一种基于它的新滤波器参数化方法。所提出的滤波器参数化方法不仅在滤波器未旋转时（类似于经典的傅里叶级数展开）可以零误差地精细表示二维滤波器，而且在滤波器旋转时（这在经典傅里叶级数展开方法中通常会出现）大大减轻了由混叠效应引起的质量下降。因此，我们基于所提出的滤波器参数化方法构建了一种新的等变卷积方法，称为 F-Conv。我们证明了所提出的 F-Conv 的等变性在连续域中是精确的，仅在离散化后才变得近似。此外，我们针对等变性近似的情况提供了理论误差分析，表明近似误差与网格大小和滤波器大小有关。广泛的实验表明了所提出方法的优越性。特别是，我们将旋转等变卷积方法应用于典型的低水平图像处理任务，即图像超分辨率。可以证明，基于所提出的 F-Conv 的方法在这个低水平图像处理任务中明显优于经典卷积方法。与之前的基于滤波器参数化的方法相比，F-Conv 在这个低水平图像处理任务上表现出更高的准确性，反映了其在局部图像特征中忠实保留旋转对称性的内在能力。
