Xu Xiangyu, Ma Yongrui, Sun Wenxiu, Yang Ming-Hsuan
IEEE Trans Pattern Anal Mach Intell. 2022 Apr;44(4):1905-1921. doi: 10.1109/TPAMI.2020.3032476. Epub 2022 Mar 4.
Super-resolution is a fundamental problem in computer vision which aims to overcome the spatial limitation of camera sensors. While significant progress has been made in single image super-resolution, most algorithms only perform well on synthetic data, which limits their applications in real scenarios. In this paper, we study the problem of real-scene single image super-resolution to bridge the gap between synthetic data and real captured images. We focus on two issues of existing super-resolution algorithms: lack of realistic training data and insufficient utilization of visual information obtained from cameras. To address the first issue, we propose a method to generate more realistic training data by mimicking the imaging process of digital cameras. For the second issue, we develop a two-branch convolutional neural network to exploit the radiance information originally-recorded in raw images. In addition, we propose a dense channel-attention block for better image restoration as well as a learning-based guided filter network for effective color correction. Our model is able to generalize to different cameras without deliberately training on images from specific camera types. Extensive experiments demonstrate that the proposed algorithm can recover fine details and clear structures, and achieve high-quality results for single image super-resolution in real scenes.
超分辨率是计算机视觉中的一个基本问题，旨在克服相机传感器的空间限制。虽然单图像超分辨率已经取得了显著进展，但大多数算法仅在合成数据上表现良好，这限制了它们在实际场景中的应用。在本文中，我们研究真实场景单图像超分辨率问题，以弥合合成数据与真实拍摄图像之间的差距。我们关注现有超分辨率算法的两个问题：缺乏真实的训练数据以及对从相机获得的视觉信息利用不足。为了解决第一个问题，我们提出一种通过模拟数码相机成像过程来生成更真实训练数据的方法。对于第二个问题，我们开发了一个双分支卷积神经网络来利用原始图像中最初记录的辐射信息。此外，我们提出了一个密集通道注意力块以实现更好的图像恢复，以及一个基于学习的引导滤波器网络以进行有效的色彩校正。我们的模型能够在不针对特定相机类型的图像进行专门训练的情况下推广到不同相机。大量实验表明，所提出的算法能够恢复精细细节和清晰结构，并在真实场景中实现单图像超分辨率的高质量结果。
