IEEE Trans Image Process. 2018 Apr;27(4):1611-1625. doi: 10.1109/TIP.2017.2781303.
Emerging sensor designs increasingly rely on novel color filter arrays (CFAs) to sample the incident spectrum in unconventional ways. In particular, capturing a near-infrared (NIR) channel along with conventional RGB color is an exciting new imaging modality. RGB+NIR sensing has broad applications in computational photography, such as low-light denoising, it has applications in computer vision, such as facial recognition and tracking, and it paves the way toward low-cost single-sensor RGB and depth imaging using structured illumination. However, cost-effective commercial CFAs suffer from severe spectral cross talk. This cross talk represents a major challenge in high-quality RGB+NIR imaging, rendering existing spatially multiplexed sensor designs impractical. In this work, we introduce a new approach to RGB+NIR image reconstruction using learned convolutional sparse priors. We demonstrate high-quality color and NIR imaging for challenging scenes, even including high-frequency structured NIR illumination. The effectiveness of the proposed method is validated on a large data set of experimental captures, and simulated benchmark results which demonstrate that this work achieves unprecedented reconstruction quality.
新兴的传感器设计越来越依赖于新型的彩色滤光片阵列 (CFA) 以非常规的方式对入射光谱进行采样。特别是，捕捉近红外 (NIR) 通道以及传统的 RGB 颜色是一种令人兴奋的新成像模式。RGB+NIR 感应在计算摄影中有广泛的应用，例如低光降噪，在计算机视觉中有应用，例如人脸识别和跟踪，并且为使用结构光的低成本单传感器 RGB 和深度成像铺平了道路。然而，具有成本效益的商业 CFA 存在严重的光谱串扰。这种串扰在高质量的 RGB+NIR 成像中是一个重大挑战，使得现有的空间复用传感器设计变得不切实际。在这项工作中，我们提出了一种使用学习卷积稀疏先验的 RGB+NIR 图像重建新方法。我们展示了具有挑战性场景的高质量彩色和近红外成像，甚至包括高频结构近红外照明。所提出方法的有效性在大量实验采集数据集和模拟基准结果上得到了验证，这些结果表明这项工作实现了前所未有的重建质量。
