Lei Chenyang, Jiang Xudong, Chen Qifeng
IEEE Trans Pattern Anal Mach Intell. 2023 Dec;45(12):15530-15545. doi: 10.1109/TPAMI.2023.3314972. Epub 2023 Nov 3.
We propose a simple yet effective reflection-free cue for robust reflection removal from a pair of flash and ambient (no-flash) images. The reflection-free cue exploits a flash-only image obtained by subtracting the ambient image from the corresponding flash image in raw data space. The flash-only image is equivalent to an image taken in a dark environment with only a flash on. This flash-only image is visually reflection-free and thus can provide robust cues to infer the reflection in the ambient image. Since the flash-only image usually has artifacts, we further propose a dedicated model that not only utilizes the reflection-free cue but also avoids introducing artifacts, which helps accurately estimate reflection and transmission. Our experiments on real-world images with various types of reflection demonstrate the effectiveness of our model with reflection-free flash-only cues: our model outperforms state-of-the-art reflection removal approaches by more than 5.23 dB in PSNR. We extend our approach to handheld photography to address the misalignment between the flash and no-flash pair. With misaligned training data and the alignment module, our aligned model outperforms our previous version by more than 3.19 dB in PSNR on a misaligned dataset. We also study using linear RGB images as training data.
我们提出了一种简单而有效的无反射线索，用于从一对闪光灯和环境光（无闪光灯）图像中稳健地去除反射。该无反射线索利用通过在原始数据空间中从相应的闪光灯图像中减去环境光图像而获得的仅闪光灯图像。仅闪光灯图像等同于在仅打开闪光灯的黑暗环境中拍摄的图像。该仅闪光灯图像在视觉上是无反射的，因此可以提供稳健的线索来推断环境光图像中的反射。由于仅闪光灯图像通常具有伪影，我们进一步提出了一种专用模型，该模型不仅利用无反射线索，还避免引入伪影，这有助于准确估计反射和透射。我们在具有各种类型反射的真实世界图像上进行的实验证明了我们具有无反射仅闪光灯线索的模型的有效性：我们的模型在峰值信噪比（PSNR）方面比最先进的反射去除方法高出超过5.23 dB。我们将我们的方法扩展到手持摄影，以解决闪光灯和无闪光灯对之间的未对准问题。通过未对准的训练数据和对准模块，我们的对准模型在未对准数据集上的PSNR方面比我们的先前版本高出超过3.19 dB。我们还研究使用线性RGB图像作为训练数据。
