Zhang Kaihao, Li Dongxu, Luo Wenhan, Ren Wenqi, Liu Wei
IEEE Trans Pattern Anal Mach Intell. 2023 Jan;45(1):1287-1293. doi: 10.1109/TPAMI.2022.3148707. Epub 2022 Dec 5.
Video deraining is an important task in computer vision as the unwanted rain hampers the visibility of videos and deteriorates the robustness of most outdoor vision systems. Despite the significant success which has been achieved for video deraining recently, two major challenges remain: 1) how to exploit the vast information among successive frames to extract powerful spatio-temporal features across both the spatial and temporal domains, and 2) how to restore high-quality derained videos with a high-speed approach. In this paper, we present a new end-to-end video deraining framework, dubbed Enhanced Spatio-Temporal Interaction Network (ESTINet), which considerably boosts current state-of-the-art video deraining quality and speed. The ESTINet takes the advantage of deep residual networks and convolutional long short-term memory, which can capture the spatial features and temporal correlations among successive frames at the cost of very little computational resource. Extensive experiments on three public datasets show that the proposed ESTINet can achieve faster speed than the competitors, while maintaining superior performance over the state-of-the-art methods. https://github.com/HDCVLab/Enhanced-Spatio-Temporal-Interaction-Learning-for-Video-Deraining.
视频去雨是计算机视觉中的一项重要任务，因为不必要的雨水会妨碍视频的可视性，并降低大多数户外视觉系统的鲁棒性。尽管最近视频去雨已经取得了显著成功，但仍存在两个主要挑战：1）如何利用连续帧之间的大量信息来跨空间和时间域提取强大的时空特征，以及2）如何用高速方法恢复高质量的去雨视频。在本文中，我们提出了一种新的端到端视频去雨框架，称为增强时空交互网络（ESTINet），它显著提高了当前最先进的视频去雨质量和速度。ESTINet利用深度残差网络和卷积长短期记忆，以极少的计算资源为代价捕捉连续帧之间的空间特征和时间相关性。在三个公共数据集上进行的大量实验表明，所提出的ESTINet可以比竞争对手实现更快的速度，同时保持优于最先进方法的性能。https://github.com/HDCVLab/Enhanced-Spatio-Temporal-Interaction-Learning-for-Video-Deraining。
