Zhang Pengyu, Ju Hao, Yu Lei, He Weihua, Wang Yaoyuan, Zhang Ziyang, Xu Qi, Li Shengming, Wang Dong, Lu Huchuan, Jia Xu
IEEE Trans Image Process. 2024;33:5824-5836. doi: 10.1109/TIP.2024.3468023. Epub 2024 Oct 15.
The goal of blurry image deblurring and unfolding task is to recover a single sharp frame or a sequence from a blurry one. Recently, its performance is greatly improved with introduction of a bio-inspired visual sensor, event camera. Most existing event-assisted deblurring methods focus on the design of powerful network architectures and effective training strategy, while ignoring the role of blur modeling in removing various blur in dynamic scenes. In this work, we propose to implicitly model blur in an image by computing blurriness representation with an event-assisted blurriness encoder. The learning of blurriness representation is formulated as a ranking problem based on specially synthesized pairs. Blurriness-aware image unfolding is achieved by integrating blur relevant information contained in the representation into a base unfolding network. The integration is mainly realized by the proposed blurriness-guided modulation and multi-scale aggregation modules. Experiments on GOPRO and HQF datasets show favorable performance of the proposed method against state-of-the-art approaches. More results on real-world data validate its effectiveness in recovering a sequence of latent sharp frames from a blurry image.
模糊图像去模糊和展开任务的目标是从模糊图像中恢复单个清晰帧或序列。最近，随着受生物启发的视觉传感器——事件相机的引入，其性能有了很大提高。大多数现有的事件辅助去模糊方法都集中在强大的网络架构设计和有效的训练策略上，而忽略了模糊建模在去除动态场景中各种模糊方面的作用。在这项工作中，我们建议通过使用事件辅助模糊编码器计算模糊度表示来隐式地对图像中的模糊进行建模。模糊度表示的学习被公式化为基于特殊合成对的排序问题。通过将表示中包含的与模糊相关的信息集成到基础展开网络中，实现了模糊感知图像展开。这种集成主要通过所提出的模糊引导调制和多尺度聚合模块来实现。在GOPRO和HQF数据集上的实验表明，所提出的方法相对于现有方法具有良好的性能。在真实世界数据上的更多结果验证了其从模糊图像中恢复一系列潜在清晰帧的有效性。
