Liu Risheng, Liu Jinyuan, Jiang Zhiying, Fan Xin, Luo Zhongxuan
IEEE Trans Image Process. 2021;30:1261-1274. doi: 10.1109/TIP.2020.3043125. Epub 2020 Dec 21.
Image fusion plays a critical role in a variety of vision and learning applications. Current fusion approaches are designed to characterize source images, focusing on a certain type of fusion task while limited in a wide scenario. Moreover, other fusion strategies (i.e., weighted averaging, choose-max) cannot undertake the challenging fusion tasks, which furthermore leads to undesirable artifacts facilely emerged in their fused results. In this paper, we propose a generic image fusion method with a bilevel optimization paradigm, targeting on multi-modality image fusion tasks. Corresponding alternation optimization is conducted on certain components decoupled from source images. Via adaptive integration weight maps, we are able to get the flexible fusion strategy across multi-modality images. We successfully applied it to three types of image fusion tasks, including infrared and visible, computed tomography and magnetic resonance imaging, and magnetic resonance imaging and single-photon emission computed tomography image fusion. Results highlight the performance and versatility of our approach from both quantitative and qualitative aspects.
图像融合在各种视觉和学习应用中起着关键作用。当前的融合方法旨在对源图像进行特征描述，专注于特定类型的融合任务，而在广泛的场景中存在局限性。此外，其他融合策略（即加权平均、最大值选择）无法承担具有挑战性的融合任务，这进而导致在其融合结果中容易出现不理想的伪影。在本文中，我们提出了一种具有双层优化范式的通用图像融合方法，针对多模态图像融合任务。对从源图像解耦的某些组件进行相应的交替优化。通过自适应积分权重图，我们能够获得跨多模态图像的灵活融合策略。我们成功地将其应用于三种类型的图像融合任务，包括红外与可见光、计算机断层扫描与磁共振成像，以及磁共振成像与单光子发射计算机断层扫描图像融合。结果从定量和定性两个方面突出了我们方法的性能和通用性。
