IEEE Trans Image Process. 2022;31:5733-5747. doi: 10.1109/TIP.2022.3201476. Epub 2022 Sep 2.
The ability to accurately overlay one modality retinal image to another is critical in ophthalmology. Our previous framework achieved the state-of-the-art results for multimodal retinal image registration. However, it requires human-annotated labels due to the supervised approach of the previous work. In this paper, we propose a self-supervised multimodal retina registration method to alleviate the burdens of time and expense to prepare for training data, that is, aiming to automatically register multimodal retinal images without any human annotations. Specially, we focus on registering color fundus images with infrared reflectance and fluorescein angiography images, and compare registration results with several conventional and supervised and unsupervised deep learning methods. From the experimental results, the proposed self-supervised framework achieves a comparable accuracy comparing to the state-of-the-art supervised learning method in terms of registration accuracy and Dice coefficient.
准确地将一种模态的视网膜图像叠加到另一种模态上，在眼科中至关重要。我们之前的框架在多模态视网膜图像配准方面取得了最先进的结果。然而，由于之前工作的监督方法，它需要人工注释标签。在本文中，我们提出了一种自监督的多模态视网膜配准方法，以减轻为训练数据做准备的时间和费用负担，也就是说，旨在自动配准多模态视网膜图像，而无需任何人工注释。特别地，我们专注于将彩色眼底图像与红外反射和荧光血管造影图像进行配准，并将配准结果与几种传统的和监督的和无监督的深度学习方法进行比较。从实验结果来看，与最先进的监督学习方法相比，所提出的自监督框架在配准精度和骰子系数方面达到了相当的精度。
