Zhang Tianwei, Wei Dong, Zhu Mengmeng, Gu Shi, Zheng Yefeng
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China.
Jarvis Research Center, Tencent YouTu Lab, Shenzhen 518057, China.
Med Image Anal. 2024 May;94:103151. doi: 10.1016/j.media.2024.103151. Epub 2024 Mar 21.
Self-supervised learning has emerged as a powerful tool for pretraining deep networks on unlabeled data, prior to transfer learning of target tasks with limited annotation. The relevance between the pretraining pretext and target tasks is crucial to the success of transfer learning. Various pretext tasks have been proposed to utilize properties of medical image data (e.g., three dimensionality), which are more relevant to medical image analysis than generic ones for natural images. However, previous work rarely paid attention to data with anatomy-oriented imaging planes, e.g., standard cardiac magnetic resonance imaging views. As these imaging planes are defined according to the anatomy of the imaged organ, pretext tasks effectively exploiting this information can pretrain the networks to gain knowledge on the organ of interest. In this work, we propose two complementary pretext tasks for this group of medical image data based on the spatial relationship of the imaging planes. The first is to learn the relative orientation between the imaging planes and implemented as regressing their intersecting lines. The second exploits parallel imaging planes to regress their relative slice locations within a stack. Both pretext tasks are conceptually straightforward and easy to implement, and can be combined in multitask learning for better representation learning. Thorough experiments on two anatomical structures (heart and knee) and representative target tasks (semantic segmentation and classification) demonstrate that the proposed pretext tasks are effective in pretraining deep networks for remarkably boosted performance on the target tasks, and superior to other recent approaches.
自监督学习已成为一种强大的工具，用于在未标记数据上对深度网络进行预训练，然后再对标注有限的目标任务进行迁移学习。预训练 pretext 与目标任务之间的相关性对于迁移学习的成功至关重要。已经提出了各种 pretext 任务来利用医学图像数据的特性（例如三维性），这些特性与医学图像分析的相关性比自然图像的通用特性更高。然而，先前的工作很少关注具有面向解剖结构成像平面的数据，例如标准心脏磁共振成像视图。由于这些成像平面是根据成像器官的解剖结构定义的，有效利用此信息的 pretext 任务可以对网络进行预训练，以获取有关感兴趣器官的知识。在这项工作中，我们基于成像平面的空间关系为这组医学图像数据提出了两个互补的 pretext 任务。第一个任务是学习成像平面之间的相对方向，并通过回归它们的交线来实现。第二个任务利用平行成像平面来回归它们在堆栈中的相对切片位置。这两个 pretext 任务在概念上都很简单且易于实现，并且可以在多任务学习中结合使用以进行更好的表示学习。对两种解剖结构（心脏和膝盖）以及代表性目标任务（语义分割和分类）进行的全面实验表明，所提出的 pretext 任务在预训练深度网络方面是有效的，能够显著提高目标任务的性能，并且优于其他近期方法。
