IEEE Trans Ultrason Ferroelectr Freq Control. 2020 Dec;67(12):2510-2518. doi: 10.1109/TUFFC.2020.3015081. Epub 2020 Nov 24.
One way of resolving the problem of scarce and expensive data in deep learning for medical applications is using transfer learning and fine-tuning a network which has been trained on a large data set. The common practice in transfer learning is to keep the shallow layers unchanged and to modify deeper layers according to the new data set. This approach may not work when using a U-Net and when moving from a different domain to ultrasound (US) images due to their drastically different appearance. In this study, we investigated the effect of fine-tuning different sets of layers of a pretrained U-Net for US image segmentation. Two different schemes were analyzed, based on two different definitions of shallow and deep layers. We studied simulated US images, as well as two human US data sets. We also included a chest X-ray data set. The results showed that choosing which layers to fine-tune is a critical task. In particular, they demonstrated that fine-tuning the last layers of the network, which is the common practice for classification networks, is often the worst strategy. It may therefore be more appropriate to fine-tune the shallow layers rather than deep layers in US image segmentation when using a U-Net. Shallow layers learn lower level features which are critical in automatic segmentation of medical images. Even when a large US data set is available, we observed that fine-tuning shallow layers is a faster approach compared to fine-tuning the whole network.
在医学应用的深度学习中，解决数据稀缺和昂贵问题的一种方法是使用迁移学习和微调已经在大数据集上训练过的网络。迁移学习中的常见做法是保持浅层不变，根据新数据集修改更深层。由于外观差异很大，当从不同领域转移到超声 (US) 图像时，这种方法可能不适用于 U-Net。在这项研究中，我们研究了微调预训练 U-Net 的不同层集对 US 图像分割的影响。根据浅层和深层的两种不同定义，分析了两种不同的方案。我们研究了模拟 US 图像以及两个人类 US 数据集。我们还包括了一个胸部 X 射线数据集。结果表明，选择要微调的层是一项关键任务。特别是，它们表明，对于分类网络而言，微调网络的最后几层通常是最糟糕的策略。因此，在使用 U-Net 进行 US 图像分割时，微调浅层而不是深层可能更合适。浅层学习低级特征，这对于医学图像的自动分割至关重要。即使有大型 US 数据集可用，我们也观察到与微调整个网络相比，微调浅层是一种更快的方法。
