Afridi Muhammad Jamal, Ross Arun, Shapiro Erik M
Department of Computer Science and Engineering, Michigan State University.
Department of Radiology, Michigan State University.
Pattern Recognit. 2018 Jan;73:65-75. doi: 10.1016/j.patcog.2017.07.019. Epub 2017 Jul 26.
Transfer learning, or inductive transfer, refers to the transfer of knowledge from a source task to a target task. In the context of convolutional neural networks (CNNs), transfer learning can be implemented by transplanting the learned feature layers from one CNN (derived from the source task) to initialize another (for the target task). Previous research has shown that the choice of the source CNN impacts the performance of the target task. In the current literature, there is no principled way for selecting a source CNN for a given target task despite the increasing availability of pre-trained source CNNs. In this paper we investigate the possibility of automatically ranking source CNNs prior to utilizing them for a target task. In particular, we present an information theoretic framework to understand the source-target relationship and use this as a basis to derive an approach to automatically rank source CNNs in an efficient, zero-shot manner. The practical utility of the approach is thoroughly evaluated using the PlacesMIT dataset, MNIST dataset and a real-world MRI database. Experimental results demonstrate the efficacy of the proposed ranking method for transfer learning.
迁移学习，或称归纳迁移，是指知识从源任务向目标任务的转移。在卷积神经网络（CNN）的背景下，迁移学习可以通过移植一个CNN（源自源任务）中学习到的特征层来初始化另一个CNN（用于目标任务）来实现。先前的研究表明，源CNN的选择会影响目标任务的性能。在当前文献中，尽管预训练源CNN的可用性不断增加，但对于给定的目标任务，尚无选择源CNN的原则性方法。在本文中，我们研究了在将源CNN用于目标任务之前自动对其进行排序的可能性。具体而言，我们提出了一个信息论框架来理解源-目标关系，并以此为基础推导出一种以高效、零样本方式自动对源CNN进行排序的方法。使用PlacesMIT数据集、MNIST数据集和一个真实世界的MRI数据库对该方法的实际效用进行了全面评估。实验结果证明了所提出的迁移学习排序方法的有效性。
