Electrical and Computer Engineering, University of Alberta, Edmonton, AB, Canada.
The Edward S. Rogers Department of Electrical and Computer Engineering, University of Toronto, Toronto, ON, Canada.
PLoS One. 2020 Oct 14;15(10):e0240530. doi: 10.1371/journal.pone.0240530. eCollection 2020.
Deep learning has achieved a great success in natural image classification. To overcome data-scarcity in computational pathology, recent studies exploit transfer learning to reuse knowledge gained from natural images in pathology image analysis, aiming to build effective pathology image diagnosis models. Since transferability of knowledge heavily depends on the similarity of the original and target tasks, significant differences in image content and statistics between pathology images and natural images raise the questions: how much knowledge is transferable? Is the transferred information equally contributed by pre-trained layers? If not, is there a sweet spot in transfer learning that balances transferred model's complexity and performance? To answer these questions, this paper proposes a framework to quantify knowledge gain by a particular layer, conducts an empirical investigation in pathology image centered transfer learning, and reports some interesting observations. Particularly, compared to the performance baseline obtained by a random-weight model, though transferability of off-the-shelf representations from deep layers heavily depend on specific pathology image sets, the general representation generated by early layers does convey transferred knowledge in various image classification applications. The trade-off between transferable performance and transferred model's complexity observed in this study encourages further investigation of specific metric and tools to quantify effectiveness of transfer learning in future.
深度学习在自然图像分类方面取得了巨大的成功。为了克服计算病理学中的数据匮乏问题，最近的研究利用迁移学习来重复利用从自然图像中获得的知识在病理学图像分析中，旨在构建有效的病理学图像诊断模型。由于知识的可转移性在很大程度上取决于原始任务和目标任务的相似性，因此病理学图像和自然图像之间在图像内容和统计方面存在显著差异，这引发了以下问题：有多少知识是可转移的？预训练层对转移信息的贡献是否相同？如果不是，在迁移学习中是否存在一个平衡点，既能平衡转移模型的复杂性和性能？为了回答这些问题，本文提出了一种框架来量化特定层的知识增益，并在以病理学图像为中心的迁移学习中进行了实证研究，并报告了一些有趣的观察结果。特别是，与通过随机权重模型获得的性能基线相比，尽管来自深层的现成表示的可转移性严重依赖于特定的病理学图像集，但早期层生成的一般表示确实在各种图像分类应用中传递了转移知识。本研究中观察到的可转移性性能和转移模型复杂性之间的权衡关系鼓励未来进一步研究特定的指标和工具来量化迁移学习的有效性。
