School of Economics and Management, University of Chinese Academy of Sciences, Beijing 100190, China; Research Center on Fictitious Economy and Data Science, Chinese Academy of Sciences, Beijing 100190, China; Key Laboratory of Big Data Mining and Knowledge Management, Chinese Academy of Sciences, Beijing 100190, China.
School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China.
Neural Netw. 2022 Feb;146:69-84. doi: 10.1016/j.neunet.2021.11.002. Epub 2021 Nov 15.
Multi-view learning aims to fully exploit the view-consistency and view-discrepancy for performance improvement. Knowledge Distillation (KD), characterized by the so-called "Teacher-Student" (T-S) learning framework, can transfer information learned from one model to another. Inspired by knowledge distillation, we propose a Multi-view Teacher-Student Network (MTS-Net), which combines knowledge distillation and multi-view learning into a unified framework. We first redefine the teacher and student for the multi-view case. Then the MTS-Net is built by optimizing both the view classification loss and the knowledge distillation loss in an end-to-end training manner. We further extend MTS-Net to image recognition tasks and present a multi-view Teacher-Student framework with convolutional neural networks called MTSCNN. To the best of our knowledge, MTS-Net and MTSCNN bring a new insight to extend the Teacher-Student framework to tackle the multi-view learning problem. We theoretically verify the mechanism of MTS-Net and MTSCNN and comprehensive experiments demonstrate the effectiveness of the proposed methods.
多视图学习旨在充分利用视图一致性和视图差异来提高性能。知识蒸馏（Knowledge Distillation，KD）以所谓的“教师-学生”（Teacher-Student，T-S）学习框架为特征，可以将从一个模型中学到的信息转移到另一个模型中。受知识蒸馏的启发，我们提出了一种多视图教师-学生网络（Multi-view Teacher-Student Network，MTS-Net），它将知识蒸馏和多视图学习结合到一个统一的框架中。我们首先重新定义了多视图情况下的教师和学生。然后，通过端到端的训练方式，同时优化视图分类损失和知识蒸馏损失来构建 MTS-Net。我们进一步将 MTS-Net 扩展到图像识别任务，并提出了一种基于卷积神经网络的多视图教师-学生框架，称为 MTSCNN。据我们所知，MTS-Net 和 MTSCNN 为将教师-学生框架扩展到解决多视图学习问题提供了新的思路。我们从理论上验证了 MTS-Net 和 MTSCNN 的机制，并通过全面的实验证明了所提出方法的有效性。
