Qin Yalan, Qin Chuan, Zhang Xinpeng, Qi Donglian, Feng Guorui
IEEE Trans Image Process. 2023;32:175-189. doi: 10.1109/TIP.2022.3226408. Epub 2022 Dec 19.
Data in real world are usually characterized in multiple views, including different types of features or different modalities. Multi-view learning has been popular in the past decades and achieved significant improvements. In this paper, we investigate three challenging problems in the field of incomplete multi-view representation learning, namely, i) how to reduce the influences produced by missing views in multi-view dataset, ii) how to learn a consistent and informative representation among different views and iii) how to alleviate the impacts of the inherent noise in multi-view data caused by high-dimensional features or varied quality for different data points. To address these challenges, we integrate these three tasks into a problem and propose a novel framework termed Noise-aware Incomplete Multi-view Learning Networks (NIM-Nets). NIM-Nets fully utilize incomplete data from different views to produce a multi-view shared representation which is consistent, informative and robust to noise. We model the inherent noise in data by defining the distribution $\Gamma $ and assuming that each observation in the incomplete dataset is sampled from the distribution $\Gamma $ . To the best of our knowledge, this is the first work to unify learning the consistent and informative representation, alleviating the impacts of noise in data and handling the view-missing patterns in multi-view learning into a framework. We also first give a definition of robustness and completeness for incomplete multi-view representation learning. Based on NIM-Nets, we present joint optimization models for classification and clustering, respectively. Extensive experiments on different datasets demonstrate the effectiveness of our method over the existing work based on classification and clustering tasks in terms of different metrics.
现实世界中的数据通常具有多种视图特征，包括不同类型的特征或不同的模态。多视图学习在过去几十年中很流行，并取得了显著进展。在本文中，我们研究了不完全多视图表示学习领域中的三个具有挑战性的问题，即：i）如何减少多视图数据集中缺失视图产生的影响；ii）如何在不同视图之间学习一致且信息丰富的表示；iii）如何减轻由高维特征或不同数据点质量变化引起的多视图数据中固有噪声的影响。为了应对这些挑战，我们将这三个任务整合为一个问题，并提出了一种新颖的框架，称为噪声感知不完全多视图学习网络（NIM-Nets）。NIM-Nets充分利用来自不同视图的不完全数据来生成一个多视图共享表示，该表示一致、信息丰富且对噪声具有鲁棒性。我们通过定义分布$\Gamma$并假设不完全数据集中的每个观测值都从分布$\Gamma$中采样来对数据中的固有噪声进行建模。据我们所知，这是第一项将学习一致且信息丰富的表示、减轻数据中的噪声影响以及处理多视图学习中的视图缺失模式统一到一个框架中的工作。我们还首次给出了不完全多视图表示学习的鲁棒性和完整性的定义。基于NIM-Nets，我们分别提出了用于分类和聚类的联合优化模型。在不同数据集上进行的大量实验表明，在不同指标方面，我们的方法在基于分类和聚类任务的现有工作上更有效。
