Australian Artificial Intelligence Institute, University of Technology Sydney NSW 2007, Australia
Neural Comput. 2020 Oct;32(10):1936-1979. doi: 10.1162/neco_a_01309. Epub 2020 Aug 14.
Multiview alignment, achieving one-to-one correspondence of multiview inputs, is critical in many real-world multiview applications, especially for cross-view data analysis problems. An increasing amount of work has studied this alignment problem with canonical correlation analysis (CCA). However, existing CCA models are prone to misalign the multiple views due to either the neglect of uncertainty or the inconsistent encoding of the multiple views. To tackle these two issues, this letter studies multiview alignment from a Bayesian perspective. Delving into the impairments of inconsistent encodings, we propose to recover correspondence of the multiview inputs by matching the marginalization of the joint distribution of multiview random variables under different forms of factorization. To realize our design, we present adversarial CCA (ACCA), which achieves consistent latent encodings by matching the marginalized latent encodings through the adversarial training paradigm. Our analysis, based on conditional mutual information, reveals that ACCA is flexible for handling implicit distributions. Extensive experiments on correlation analysis and cross-view generation under noisy input settings demonstrate the superiority of our model.
多视图对齐，实现多视图输入的一一对应，在许多现实世界的多视图应用中至关重要，特别是对于跨视图数据分析问题。越来越多的工作使用典型相关分析（CCA）研究了这个对齐问题。然而，现有的 CCA 模型由于忽略不确定性或多视图的不一致编码，容易导致多视图的错位。为了解决这两个问题，本信从贝叶斯的角度研究了多视图对齐。深入研究不一致编码的缺陷，我们提出通过匹配多视图随机变量联合分布在不同因子分解形式下的边缘化来恢复多视图输入的对应关系。为了实现我们的设计，我们提出了对抗性 CCA（ACCA），通过对抗训练范式匹配边缘化的潜在编码来实现一致的潜在编码。我们基于条件互信息的分析表明，ACCA 灵活适用于处理隐式分布。在噪声输入设置下进行相关性分析和跨视图生成的广泛实验证明了我们模型的优越性。
