Hyvärinen Aapo, Khemakhem Ilyes, Morioka Hiroshi
Department of Computer Science, University of Helsinki, Helsinki, Finland.
Gatsby Computational Neuroscience Unit, University College London, London, UK.
Patterns (N Y). 2023 Oct 13;4(10):100844. doi: 10.1016/j.patter.2023.100844.
A central problem in unsupervised deep learning is how to find useful representations of high-dimensional data, sometimes called "disentanglement." Most approaches are heuristic and lack a proper theoretical foundation. In linear representation learning, independent component analysis (ICA) has been successful in many applications areas, and it is principled, i.e., based on a well-defined probabilistic model. However, extension of ICA to the nonlinear case has been problematic because of the lack of identifiability, i.e., uniqueness of the representation. Recently, nonlinear extensions that utilize temporal structure or some auxiliary information have been proposed. Such models are in fact identifiable, and consequently, an increasing number of algorithms have been developed. In particular, some self-supervised algorithms can be shown to estimate nonlinear ICA, even though they have initially been proposed from heuristic perspectives. This paper reviews the state of the art of nonlinear ICA theory and algorithms.
无监督深度学习中的一个核心问题是如何找到高维数据的有用表示，有时也称为“解纠缠”。大多数方法都是启发式的，缺乏适当的理论基础。在线性表示学习中，独立成分分析（ICA）在许多应用领域都取得了成功，并且它是有原则的，即基于一个定义明确的概率模型。然而，由于缺乏可识别性，即表示的唯一性，将ICA扩展到非线性情况一直存在问题。最近，已经提出了利用时间结构或一些辅助信息的非线性扩展。这样的模型实际上是可识别的，因此，已经开发了越来越多的算法。特别是，一些自监督算法可以被证明能够估计非线性ICA，尽管它们最初是从启发式的角度提出的。本文综述了非线性ICA理论和算法的现状。
