Zheng Wenming
Research Center for Learning Science, Southeast University, Nanjing, Jiangsu 210096, China.
Neural Comput. 2006 Apr;18(4):979-1006. doi: 10.1162/089976606775774633.
Generalized discriminant analysis (GDA) is the nonlinear extension of the classical linear discriminant analysis (LDA) via the kernel trick. Mathematically, GDA aims to solve a generalized eigenequation problem, which is always implemented by the use of singular value decomposition (SVD) in the previously proposed GDA algorithms. A major drawback of SVD, however, is the difficulty of designing an incremental solution for the eigenvalue problem. Moreover, there are still numerical problems of computing the eigenvalue problem of large matrices. In this article, we propose another algorithm for solving GDA as for the case of small sample size problem, which applies QR decomposition rather than SVD. A major contribution of the proposed algorithm is that it can incrementally update the discriminant vectors when new classes are inserted into the training set. The other major contribution of this article is the presentation of the modified kernel Gram-Schmidt (MKGS) orthogonalization algorithm for implementing the QR decomposition in the feature space, which is more numerically stable than the kernel Gram-Schmidt (KGS) algorithm. We conduct experiments on both simulated and real data to demonstrate the better performance of the proposed methods.
广义判别分析（GDA）是通过核技巧对经典线性判别分析（LDA）的非线性扩展。在数学上，GDA旨在解决一个广义特征方程问题，在先前提出的GDA算法中，该问题总是通过奇异值分解（SVD）来实现。然而，SVD的一个主要缺点是难以设计特征值问题的增量解。此外，在计算大矩阵的特征值问题时仍然存在数值问题。在本文中，针对小样本规模问题的情况，我们提出了另一种求解GDA的算法，该算法应用QR分解而非SVD。所提算法的一个主要贡献在于，当新类别插入训练集时，它能够增量更新判别向量。本文的另一个主要贡献是提出了用于在特征空间中实现QR分解的修正核Gram - Schmidt（MKGS）正交化算法，该算法在数值上比核Gram - Schmidt（KGS）算法更稳定。我们在模拟数据和真实数据上都进行了实验，以证明所提方法具有更好的性能。
