Kim Jiae, Lee Yoonkyung, Liang Zhiyu
IEEE Trans Pattern Anal Mach Intell. 2023 Apr;45(4):5203-5217. doi: 10.1109/TPAMI.2022.3192726. Epub 2023 Mar 7.
Fisher's linear discriminant analysis is a classical method for classification, yet it is limited to capturing linear features only. Kernel discriminant analysis as an extension is known to successfully alleviate the limitation through a nonlinear feature mapping. We study the geometry of nonlinear embeddings in discriminant analysis with polynomial kernels and Gaussian kernel by identifying the population-level discriminant function that depends on the data distribution and the kernel. In order to obtain the discriminant function, we solve a generalized eigenvalue problem with between-class and within-class covariance operators. The polynomial discriminants are shown to capture the class difference through the population moments explicitly. For approximation of the Gaussian discriminant, we use a particular representation of the Gaussian kernel by utilizing the exponential generating function for Hermite polynomials. We also show that the Gaussian discriminant can be approximated using randomized projections of the data. Our results illuminate how the data distribution and the kernel interact in determination of the nonlinear embedding for discrimination, and provide a guideline for choice of the kernel and its parameters.
费希尔线性判别分析是一种经典的分类方法，但它仅限于捕捉线性特征。作为一种扩展，核判别分析通过非线性特征映射成功地缓解了这一限制。我们通过识别依赖于数据分布和核的总体水平判别函数，研究了多项式核和高斯核在判别分析中的非线性嵌入几何。为了获得判别函数，我们求解一个具有类间和类内协方差算子的广义特征值问题。多项式判别式通过总体矩明确地捕捉类差异。对于高斯判别的近似，我们利用埃尔米特多项式的指数生成函数对高斯核进行了特殊表示。我们还表明，可以使用数据的随机投影来近似高斯判别。我们的结果阐明了数据分布和核在确定用于判别的非线性嵌入时是如何相互作用的，并为核及其参数的选择提供了指导。
