Li Haifeng, Jiang Tao, Zhang Keshu
Department of Computer Science and Engineering, University of California, Riverside, CA 92521, USA.
IEEE Trans Neural Netw. 2006 Jan;17(1):157-65. doi: 10.1109/TNN.2005.860852.
In pattern recognition, feature extraction techniques are widely employed to reduce the dimensionality of data and to enhance the discriminatory information. Principal component analysis (PCA) and linear discriminant analysis (LDA) are the two most popular linear dimensionality reduction methods. However, PCA is not very effective for the extraction of the most discriminant features, and LDA is not stable due to the small sample size problem. In this paper, we propose some new (linear and nonlinear) feature extractors based on maximum margin criterion (MMC). Geometrically, feature extractors based on MMC maximize the (average) margin between classes after dimensionality reduction. It is shown that MMC can represent class separability better than PCA. As a connection to LDA, we may also derive LDA from MMC by incorporating some constraints. By using some other constraints, we establish a new linear feature extractor that does not suffer from the small sample size problem, which is known to cause serious stability problems for LDA. The kernelized (nonlinear) counterpart of this linear feature extractor is also established in the paper. Our extensive experiments demonstrate that the new feature extractors are effective, stable, and efficient.
在模式识别中，特征提取技术被广泛用于降低数据维度并增强判别信息。主成分分析（PCA）和线性判别分析（LDA）是两种最常用的线性降维方法。然而，PCA在提取最具判别力的特征方面效果不佳，而LDA由于小样本问题而不稳定。在本文中，我们基于最大间隔准则（MMC）提出了一些新的（线性和非线性）特征提取器。从几何角度来看，基于MMC的特征提取器在降维后最大化了类间（平均）间隔。结果表明，MMC比PCA能更好地表示类可分性。作为与LDA的联系，我们还可以通过纳入一些约束从MMC推导出LDA。通过使用其他一些约束，我们建立了一种新的线性特征提取器，它不会受到小样本问题的影响，而小样本问题已知会给LDA带来严重的稳定性问题。本文还建立了这种线性特征提取器的核化（非线性）对应物。我们广泛的实验表明，新的特征提取器是有效、稳定且高效的。
