Nenadic Zoran
Department of Biomedical Engineering, University of California, Irvine, 3120 Natural Sciences II, Irvine, CA 92697-2715, USA.
IEEE Trans Pattern Anal Mach Intell. 2007 Aug;29(8):1394-407. doi: 10.1109/TPAMI.2007.1156.
Using elementary information-theoretic tools, we develop a novel technique for linear transformation from the space of observations into a low-dimensional (feature) subspace for the purpose of classification. The technique is based on a numerical optimization of an information-theoretic objective function, which can be computed analytically. The advantages of the proposed method over several other techniques are discussed and the conditions under which the method reduces to linear discriminant analysis are given. We show that the novel objective function enjoys many of the properties of the mutual information and the Bayes error and we give sufficient conditions for the method to be Bayes-optimal. Since the objective function is maximized numerically, we show how the calculations can be accelerated to yield feasible solutions. The performance of the method compares favorably to other linear discriminant-based feature extraction methods on a number of simulated and real-world data sets.
利用基本的信息论工具，我们开发了一种新颖的技术，用于将观测空间线性变换到低维（特征）子空间以进行分类。该技术基于对一个信息论目标函数的数值优化，该目标函数可以通过解析计算得到。讨论了所提出方法相对于其他几种技术的优势，并给出了该方法简化为线性判别分析的条件。我们表明，这个新颖的目标函数具有互信息和贝叶斯误差的许多性质，并给出了该方法成为贝叶斯最优的充分条件。由于目标函数是通过数值最大化的，我们展示了如何加速计算以产生可行解。在一些模拟和真实世界的数据集上，该方法的性能优于其他基于线性判别的特征提取方法。
