Chen Badong, Zhu Yu, Hu Jinchun
Department of Precision Instruments and Mechanology, Institute of Manufacturing Engineering, Tsinghua University, Beijing, China. chenbd04@ mails.tsinghua.edu.cn
IEEE Trans Neural Netw. 2010 Jul;21(7):1168-79. doi: 10.1109/TNN.2010.2050212. Epub 2010 Jun 21.
Recently, the minimum error entropy (MEE) criterion has been used as an information theoretic alternative to traditional mean-square error criterion in supervised learning systems. MEE yields nonquadratic, nonconvex performance surface even for adaptive linear neuron (ADALINE) training, which complicates the theoretical analysis of the method. In this paper, we develop a unified approach for mean-square convergence analysis for ADALINE training under MEE criterion. The weight update equation is formulated in the form of block-data. Based on a block version of energy conservation relation, and under several assumptions, we carry out the mean-square convergence analysis of this class of adaptation algorithm, including mean-square stability, mean-square evolution (transient behavior) and the mean-square steady-state performance. Simulation experimental results agree with the theoretical predictions very well.
最近，最小误差熵（MEE）准则已被用作监督学习系统中传统均方误差准则的信息论替代方法。即使对于自适应线性神经元（ADALINE）训练，MEE也会产生非二次、非凸的性能曲面，这使得该方法的理论分析变得复杂。在本文中，我们开发了一种统一的方法，用于在MEE准则下对ADALINE训练进行均方收敛分析。权重更新方程以块数据的形式制定。基于能量守恒关系的块版本，并在几个假设下，我们对这类自适应算法进行了均方收敛分析，包括均方稳定性、均方演化（瞬态行为）和均方稳态性能。仿真实验结果与理论预测非常吻合。
