Instituto de Engenharia Biomédica, Divisão de Sinal e Imagem, Porto, Portugal.
Neural Comput. 2010 Oct;22(10):2698-728. doi: 10.1162/NECO_a_00013.
This letter focuses on the issue of whether risk functionals derived from information-theoretic principles, such as Shannon or Rényi's entropies, are able to cope with the data classification problem in both the sense of attaining the risk functional minimum and implying the minimum probability of error allowed by the family of functions implemented by the classifier, here denoted by min Pe. The analysis of this so-called minimization of error entropy (MEE) principle is carried out in a single perceptron with continuous activation functions, yielding continuous error distributions. In spite of the fact that the analysis is restricted to single perceptrons, it reveals a large spectrum of behaviors that MEE can be expected to exhibit in both theory and practice. In what concerns the theoretical MEE, our study clarifies the role of the parameters controlling the perceptron activation function (of the squashing type) in often reaching the minimum probability of error. Our study also clarifies the role of the kernel density estimator of the error density in achieving the minimum probability of error in practice.
这封信主要关注的问题是，基于信息论原理（如香农熵或雷恩熵）推导出的风险泛函，是否能够在实现风险泛函最小化和隐含分类器实现的函数族允许的最小错误概率（这里表示为 min Pe）这两个方面，解决数据分类问题。该所谓的最小错误熵（MEE）原理的分析是在具有连续激活函数的单个感知机中进行的，产生连续的错误分布。尽管分析仅限于单个感知机，但它揭示了 MEE 在理论和实践中可能表现出的大量行为。在理论 MEE 方面，我们的研究阐明了控制感知机激活函数（挤压型）的参数在经常达到最小错误概率方面的作用。我们的研究还阐明了错误密度核密度估计器在实际中达到最小错误概率的作用。
