Van Hulle Marc M
KU Leuven, Laboratorium voor Neuro- en Psychofysiologie, Belgium.
Neural Netw. 2002 Oct-Nov;15(8-9):1029-39. doi: 10.1016/s0893-6080(02)00077-1.
A new information-theoretic learning algorithm is introduced for kernel-based topographic map formation. The kernels are allowed to overlap and move freely in the input space, and to have differing kernel ranges. We start with Linsker's infomax principle and observe that it cannot be readily extended to our case, exactly due to the presence of kernels. We then consider Bell and Sejnowski's generalization of Linsker's infomax principle, which suggests differential entropy maximization, and add a second component to be optimized, namely, mutual information minimization between the kernel outputs, in order to take into account the kernel overlap, and thus the topographic map's output redundancy. The result is joint entropy maximization of the kernel outputs, which we adopt as our learning criterion. We derive a learning algorithm and verify its performance both for a synthetic example, for which the optimal result can be derived analytically, and for a classic real-world example.
本文介绍了一种用于基于核的地形图形成的新信息论学习算法。允许核在输入空间中重叠并自由移动，且具有不同的核范围。我们从林斯克的信息最大化原理出发，发现由于核的存在，该原理不能直接扩展到我们的情况。然后我们考虑贝尔和塞乔夫斯基对林斯克信息最大化原理的推广，该推广建议最大化微分熵，并添加第二个要优化的分量，即核输出之间的互信息最小化，以考虑核的重叠，从而考虑地形图输出的冗余性。结果是核输出的联合熵最大化，我们将其作为学习准则。我们推导了一种学习算法，并针对一个合成示例（其最优结果可通过解析得出）和一个经典的实际示例验证了其性能。
