Department of Mathematics, Natural and Computer Sciences, University of Applied Sciences Mittweida, 09648 Mittweida, Germany.
Neural Comput. 2011 May;23(5):1343-92. doi: 10.1162/NECO_a_00110. Epub 2011 Feb 7.
Supervised and unsupervised vector quantization methods for classification and clustering traditionally use dissimilarities, frequently taken as Euclidean distances. In this article, we investigate the applicability of divergences instead, focusing on online learning. We deduce the mathematical fundamentals for its utilization in gradient-based online vector quantization algorithms. It bears on the generalized derivatives of the divergences known as Fréchet derivatives in functional analysis, which reduces in finite-dimensional problems to partial derivatives in a natural way. We demonstrate the application of this methodology for widely applied supervised and unsupervised online vector quantization schemes, including self-organizing maps, neural gas, and learning vector quantization. Additionally, principles for hyperparameter optimization and relevance learning for parameterized divergences in the case of supervised vector quantization are given to achieve improved classification accuracy.
监督和无监督的矢量量化方法传统上用于分类和聚类，通常使用相似度，通常采用欧几里得距离。在本文中，我们研究了发散的适用性，重点是在线学习。我们推导出了在基于梯度的在线矢量量化算法中利用它的数学基础。它涉及到发散的广义导数，在泛函分析中称为弗雷歇导数，在有限维问题中，它自然地简化为偏导数。我们展示了该方法在广泛应用的监督和无监督在线矢量量化方案中的应用，包括自组织映射、神经气体和学习矢量量化。此外，还给出了在监督矢量量化情况下，针对参数化发散的超参数优化和相关性学习的原则，以提高分类准确性。
