SENIOR MEMBER, IEEE, Information Technology Division, Naval Research Laboratory, Washington, DC 20375.
IEEE Trans Pattern Anal Mach Intell. 1982 Jan;4(1):11-7. doi: 10.1109/tpami.1982.4767189.
This paper considers the problem of classifying an input vector of measurements by a nearest neighbor rule applied to a fixed set of vectors. The fixed vectors are sometimes called characteristic feature vectors, codewords, cluster centers, models, reproductions, etc. The nearest neighbor rule considered uses a non-Euclidean information-theoretic distortion measure that is not a metric, but that nevertheless leads to a classification method that is optimal in a well-defined sense and is also computationally attractive. Furthermore, the distortion measure results in a simple method of computing cluster centroids. Our approach is based on the minimization of cross-entropy (also called discrimination information, directed divergence, K-L number), and can be viewed as a refinement of a general classification method due to Kullback. The refinement exploits special properties of cross-entropy that hold when the probability densities involved happen to be minimum cross-entropy densities. The approach is a generalization of a recently developed speech coding technique called speech coding by vector quantization.
本文考虑了通过应用于固定向量集的最近邻规则对输入测量向量进行分类的问题。这些固定向量有时被称为特征向量、码字、聚类中心、模型、复制品等。所考虑的最近邻规则使用的是非欧几里得信息论失真度量，它不是度量，但它导致了一种在明确定义的意义上是最优的分类方法，并且在计算上也很有吸引力。此外，失真度量还导致了一种计算聚类中心的简单方法。我们的方法基于交叉熵（也称为鉴别信息、有向散度、K-L 数）的最小化，可以看作是 Kullback 的一种通用分类方法的改进。这种改进利用了交叉熵的特殊性质，当所涉及的概率密度恰好是最小交叉熵密度时，这些性质就成立。这种方法是最近开发的一种语音编码技术的推广，称为矢量量化语音编码。
