Ruan Yibang, Xiao Yanshan, Hao Zhifeng, Liu Bo
IEEE Trans Neural Netw Learn Syst. 2022 Aug;33(8):3533-3546. doi: 10.1109/TNNLS.2021.3053266. Epub 2022 Aug 3.
Distance metric learning (DML) aims to learn a distance metric to process the data distribution. However, most of the existing methods are k NN DML methods and employ the k NN model to classify the test instances. The drawback of k NN DML is that all training instances need to be accessed and stored to classify the test instances, and the classification performance is influenced by the setting of the nearest neighbor number k . To solve these problems, there are several DML methods that employ the SVM model to classify the test instances. However, all of them are nonconvex and the convex support vector DML method has not been explicitly proposed. In this article, we propose a convex model for support vector DML (CSV-DML), which is capable of replacing the k NN model of DML with the SVM model. To make CSV-DML can use the most kernel functions of the existing SVM methods, a nonlinear mapping is used to map the original instances into a feature space. Since the explicit form of nonlinear mapped instances is unknown, the original instances are further transformed into the kernel form, which can be calculated explicitly. CSV-DML is constructed to work directly on the kernel-transformed instances. Specifically, we learn a specific Mahalanobis distance metric from the kernel-transformed training instances and train a DML-based separating hyperplane based on it. An iterated approach is formulated to optimize CSV-DML, which is based on generalized block coordinate descent and can converge to the global optimum. In CSV-DML, since the dimension of kernel-transformed instances is only related to the number of original training instances, we develop a novel parameter reduction scheme for reducing the feature dimension. Extensive experiments show that the proposed CSV-DML method outperforms the previous methods.
距离度量学习（DML）旨在学习一种距离度量来处理数据分布。然而，现有的大多数方法都是k近邻DML方法，并使用k近邻模型对测试实例进行分类。k近邻DML的缺点是，为了对测试实例进行分类，需要访问和存储所有训练实例，并且分类性能受最近邻数量k设置的影响。为了解决这些问题，有几种DML方法使用支持向量机（SVM）模型对测试实例进行分类。然而，它们都是非凸的，并且尚未明确提出凸支持向量DML方法。在本文中，我们提出了一种用于支持向量DML的凸模型（CSV-DML），它能够用SVM模型替代DML的k近邻模型。为了使CSV-DML能够使用现有SVM方法的大多数核函数，使用非线性映射将原始实例映射到特征空间。由于非线性映射实例的显式形式未知，将原始实例进一步转换为核形式，其可以显式计算。CSV-DML被构建为直接对核变换后的实例进行操作。具体而言，我们从核变换后的训练实例中学习一种特定的马氏距离度量，并基于此训练一个基于DML的分离超平面。制定了一种迭代方法来优化CSV-DML，该方法基于广义块坐标下降，并且可以收敛到全局最优解。在CSV-DML中，由于核变换后实例的维度仅与原始训练实例的数量有关，我们开发了一种新颖的参数约简方案来降低特征维度。大量实验表明所提出的CSV-DML方法优于先前的方法。
