Wu Xiaohe, Zuo Wangmeng, Lin Liang, Jia Wei, Zhang David
IEEE Trans Neural Netw Learn Syst. 2018 Nov;29(11):5185-5199. doi: 10.1109/TNNLS.2018.2791507. Epub 2018 Feb 5.
The generalization error bound of the support vector machine (SVM) depends on the ratio of the radius and margin. However, conventional SVM only considers the maximization of the margin but ignores the minimization of the radius, which restricts its performance when applied to joint learning of feature transformation and the SVM classifier. Although several approaches have been proposed to integrate the radius and margin information, most of them either require the form of the transformation matrix to be diagonal, or are nonconvex and computationally expensive. In this paper, we suggest a novel approximation for the radius of the minimum enclosing ball in feature space, and then propose a convex radius-margin-based SVM model for joint learning of feature transformation and the SVM classifier, i.e., F-SVM. A generalized block coordinate descent method is adopted to solve the F-SVM model, where the feature transformation is updated via the gradient descent and the classifier is updated by employing the existing SVM solver. By incorporating with kernel principal component analysis, F-SVM is further extended for joint learning of nonlinear transformation and the classifier. F-SVM can also be incorporated with deep convolutional networks to improve image classification performance. Experiments on the UCI, LFW, MNIST, CIFAR-10, CIFAR-100, and Caltech101 data sets demonstrate the effectiveness of F-SVM.
支持向量机（SVM）的泛化误差界取决于半径与间隔的比率。然而，传统的SVM只考虑间隔最大化，却忽略了半径最小化，这限制了其在特征变换与SVM分类器联合学习中的应用性能。尽管已经提出了几种方法来整合半径和间隔信息，但大多数方法要么要求变换矩阵形式为对角矩阵，要么是非凸的且计算成本高昂。在本文中，我们针对特征空间中最小包围球的半径提出了一种新颖的近似方法，然后提出了一种基于半径 - 间隔的凸SVM模型，用于特征变换与SVM分类器的联合学习，即F - SVM。采用一种广义块坐标下降法来求解F - SVM模型，其中通过梯度下降更新特征变换，利用现有的SVM求解器更新分类器。通过结合核主成分分析，F - SVM进一步扩展用于非线性变换与分类器的联合学习。F - SVM还可以与深度卷积网络相结合以提高图像分类性能。在UCI、LFW、MNIST、CIFAR - 10、CIFAR - 100和Caltech101数据集上的实验证明了F - SVM的有效性。
