School of Computer and Systems Sciences, Jawaharlal Nehru University, New Delhi 110067, India.
Neural Netw. 2014 Mar;51:67-79. doi: 10.1016/j.neunet.2013.12.003. Epub 2013 Dec 11.
In this paper, a simple reformulation of the Lagrangian dual of the 2-norm support vector regression (SVR) is proposed as an unconstrained minimization problem. This formulation has the advantage that its objective function is strongly convex and further having only m variables, where m is the number of input data points. The proposed unconstrained Lagrangian SVR (ULSVR) is solvable by computing the zeros of its gradient. However, since its objective function contains the non-smooth 'plus' function, two approaches are followed to solve the proposed optimization problem: (i) by introducing a smooth approximation, generate a slightly modified unconstrained minimization problem and solve it; (ii) solve the problem directly by applying generalized derivative. Computational results obtained on a number of synthetic and real-world benchmark datasets showing similar generalization performance with much faster learning speed in accordance with the conventional SVR and training time very close to least squares SVR clearly indicate the superiority of ULSVR solved by smooth and generalized derivative approaches.
本文提出了一种 2-范数支持向量回归（SVR）拉格朗日对偶的简单重构，作为无约束最小化问题。这种表示形式的优点是其目标函数具有强凸性，并且只有 m 个变量，其中 m 是输入数据点的数量。所提出的无约束拉格朗日 SVR（ULSVR）可以通过计算其梯度的零点来求解。然而，由于其目标函数包含非光滑的“加号”函数，因此有两种方法可以解决所提出的优化问题：（i）通过引入平滑逼近，生成一个略有修改的无约束最小化问题并求解；（ii）通过应用广义导数直接求解问题。在许多合成和真实基准数据集上获得的计算结果表明，与传统的 SVR 相比，学习速度更快，泛化性能相似，并且与最小二乘 SVR 的训练时间非常接近，这清楚地表明了通过平滑和广义导数方法求解的 ULSVR 的优越性。
