Zhang Xiang, Wu Yichao, Wang Lan, Li Runze
Department of Statistics, North Carolina State University, Raleigh, NC 27695, USA.
Department of Statistics, The University of Minnesota, Minneapolis, MN 55455, USA.
J Mach Learn Res. 2016;17(16):1-26.
Information criteria have been popularly used in model selection and proved to possess nice theoretical properties. For classification, Claeskens et al. (2008) proposed support vector machine information criterion for feature selection and provided encouraging numerical evidence. Yet no theoretical justification was given there. This work aims to fill the gap and to provide some theoretical justifications for support vector machine information criterion in both fixed and diverging model spaces. We first derive a uniform convergence rate for the support vector machine solution and then show that a modification of the support vector machine information criterion achieves model selection consistency even when the number of features diverges at an exponential rate of the sample size. This consistency result can be further applied to selecting the optimal tuning parameter for various penalized support vector machine methods. Finite-sample performance of the proposed information criterion is investigated using Monte Carlo studies and one real-world gene selection problem.
信息准则在模型选择中已被广泛使用，并被证明具有良好的理论性质。对于分类问题，Claeskens等人（2008年）提出了用于特征选择的支持向量机信息准则，并给出了令人鼓舞的数值证据。然而，那里没有给出理论依据。这项工作旨在填补这一空白，并为固定模型空间和发散模型空间中的支持向量机信息准则提供一些理论依据。我们首先推导了支持向量机解的一致收敛速度，然后表明，即使特征数量以样本量的指数速率发散，支持向量机信息准则的一种修正也能实现模型选择一致性。这一一致性结果可以进一步应用于为各种惩罚支持向量机方法选择最优调谐参数。使用蒙特卡罗研究和一个实际的基因选择问题研究了所提出的信息准则的有限样本性能。
