Ding Lizhong, Liao Shizhong, Liu Yong, Liu Li, Zhu Fan, Yao Yazhou, Shao Ling, Gao Xin
IEEE Trans Neural Netw Learn Syst. 2020 Nov;31(11):4881-4891. doi: 10.1109/TNNLS.2019.2958922. Epub 2020 Oct 29.
Kernel selection is of fundamental importance for the generalization of kernel methods. This article proposes an approximate approach for kernel selection by exploiting the approximability of kernel selection and the computational virtue of kernel matrix approximation. We define approximate consistency to measure the approximability of the kernel selection problem. Based on the analysis of approximate consistency, we solve the theoretical problem of whether, under what conditions, and at what speed, the approximate criterion is close to the accurate one, establishing the foundations of approximate kernel selection. We introduce two selection criteria based on error estimation and prove the approximate consistency of the multilevel circulant matrix (MCM) approximation and Nyström approximation under these criteria. Under the theoretical guarantees of the approximate consistency, we design approximate algorithms for kernel selection, which exploits the computational advantages of the MCM and Nyström approximations to conduct kernel selection in a linear or quasi-linear complexity. We experimentally validate the theoretical results for the approximate consistency and evaluate the effectiveness of the proposed kernel selection algorithms.
核函数选择对于核方法的泛化至关重要。本文通过利用核函数选择的可近似性和核矩阵近似的计算优势，提出了一种核函数选择的近似方法。我们定义近似一致性来衡量核函数选择问题的可近似性。基于对近似一致性的分析，我们解决了近似准则在何种条件下、以何种速度接近精确准则的理论问题，奠定了近似核函数选择的基础。我们引入了基于误差估计的两个选择准则，并证明了在这些准则下多级循环矩阵（MCM）近似和Nyström近似的近似一致性。在近似一致性的理论保证下，我们设计了核函数选择的近似算法，该算法利用MCM和Nyström近似的计算优势，以线性或准线性复杂度进行核函数选择。我们通过实验验证了近似一致性的理论结果，并评估了所提出的核函数选择算法的有效性。
