Business School, Shandong Normal University, Jinan 250358, China.
Faculty of Mathematics and Artificial Intelligence, Qilu University of Technology (Shandong Academy of Sciences), Jinan 250353, China.
Neural Netw. 2022 Nov;155:523-535. doi: 10.1016/j.neunet.2022.09.008. Epub 2022 Sep 13.
The L-regularized regression with Kullback-Leibler divergence (KL-LR) is a popular regression technique. Although many efforts have been devoted to its efficient implementation, it remains challenging when the number of features is extremely large. In this paper, to accelerate KL-LR, we introduce a novel and fast sequential safe feature elimination rule (FER) based on its sparsity, local regularity properties, and duality theory. It takes negligible time to select and delete most redundant features before and during the training process. Only one reduced model needs to be solved, which makes the computational time shortened. To further speed up the reduced model, the Newton coordinate descent method (Newton-CDM) is chosen as a solver. The superiority of FER is safety, i.e., its solution is exactly the same as the original KL-LR. Numerical experiments on three artificial datasets, five real-world datasets, and one handwritten digit dataset demonstrate the feasibility and validity of our FER.
L 正则化的 Kullback-Leibler 散度回归（KL-LR）是一种流行的回归技术。尽管已经做了很多努力来实现其高效的实现，但当特征数量极多时，它仍然具有挑战性。在本文中，为了加速 KL-LR，我们基于其稀疏性、局部正则性和对偶理论，引入了一种新颖的、快速的序列安全特征消除规则（FER）。它在训练过程之前和期间，只需花费很少的时间即可选择和删除大多数冗余特征。只需解决一个简化模型，这使得计算时间缩短了。为了进一步加快简化模型的速度，选择牛顿坐标下降法（Newton-CDM）作为求解器。FER 的优势在于其安全性，即其解决方案与原始 KL-LR 完全相同。在三个人工数据集、五个真实数据集和一个手写数字数据集上的数值实验证明了我们的 FER 的可行性和有效性。
