Fang Zhiying, Mao Tong, Fan Jun
Institute of Applied Mathematics, Shenzhen Polytechnic University, Shenzhen, Guangdong, China
Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology, Thuwal 4700, Kingdom of Saudi Arabia
Neural Comput. 2024 Mar 21;36(4):718-743. doi: 10.1162/neco_a_01650.
Combining information-theoretic learning with deep learning has gained significant attention in recent years, as it offers a promising approach to tackle the challenges posed by big data. However, the theoretical understanding of convolutional structures, which are vital to many structured deep learning models, remains incomplete. To partially bridge this gap, this letter aims to develop generalization analysis for deep convolutional neural network (CNN) algorithms using learning theory. Specifically, we focus on investigating robust regression using correntropy-induced loss functions derived from information-theoretic learning. Our analysis demonstrates an explicit convergence rate for deep CNN-based robust regression algorithms when the target function resides in the Korobov space. This study sheds light on the theoretical underpinnings of CNNs and provides a framework for understanding their performance and limitations.
近年来，将信息论学习与深度学习相结合受到了广泛关注，因为它为应对大数据带来的挑战提供了一种很有前景的方法。然而，对于许多结构化深度学习模型至关重要的卷积结构，其理论理解仍不完整。为了部分弥补这一差距，本文旨在利用学习理论对深度卷积神经网络（CNN）算法进行泛化分析。具体而言，我们专注于研究使用从信息论学习中导出的核相关损失函数进行稳健回归。我们的分析表明，当目标函数位于科罗布夫空间时，基于深度CNN的稳健回归算法具有明确的收敛速度。这项研究揭示了CNN的理论基础，并为理解其性能和局限性提供了一个框架。
