Kozyrev Sergei V, Lopatin Ilya A, Pechen Alexander N
Steklov Mathematical Institute of Russian Academy of Sciences, Gubkina St. 8, Moscow 119991, Russia.
Ivannikov Institute for System Programming of the Russian Academy of Sciences, Alexandra Solzhenitsyna Str. 25, Moscow 109004, Russia.
Entropy (Basel). 2024 Dec 13;26(12):1090. doi: 10.3390/e26121090.
While there are many works on the applications of machine learning, not so many of them are trying to understand the theoretical justifications to explain their efficiency. In this work, overfitting control (or generalization property) in machine learning is explained using analogies from physics and biology. For stochastic gradient Langevin dynamics, we show that the Eyring formula of kinetic theory allows to control overfitting in the algorithmic stability approach-when wide minima of the risk function with low free energy correspond to low overfitting. For the generative adversarial network (GAN) model, we establish an analogy between GAN and the predator-prey model in biology. An application of this analogy allows us to explain the selection of wide likelihood maxima and ab overfitting reduction for GANs.
虽然有许多关于机器学习应用的著作，但其中试图理解其效率理论依据的却不多。在这项工作中，我们使用物理学和生物学的类比来解释机器学习中的过拟合控制（或泛化特性）。对于随机梯度朗之万动力学，我们表明动力学理论的艾林公式允许在算法稳定性方法中控制过拟合——当具有低自由能的风险函数的宽最小值对应于低过拟合时。对于生成对抗网络（GAN）模型，我们在GAN和生物学中的捕食者 - 猎物模型之间建立了类比。这种类比的应用使我们能够解释GAN中宽似然最大值的选择和过拟合减少的问题。
