Kunkel Lea, Trabs Mathias
Karlsruhe Institute of Technology, Department of Mathematics, Germany.
Neural Netw. 2025 Jan;181:106770. doi: 10.1016/j.neunet.2024.106770. Epub 2024 Oct 6.
The empirical success of Generative Adversarial Networks (GANs) caused an increasing interest in theoretical research. The statistical literature is mainly focused on Wasserstein GANs and generalizations thereof, which especially allow for good dimension reduction properties. Statistical results for Vanilla GANs, the original optimization problem, are still rather limited and require assumptions such as smooth activation functions and equal dimensions of the latent space and the ambient space. To bridge this gap, we draw a connection from Vanilla GANs to the Wasserstein distance. By doing so, existing results for Wasserstein GANs can be extended to Vanilla GANs. In particular, we obtain an oracle inequality for Vanilla GANs in Wasserstein distance. The assumptions of this oracle inequality are designed to be satisfied by network architectures commonly used in practice, such as feedforward ReLU networks. By providing a quantitative result for the approximation of a Lipschitz function by a feedforward ReLU network with bounded Hölder norm, we conclude a rate of convergence for Vanilla GANs as well as Wasserstein GANs as estimators of the unknown probability distribution.
生成对抗网络（GAN）在实际应用中的成功引发了对其理论研究的浓厚兴趣。统计文献主要聚焦于瓦瑟斯坦生成对抗网络（WGAN）及其推广形式，这类网络尤其具有良好的降维特性。对于原始的优化问题——香草生成对抗网络（Vanilla GAN），其统计结果仍然相当有限，并且需要诸如平滑激活函数以及潜在空间和环境空间维度相等之类的假设。为了弥合这一差距，我们建立了香草生成对抗网络与瓦瑟斯坦距离之间的联系。通过这样做，瓦瑟斯坦生成对抗网络的现有结果可以扩展到香草生成对抗网络。特别地，我们得到了香草生成对抗网络在瓦瑟斯坦距离下的一个神谕不等式。这个神谕不等式的假设旨在被实际中常用的网络架构所满足，比如前馈ReLU网络。通过给出一个具有有界赫尔德范数的前馈ReLU网络对利普希茨函数逼近的定量结果，我们得出了香草生成对抗网络以及瓦瑟斯坦生成对抗网络作为未知概率分布估计器的收敛速率。
