Fischer Ian
Google Research, Mountain View, CA 94043, USA.
Entropy (Basel). 2020 Sep 8;22(9):999. doi: 10.3390/e22090999.
Much of the field of Machine Learning exhibits a prominent set of failure modes, including vulnerability to adversarial examples, poor out-of-distribution (OoD) detection, miscalibration, and willingness to memorize random labelings of datasets. We characterize these as failures of , which extends the traditional measure of generalization as accuracy or related metrics on a held-out set. We hypothesize that these failures to robustly generalize are due to the learning systems retaining information about the training data. To test this hypothesis, we propose the (MNI) criterion for evaluating the quality of a model. In order to train models that perform well with respect to the MNI criterion, we present a new objective function, the (CEB), which is closely related to the (IB). We experimentally test our hypothesis by comparing the performance of CEB models with deterministic models and Variational Information Bottleneck (VIB) models on a variety of different datasets and robustness challenges. We find strong empirical evidence supporting our hypothesis that MNI models improve on these problems of robust generalization.
机器学习领域的许多方面都呈现出一系列突出的失败模式，包括易受对抗样本攻击、分布外（OoD）检测能力差、校准错误以及倾向于记住数据集的随机标注。我们将这些特征化为 的失败，它扩展了传统的泛化度量，即作为在留出集上的准确率或相关指标。我们假设这些未能稳健泛化的情况是由于学习系统保留了有关训练数据的信息。为了检验这一假设，我们提出了用于评估模型质量的 （MNI）准则。为了训练在MNI准则方面表现良好的模型，我们提出了一个新的目标函数，即 （CEB），它与 （IB）密切相关。我们通过在各种不同的数据集和鲁棒性挑战上比较CEB模型与确定性模型以及变分信息瓶颈（VIB）模型的性能，对我们的假设进行了实验测试。我们发现有力的实证证据支持我们的假设，即MNI模型在这些稳健泛化问题上有所改进。
