IEEE Trans Cybern. 2021 May;51(5):2748-2760. doi: 10.1109/TCYB.2019.2923756. Epub 2021 Apr 15.
The training of autoencoder (AE) focuses on the selection of connection weights via a minimization of both the training error and a regularized term. However, the ultimate goal of AE training is to autoencode future unseen samples correctly (i.e., good generalization). Minimizing the training error with different regularized terms only indirectly minimizes the generalization error. Moreover, the trained model may not be robust to small perturbations of inputs which may lead to a poor generalization capability. In this paper, we propose a localized stochastic sensitive AE (LiSSA) to enhance the robustness of AE with respect to input perturbations. With the local stochastic sensitivity regularization, LiSSA reduces sensitivity to unseen samples with small differences (perturbations) from training samples. Meanwhile, LiSSA preserves the local connectivity from the original input space to the representation space that learns a more robustness features (intermediate representation) for unseen samples. The classifier using these learned features yields a better generalization capability. Extensive experimental results on 36 benchmarking datasets indicate that LiSSA outperforms several classical and recent AE training methods significantly on classification tasks.
自动编码器 (AE) 的训练重点是通过最小化训练误差和正则化项来选择连接权重。然而，AE 训练的最终目标是正确地自动编码未来未见过的样本（即良好的泛化能力）。通过不同的正则化项最小化训练误差只能间接地最小化泛化误差。此外，训练后的模型可能对输入的小扰动不稳健，这可能导致较差的泛化能力。在本文中，我们提出了一种局部随机敏感自动编码器 (LiSSA)，以增强 AE 对输入扰动的稳健性。通过局部随机敏感性正则化，LiSSA 降低了对来自训练样本的小差异（扰动）的未见样本的敏感性。同时，LiSSA 保留了原始输入空间到表示空间的局部连通性，从而为未见样本学习更稳健的特征（中间表示）。使用这些学习特征的分类器产生更好的泛化能力。在 36 个基准数据集上的广泛实验结果表明，LiSSA 在分类任务上明显优于几种经典和最近的 AE 训练方法。
