Shenzhen Key Laboratory of Smart Healthcare Engineering, Department of Biomedical Engineering, Southern University of Science and Technology, Shenzhen 518055, China; College of Mathematics and Statistics, Chongqing Jiaotong University, Chongqing 400074, China.
Center for Brain Inspired Computing Research, Department of Precision Instrument, Tsinghua University, Beijing 100084, China.
Neural Netw. 2022 Jan;145:199-208. doi: 10.1016/j.neunet.2021.10.020. Epub 2021 Oct 28.
Variational autoencoders (VAEs) are influential generative models with rich representation capabilities from the deep neural network architecture and Bayesian method. However, VAE models have a weakness that assign a higher likelihood to out-of-distribution (OOD) inputs than in-distribution (ID) inputs. To address this problem, a reliable uncertainty estimation is considered to be critical for in-depth understanding of OOD inputs. In this study, we propose an improved noise contrastive prior (INCP) to be able to integrate into the encoder of VAEs, called INCPVAE. INCP is scalable, trainable and compatible with VAEs, and it also adopts the merits from the INCP for uncertainty estimation. Experiments on various datasets demonstrate that compared to the standard VAEs, our model is superior in uncertainty estimation for the OOD data and is robust in anomaly detection tasks. The INCPVAE model obtains reliable uncertainty estimation for OOD inputs and solves the OOD problem in VAE models.
变分自编码器（VAEs）是一种具有影响力的生成模型，它结合了深度学习神经网络架构和贝叶斯方法，具有丰富的表示能力。然而，VAE 模型有一个弱点，即对离群（OOD）输入的可能性分配高于在分布（ID）输入。为了解决这个问题，可靠的不确定性估计被认为是深入了解 OOD 输入的关键。在这项研究中，我们提出了一种改进的噪声对比先验（INCP），以便能够集成到 VAEs 的编码器中，称为 INCPVAE。INCP 具有可扩展性、可训练性和与 VAEs 的兼容性，它还采用了 INCP 用于不确定性估计的优点。在各种数据集上的实验表明，与标准 VAEs 相比，我们的模型在 OOD 数据的不确定性估计方面更优，并且在异常检测任务中具有鲁棒性。INCPVAE 模型对 OOD 输入进行了可靠的不确定性估计，并解决了 VAE 模型中的 OOD 问题。
