Sun Yan, Song Qifan, Liang Faming
Department of Statistics, Purdue University, West Lafayette, IN 47907, USA.
Stat Probab Lett. 2022 Jan;180. doi: 10.1016/j.spl.2021.109246. Epub 2021 Sep 24.
Deep learning has achieved great successes in many machine learning tasks. However, the deep neural networks (DNNs) are often severely over-parameterized, making them computationally expensive, memory intensive, less interpretable and mis-calibrated. We study sparse DNNs under the Bayesian framework: we establish posterior consistency and structure selection consistency for Bayesian DNNs with a spike-and-slab prior, and illustrate their performance using examples on high-dimensional nonlinear variable selection, large network compression and model calibration. Our numerical results indicate that sparsity is essential for improving the prediction accuracy and calibration of the DNN.
深度学习在许多机器学习任务中取得了巨大成功。然而，深度神经网络（DNN）往往参数严重过多，这使得它们计算成本高昂、内存密集、难以解释且校准错误。我们在贝叶斯框架下研究稀疏DNN：我们为具有尖峰和平板先验的贝叶斯DNN建立后验一致性和结构选择一致性，并通过高维非线性变量选择、大型网络压缩和模型校准的示例来说明它们的性能。我们的数值结果表明，稀疏性对于提高DNN的预测准确性和校准至关重要。
