Ju Cheng, Bibaut Aurélien, van der Laan Mark
University of California, Berkeley.
J Appl Stat. 2018;45(15):2800-2818. doi: 10.1080/02664763.2018.1441383. Epub 2018 Feb 26.
Artificial neural networks have been successfully applied to a variety of machine learning tasks, including image recognition, semantic segmentation, and machine translation. However, few studies fully investigated ensembles of artificial neural networks. In this work, we investigated multiple widely used ensemble methods, including unweighted averaging, majority voting, the Bayes Optimal Classifier, and the (discrete) Super Learner, for image recognition tasks, with deep neural networks as candidate algorithms. We designed several experiments, with the candidate algorithms being the same network structure with different model checkpoints within a single training process, networks with same structure but trained multiple times stochastically, and networks with different structure. In addition, we further studied the over-confidence phenomenon of the neural networks, as well as its impact on the ensemble methods. Across all of our experiments, the Super Learner achieved best performance among all the ensemble methods in this study.
人工神经网络已成功应用于各种机器学习任务，包括图像识别、语义分割和机器翻译。然而，很少有研究对人工神经网络的集成进行全面研究。在这项工作中，我们研究了多种广泛使用的集成方法，包括无加权平均、多数投票、贝叶斯最优分类器和（离散）超级学习器，用于图像识别任务，以深度神经网络作为候选算法。我们设计了几个实验，候选算法包括在单个训练过程中具有不同模型检查点的相同网络结构、具有相同结构但随机训练多次的网络以及具有不同结构的网络。此外，我们进一步研究了神经网络的过度自信现象及其对集成方法的影响。在我们所有的实验中，超级学习器在本研究中的所有集成方法中表现最佳。
