Martínez-Muñoz Gonzalo, Hernández-Lobato Daniel, Suárez Alberto
Computer Science Department, Universidad Autónoma de Madrid, Cantoblanco, Spain.
IEEE Trans Pattern Anal Mach Intell. 2009 Feb;31(2):245-59. doi: 10.1109/TPAMI.2008.78.
Several pruning strategies that can be used to reduce the size and increase the accuracy of bagging ensembles are analyzed. These heuristics select subsets of complementary classifiers that, when combined, can perform better than the whole ensemble. The pruning methods investigated are based on modifying the order of aggregation of classifiers in the ensemble. In the original bagging algorithm, the order of aggregation is left unspecified. When this order is random, the generalization error typically decreases as the number of classifiers in the ensemble increases. If an appropriate ordering for the aggregation process is devised, the generalization error reaches a minimum at intermediate numbers of classifiers. This minimum lies below the asymptotic error of bagging. Pruned ensembles are obtained by retaining a fraction of the classifiers in the ordered ensemble. The performance of these pruned ensembles is evaluated in several benchmark classification tasks under different training conditions. The results of this empirical investigation show that ordered aggregation can be used for the efficient generation of pruned ensembles that are competitive, in terms of performance and robustness of classification, with computationally more costly methods that directly select optimal or near-optimal subensembles.
分析了几种可用于减小装袋集成规模并提高其准确性的剪枝策略。这些启发式方法选择互补分类器的子集，这些子集组合在一起时，性能可能优于整个集成。所研究的剪枝方法基于修改集成中分类器的聚合顺序。在原始的装袋算法中，聚合顺序未作规定。当此顺序为随机时，泛化误差通常会随着集成中分类器数量的增加而减小。如果为聚合过程设计适当的排序，则泛化误差在分类器数量处于中间值时达到最小值。此最小值低于装袋的渐近误差。通过在有序集成中保留一部分分类器来获得剪枝后的集成。在不同训练条件下的几个基准分类任务中评估了这些剪枝后集成的性能。这项实证研究的结果表明，有序聚合可用于高效生成剪枝后的集成，在分类性能和稳健性方面，这些剪枝后的集成与直接选择最优或接近最优子集成的计算成本更高的方法具有竞争力。
