Opper M, Winther O
Department of Computer Science and Applied Mathematics, Aston University, Birmingham, UK.
Neural Comput. 2000 Nov;12(11):2655-84. doi: 10.1162/089976600300014881.
We derive a mean-field algorithm for binary classification with gaussian processes that is based on the TAP approach originally proposed in statistical physics of disordered systems. The theory also yields an approximate leave-one-out estimator for the generalization error, which is computed with no extra computational cost. We show that from the TAP approach, it is possible to derive both a simpler "naive" mean-field theory and support vector machines (SVMs) as limiting cases. For both mean-field algorithms and support vector machines, simulation results for three small benchmark data sets are presented. They show that one may get state-of-the-art performance by using the leave-one-out estimator for model selection and the built-in leave-one-out estimators are extremely precise when compared to the exact leave-one-out estimate. The second result is taken as strong support for the internal consistency of the mean-field approach.
我们推导了一种基于无序系统统计物理学中最初提出的TAP方法的高斯过程二元分类平均场算法。该理论还产生了一种用于泛化误差的近似留一法估计器，其计算无需额外的计算成本。我们表明，从TAP方法出发，有可能推导出更简单的“朴素”平均场理论和作为极限情况的支持向量机（SVM）。对于平均场算法和支持向量机，都给出了三个小型基准数据集的模拟结果。结果表明，通过使用留一法估计器进行模型选择可以获得最优性能，并且与精确的留一法估计相比，内置的留一法估计器极其精确。第二个结果被视为对平均场方法内部一致性的有力支持。
