Posgrado en Ingeniería de Sistemas, Centro de Innovación, Investigación y Desarrollo en Ingeniería y Tecnología, Facultad de Ingeniería Mecánica y Eléctrica, Universidad Autónoma de Nuevo León, San Nicolás de los Garza, NL 66450, México.
Neural Comput. 2010 Jun;22(6):1573-96. doi: 10.1162/neco.2010.01-09-943.
A novel formalism for bayesian learning in the context of complex inference models is proposed. The method is based on the use of the stationary Fokker-Planck (SFP) approach to sample from the posterior density. Stationary Fokker-Planck sampling generalizes the Gibbs sampler algorithm for arbitrary and unknown conditional densities. By the SFP procedure, approximate analytical expressions for the conditionals and marginals of the posterior can be constructed. At each stage of SFP, the approximate conditionals are used to define a Gibbs sampling process, which is convergent to the full joint posterior. By the analytical marginals efficient learning methods in the context of artificial neural networks are outlined. Offline and incremental bayesian inference and maximum likelihood estimation from the posterior are performed in classification and regression examples. A comparison of SFP with other Monte Carlo strategies in the general problem of sampling from arbitrary densities is also presented. It is shown that SFP is able to jump large low-probability regions without the need of a careful tuning of any step-size parameter. In fact, the SFP method requires only a small set of meaningful parameters that can be selected following clear, problem-independent guidelines. The computation cost of SFP, measured in terms of loss function evaluations, grows linearly with the given model's dimension.
提出了一种新的贝叶斯学习形式主义，用于复杂推理模型的背景下。该方法基于使用静止福克-普朗克（SFP）方法从后验密度中抽样。静止福克-普朗克抽样法将吉布斯抽样算法推广到任意和未知的条件密度。通过 SFP 过程，可以构建后验条件和边缘的近似解析表达式。在 SFP 的每个阶段，使用近似条件来定义吉布斯抽样过程，该过程收敛于完整的联合后验。通过解析边缘，概述了在人工神经网络背景下有效的学习方法。在分类和回归示例中，从后验进行离线和增量贝叶斯推断和最大似然估计。还在从任意密度抽样的一般问题中对 SFP 与其他蒙特卡罗策略进行了比较。结果表明，SFP 能够在不需要仔细调整任何步长参数的情况下跳过大的低概率区域。事实上，SFP 方法只需要一组有意义的参数，这些参数可以按照明确的、与问题无关的准则进行选择。以损失函数评估为衡量标准，SFP 的计算成本随给定模型的维度呈线性增长。
