Harva Markus
Laboratory of Computer and Information Science, Helsinki University of Technology, Espoo, Finland.
Neural Netw. 2007 May;20(4):550-8. doi: 10.1016/j.neunet.2007.04.010. Epub 2007 Apr 30.
In many applications of regression, the conditional average of the target variable is not sufficient for prediction. The dependencies between the explanatory variables and the target variable can be complex calling for modelling of the full conditional probability density. The ubiquitous problem with such methods is overfitting since due to the flexibility of the model the likelihood of any datapoint can be made arbitrarily large. In this paper a method for predicting uncertainty by modelling the conditional density is presented based on conditioning the scale parameter of the noise process on the explanatory variables. The model is constructed in such a manner that the unpredictability of the scale of the target distribution translates into a more robust predictive distribution. The overfitting problems are solved by learning the model using variational EM. The method is experimentally demonstrated with synthetic data as well as with real-world environmental data. The viability of the approach was put to test in the 'Predictive uncertainty in environmental modelling' competition held at WCCI'06. The proposed method won the competition.
在回归的许多应用中，目标变量的条件均值不足以进行预测。解释变量与目标变量之间的依赖关系可能很复杂，需要对完整的条件概率密度进行建模。此类方法普遍存在的问题是过拟合，因为由于模型的灵活性，任何数据点的似然性都可以被任意增大。本文提出了一种通过对条件密度进行建模来预测不确定性的方法，该方法基于根据解释变量对噪声过程的尺度参数进行条件设定。模型的构建方式使得目标分布尺度的不可预测性转化为更稳健的预测分布。通过使用变分期望最大化算法学习模型来解决过拟合问题。该方法通过合成数据以及真实世界环境数据进行了实验验证。该方法的可行性在2006年世界计算智能大会举办的“环境建模中的预测不确定性”竞赛中得到了检验。所提出的方法赢得了该竞赛。
