Wood Simon N, Fasiolo Matteo
School of Mathematics, University of Bristol, Bristol, UK.
Biometrics. 2017 Dec;73(4):1071-1081. doi: 10.1111/biom.12666. Epub 2017 Feb 13.
We consider the optimization of smoothing parameters and variance components in models with a regular log likelihood subject to quadratic penalization of the model coefficients, via a generalization of the method of Fellner (1986) and Schall (1991). In particular: (i) we generalize the original method to the case of penalties that are linear in several smoothing parameters, thereby covering the important cases of tensor product and adaptive smoothers; (ii) we show why the method's steps increase the restricted marginal likelihood of the model, that it tends to converge faster than the EM algorithm, or obvious accelerations of this, and investigate its relation to Newton optimization; (iii) we generalize the method to any Fisher regular likelihood. The method represents a considerable simplification over existing methods of estimating smoothing parameters in the context of regular likelihoods, without sacrificing generality: for example, it is only necessary to compute with the same first and second derivatives of the log-likelihood required for coefficient estimation, and not with the third or fourth order derivatives required by alternative approaches. Examples are provided which would have been impossible or impractical with pre-existing Fellner-Schall methods, along with an example of a Tweedie location, scale and shape model which would be a challenge for alternative methods, and a sparse additive modeling example where the method facilitates computational efficiency gains of several orders of magnitude. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.
我们通过推广费尔纳（1986年）和沙尔（1991年）的方法，考虑在具有正则对数似然且模型系数受二次惩罚的模型中，对平滑参数和方差分量进行优化。具体而言：（i）我们将原始方法推广到在多个平滑参数中呈线性的惩罚情况，从而涵盖张量积和自适应平滑器的重要情形；（ii）我们说明了该方法的步骤为何会增加模型的受限边际似然，它往往比期望最大化（EM）算法或其明显加速方法收敛得更快，并研究了它与牛顿优化的关系；（iii）我们将该方法推广到任何费希尔正则似然。该方法相较于在正则似然背景下估计平滑参数的现有方法有相当大的简化，且不牺牲一般性：例如，只需要使用系数估计所需的对数似然的一阶和二阶导数进行计算，而无需使用替代方法所需的三阶或四阶导数。文中给出了一些例子，这些例子用先前的费尔纳 - 沙尔方法是不可能做到或不切实际的，还有一个特威迪位置、尺度和形状模型的例子，这对替代方法来说是个挑战，以及一个稀疏加法建模的例子，该方法在这个例子中实现了几个数量级的计算效率提升。本文是一篇根据知识共享署名许可协议条款的开放获取文章，允许在任何媒介中使用、分发和复制，前提是正确引用原始作品。
