Hunter David R, Li Runze
Department of Statistics, The Pennsylvania State University University Park, Pennsylvania 16802-2111, E-mail:
Ann Stat. 2005;33(4):1617-1642. doi: 10.1214/009053605000000200.
Variable selection is fundamental to high-dimensional statistical modeling. Many variable selection techniques may be implemented by maximum penalized likelihood using various penalty functions. Optimizing the penalized likelihood function is often challenging because it may be nondifferentiable and/or nonconcave. This article proposes a new class of algorithms for finding a maximizer of the penalized likelihood for a broad class of penalty functions. These algorithms operate by perturbing the penalty function slightly to render it differentiable, then optimizing this differentiable function using a minorize-maximize (MM) algorithm. MM algorithms are useful extensions of the well-known class of EM algorithms, a fact that allows us to analyze the local and global convergence of the proposed algorithm using some of the techniques employed for EM algorithms. In particular, we prove that when our MM algorithms converge, they must converge to a desirable point; we also discuss conditions under which this convergence may be guaranteed. We exploit the Newton-Raphson-like aspect of these algorithms to propose a sandwich estimator for the standard errors of the estimators. Our method performs well in numerical tests.
变量选择是高维统计建模的基础。许多变量选择技术可以通过使用各种惩罚函数的最大惩罚似然来实现。优化惩罚似然函数通常具有挑战性，因为它可能不可微和/或非凹。本文提出了一类新的算法，用于为广泛的惩罚函数找到惩罚似然的最大化者。这些算法通过对惩罚函数进行轻微扰动使其可微，然后使用最小化-最大化（MM）算法优化这个可微函数来运行。MM算法是著名的EM算法类的有用扩展，这一事实使我们能够使用一些用于EM算法的技术来分析所提出算法的局部和全局收敛性。特别是，我们证明了当我们的MM算法收敛时，它们必须收敛到一个理想点；我们还讨论了可以保证这种收敛的条件。我们利用这些算法类似牛顿-拉夫森的方面，为估计量的标准误差提出了一种三明治估计量。我们的方法在数值测试中表现良好。
