Daniels M J, Kass R E
Department of Statistics, Iowa State University, Ames 50011, USA.
Biometrics. 2001 Dec;57(4):1173-84. doi: 10.1111/j.0006-341x.2001.01173.x.
Estimation of covariance matrices in small samples has been studied by many authors. Standard estimators, like the unstructured maximum likelihood estimator (ML) or restricted maximum likelihood (REML) estimator, can be very unstable with the smallest estimated eigenvalues being too small and the largest too big. A standard approach to more stably estimating the matrix in small samples is to compute the ML or REML estimator under some simple structure that involves estimation of fewer parameters, such as compound symmetry or independence. However, these estimators will not be consistent unless the hypothesized structure is correct. If interest focuses on estimation of regression coefficients with correlated (or longitudinal) data, a sandwich estimator of the covariance matrix may be used to provide standard errors for the estimated coefficients that are robust in the sense that they remain consistent under misspecification of the covariance structure. With large matrices, however, the inefficiency of the sandwich estimator becomes worrisome. We consider here two general shrinkage approaches to estimating the covariance matrix and regression coefficients. The first involves shrinking the eigenvalues of the unstructured ML or REML estimator. The second involves shrinking an unstructured estimator toward a structured estimator. For both cases, the data determine the amount of shrinkage. These estimators are consistent and give consistent and asymptotically efficient estimates for regression coefficients. Simulations show the improved operating characteristics of the shrinkage estimators of the covariance matrix and the regression coefficients in finite samples. The final estimator chosen includes a combination of both shrinkage approaches, i.e., shrinking the eigenvalues and then shrinking toward structure. We illustrate our approach on a sleep EEG study that requires estimation of a 24 x 24 covariance matrix and for which inferences on mean parameters critically depend on the covariance estimator chosen. We recommend making inference using a particular shrinkage estimator that provides a reasonable compromise between structured and unstructured estimators.
许多作者研究了小样本协方差矩阵的估计。标准估计器，如无结构最大似然估计器（ML）或限制最大似然（REML）估计器，可能非常不稳定，估计出的最小特征值过小，最大特征值过大。在小样本中更稳定地估计矩阵的一种标准方法是在某种简单结构下计算ML或REML估计器，这种结构涉及较少参数的估计，如复合对称或独立性。然而，除非假设的结构正确，这些估计器将不一致。如果关注的是具有相关（或纵向）数据的回归系数估计，可以使用协方差矩阵的三明治估计器来为估计系数提供标准误差，这些标准误差在协方差结构指定错误的情况下仍保持一致，具有稳健性。然而，对于大型矩阵，三明治估计器的低效率令人担忧。我们在此考虑两种估计协方差矩阵和回归系数的一般收缩方法。第一种方法是收缩无结构ML或REML估计器的特征值。第二种方法是将无结构估计器向结构化估计器收缩。对于这两种情况，数据决定收缩量。这些估计器是一致的，并且对回归系数给出一致且渐近有效的估计。模拟显示了有限样本中协方差矩阵和回归系数收缩估计器改进的操作特性。最终选择的估计器包括两种收缩方法的组合，即先收缩特征值，然后向结构收缩。我们在一项睡眠脑电图研究中说明了我们的方法，该研究需要估计一个24×24的协方差矩阵，并且对均值参数的推断严重依赖于所选择的协方差估计器。我们建议使用一种特定的收缩估计器进行推断，该估计器在结构化和非结构化估计器之间提供了合理的折衷。
