Koehler Elizabeth, Brown Elizabeth, Haneuse Sebastien J-P A
Department of Biostatistics, Vanderbilt University, Nashville, TN 37232.
Am Stat. 2009 May 1;63(2):155-162. doi: 10.1198/tast.2009.0030.
Statistical experiments, more commonly referred to as Monte Carlo or simulation studies, are used to study the behavior of statistical methods and measures under controlled situations. Whereas recent computing and methodological advances have permitted increased efficiency in the simulation process, known as variance reduction, such experiments remain limited by their finite nature and hence are subject to uncertainty; when a simulation is run more than once, different results are obtained. However, virtually no emphasis has been placed on reporting the uncertainty, referred to here as Monte Carlo error, associated with simulation results in the published literature, or on justifying the number of replications used. These deserve broader consideration. Here we present a series of simple and practical methods for estimating Monte Carlo error as well as determining the number of replications required to achieve a desired level of accuracy. The issues and methods are demonstrated with two simple examples, one evaluating operating characteristics of the maximum likelihood estimator for the parameters in logistic regression and the other in the context of using the bootstrap to obtain 95% confidence intervals. The results suggest that in many settings, Monte Carlo error may be more substantial than traditionally thought.
统计实验，更常见的是被称为蒙特卡洛或模拟研究，用于在可控情况下研究统计方法和度量的行为。尽管最近的计算和方法进展提高了模拟过程的效率，即所谓的方差缩减，但此类实验仍受其有限性质的限制，因此存在不确定性；当多次运行模拟时，会得到不同的结果。然而，在已发表的文献中，几乎没有强调报告与模拟结果相关的不确定性（这里称为蒙特卡洛误差），也没有对所使用的重复次数进行合理性说明。这些值得更广泛的考虑。在此，我们提出一系列简单实用的方法来估计蒙特卡洛误差，以及确定达到所需精度水平所需的重复次数。通过两个简单示例展示了这些问题和方法，一个示例评估逻辑回归中参数的最大似然估计器的操作特性，另一个示例是在使用自助法获得95%置信区间的背景下。结果表明，在许多情况下，蒙特卡洛误差可能比传统认为的更为显著。
