Department of Psychology, The Ohio State University.
J Exp Psychol Gen. 2024 Apr;153(4):1139-1151. doi: 10.1037/xge0001273.
The calculation of statistical power has been taken up as a simple yet informative tool to assist in designing an experiment, particularly in justifying sample size. A difficulty with using power for this purpose is that the classical power formula does not incorporate sources of uncertainty (e.g., sampling variability) that can impact the computed power value, leading to a false sense of precision and confidence in design choices. We use simulations to demonstrate the consequences of adding two common sources of uncertainty to the calculation of power. Sampling variability in the estimated effect size (Cohen's ) can introduce a large amount of uncertainty (e.g., sometimes producing rather flat distributions) in power and sample-size determination. The addition of random fluctuations in the population effect size can cause values of its estimates to take on a sign opposite the population value, making calculated power values meaningless. These results suggest that calculated power values or use of such values to justify sample size add little to planning a study. As a result, researchers should put little confidence in power-based choices when planning future studies. (PsycInfo Database Record (c) 2024 APA, all rights reserved).
统计功效的计算已被用作一种简单而有用的工具，以协助实验设计，特别是在确定样本量方面。在这方面使用功效时的一个困难是，经典的功效公式并未纳入可能影响计算出的功效值的不确定性来源（例如，抽样变异性），从而导致对设计选择的精度和信心产生错误的感觉。我们使用模拟来演示向功效计算中添加两个常见的不确定性来源的后果。估计效果大小（Cohen's ）的抽样变异性会在功效和样本量确定中引入大量不确定性（例如，有时会产生相当平坦的分布）。在总体效果大小中添加随机波动会导致其估计值的符号与总体值相反，从而使计算出的功效值变得毫无意义。这些结果表明，计算出的功效值或使用这些值来证明样本量的合理性对规划研究没有多大帮助。因此，研究人员在规划未来研究时不应过分相信基于功效的选择。
