Centre for Prognosis Research, School of Medicine, Keele University, Keele, UK.
Population Health Research Institute, St George's, University of London, London, UK.
Stat Med. 2021 Jan 15;40(1):133-146. doi: 10.1002/sim.8766. Epub 2020 Nov 4.
Clinical prediction models provide individualized outcome predictions to inform patient counseling and clinical decision making. External validation is the process of examining a prediction model's performance in data independent to that used for model development. Current external validation studies often suffer from small sample sizes, and subsequently imprecise estimates of a model's predictive performance. To address this, we propose how to determine the minimum sample size needed for external validation of a clinical prediction model with a continuous outcome. Four criteria are proposed, that target precise estimates of (i) R (the proportion of variance explained), (ii) calibration-in-the-large (agreement between predicted and observed outcome values on average), (iii) calibration slope (agreement between predicted and observed values across the range of predicted values), and (iv) the variance of observed outcome values. Closed-form sample size solutions are derived for each criterion, which require the user to specify anticipated values of the model's performance (in particular R ) and the outcome variance in the external validation dataset. A sensible starting point is to base values on those for the model development study, as obtained from the publication or study authors. The largest sample size required to meet all four criteria is the recommended minimum sample size needed in the external validation dataset. The calculations can also be applied to estimate expected precision when an existing dataset with a fixed sample size is available, to help gauge if it is adequate. We illustrate the proposed methods on a case-study predicting fat-free mass in children.
临床预测模型提供个体化的预后预测，以告知患者咨询和临床决策。外部验证是检验预测模型在与模型开发所用数据不同的数据中的性能的过程。目前的外部验证研究往往受到样本量小的影响，因此模型预测性能的估计不准确。为了解决这个问题，我们提出了如何确定具有连续结局的临床预测模型外部验证所需的最小样本量。提出了四个标准，旨在精确估计：（i）R（解释方差的比例），（ii）大校准（预测和观察结局值之间的平均一致性），（iii）校准斜率（预测值范围内预测和观察值之间的一致性），以及（iv）观察结局值的方差。为每个标准推导出了闭式样本量解，这些解要求用户指定模型性能（特别是 R ）和外部验证数据集的结局方差的预期值。一个合理的起点是基于模型开发研究中获得的值，这些值可以从出版物或研究作者处获得。满足所有四个标准所需的最大样本量是外部验证数据集中建议的最小样本量。这些计算也可用于估计具有固定样本量的现有数据集的预期精度，以帮助评估其是否足够。我们以一个预测儿童去脂体重的案例研究说明了所提出的方法。
