Department of Welfare, EAH Jena University of Applied Sciences, Jena, Germany.
Department of Clinical Epidemiology, Leiden University Medical Center, Leiden, The Netherlands.
Biom J. 2024 Jan;66(1):e2200108. doi: 10.1002/bimj.202200108. Epub 2023 May 18.
Logistic regression is one of the most commonly used approaches to develop clinical risk prediction models. Developers of such models often rely on approaches that aim to minimize the risk of overfitting and improve predictive performance of the logistic model, such as through likelihood penalization and variance decomposition techniques. We present an extensive simulation study that compares the out-of-sample predictive performance of risk prediction models derived using the elastic net, with Lasso and ridge as special cases, and variance decomposition techniques, namely, incomplete principal component regression and incomplete partial least squares regression. We varied the expected events per variable, event fraction, number of candidate predictors, presence of noise predictors, and the presence of sparse predictors in a full-factorial design. Predictive performance was compared on measures of discrimination, calibration, and prediction error. Simulation metamodels were derived to explain the performance differences within model derivation approaches. Our results indicate that, on average, prediction models developed using penalization and variance decomposition approaches outperform models developed using ordinary maximum likelihood estimation, with penalization approaches being consistently superior over the variance decomposition approaches. Differences in performance were most pronounced on the calibration of the model. Performance differences regarding prediction error and concordance statistic outcomes were often small between approaches. The use of likelihood penalization and variance decomposition techniques methods was illustrated in the context of peripheral arterial disease.
逻辑回归是开发临床风险预测模型最常用的方法之一。此类模型的开发者通常依赖于旨在最小化过度拟合风险并提高逻辑模型预测性能的方法，例如通过似然惩罚和方差分解技术。我们进行了一项广泛的模拟研究，比较了使用弹性网络、Lasso 和岭回归作为特例以及方差分解技术（即不完全主成分回归和不完全偏最小二乘回归）得出的风险预测模型的样本外预测性能。我们在全因子设计中改变了每个变量的预期事件数、事件分数、候选预测器数量、噪声预测器的存在和稀疏预测器的存在。预测性能通过判别、校准和预测误差度量进行比较。模拟元模型用于解释模型推导方法内的性能差异。我们的结果表明，平均而言，使用惩罚和方差分解方法开发的预测模型优于使用普通最大似然估计开发的模型，其中惩罚方法始终优于方差分解方法。模型校准方面的性能差异最为明显。在预测误差和一致性统计结果方面，方法之间的差异通常很小。在周围血管疾病的背景下，说明了似然惩罚和方差分解技术方法的使用。
