Lusa Lara, Kappenberg Franziska, Collins Gary S, Schmid Matthias, Sauerbrei Willi, Rahnenführer Jörg
Department of Mathematics, Faculty of Mathematics, Natural Sciences and Information Technologies, University of Primorska, Koper/Capodistria, Slovenia.
Institute for Biostatistics and Medical Informatics, Faculty of Medicine, University of Ljubljana, Ljubljana, Slovenia.
BMC Med Res Methodol. 2025 Jul 1;25(1):167. doi: 10.1186/s12874-025-02605-2.
The number of prediction models proposed in the biomedical literature has been growing year on year. In the last few years there has been an increasing attention to the changes occurring in the prediction modeling landscape. It is suggested that machine learning techniques are becoming more popular to develop prediction models to exploit complex data structures, higher-dimensional predictor spaces, very large number of participants, heterogeneous subgroups, with the ability to capture higher-order interactions. We examine the changes in modelling practices by investigating a selection of systematic reviews on prediction models published in the biomedical literature. We selected systematic reviews published between 2020 and 2022 which included at least 50 prediction models. Information was extracted guided by the CHARMS checklist. Time trends were explored using the models published since 2005. We identified 8 reviews, which included 1448 prediction models published in 887 papers. The average number of study participants and outcome events increased considerably between 2015 and 2019 but remained stable afterwards. The number of candidate and final predictors did not noticeably increase over the study period, with a few recent studies using very large numbers of predictors. Internal validation and reporting of discrimination measures became more common, but assessing calibration and carrying out external validation were less common. Information about missing values was not reported in about half of the papers, however the use of imputation methods increased. There was no sign of an increase in using of machine learning methods. Overall, most of the findings were heterogeneous across reviews. Our findings indicate that changes in the prediction modeling landscape in biomedicine are smaller than expected and that poor reporting is still common; adherence to well established best practice recommendations from the traditional biostatistics literature is still needed. For machine learning best practice recommendations are still missing, whereas such recommendations are available in the traditional biostatistics literature, but adherence is still inadequate.
生物医学文献中提出的预测模型数量逐年增加。在过去几年中，人们越来越关注预测建模领域发生的变化。有人认为，机器学习技术在开发预测模型以利用复杂数据结构、高维预测变量空间、大量参与者、异质子组以及捕捉高阶相互作用方面正变得越来越受欢迎。我们通过调查生物医学文献中发表的一系列关于预测模型的系统评价来研究建模实践的变化。我们选择了2020年至2022年期间发表的至少包含50个预测模型的系统评价。信息提取以CHARM清单为指导。利用2005年以来发表的模型探索时间趋势。我们确定了8篇综述，其中包括887篇论文中发表的1448个预测模型。2015年至2019年期间，研究参与者和结局事件的平均数量大幅增加，但此后保持稳定。在研究期间，候选预测变量和最终预测变量的数量没有明显增加，最近有一些研究使用了大量的预测变量。内部验证和鉴别测量的报告变得更加普遍，但评估校准和进行外部验证则不太常见。大约一半的论文没有报告关于缺失值的信息，然而插补方法的使用有所增加。没有迹象表明机器学习方法的使用有所增加。总体而言，大多数研究结果在不同综述中存在异质性。我们的研究结果表明，生物医学中预测建模领域的变化比预期的要小，报告不佳仍然很常见；仍然需要遵循传统生物统计学文献中既定的最佳实践建议。对于机器学习，最佳实践建议仍然缺失，而传统生物统计学文献中有此类建议，但遵循情况仍然不足。
