Dijkstra Hidde, van de Kuit Anouk, de Groot Tom M, Canta Olga, Groot Olivier Q, Oosterhoff Jacobien H, Doornberg Job N, van den Bekerom Michel, Calderon Santiago L, Colaris Joost, Duis Kaj T, Esfahani Soheil A, DiGiovanni Chris, Gordon Max, Guss Daniel, IJpma Frank, Jaarsma Ruurd, Janssen Michiel, Jayakumar Prakash, Kerkhoffs Gino M, Leighton Ross, van Munster Barbara, Poolman Rudolf, Ring David, Schemtisch Emil, Stirler Vincent, Tornetta Paul, Wijffels Mathieu
Department of Orthopaedic Surgery, University Medical Centre Groningen, Groningen, Netherlands.
University Center for Geriatric Medicine, University of Groningen, University Medical Center Groningen, Groningen, Netherlands.
Bone Jt Open. 2024 Jan 16;5(1):9-19. doi: 10.1302/2633-1462.51.BJO-2023-0095.R1.
Machine-learning (ML) prediction models in orthopaedic trauma hold great promise in assisting clinicians in various tasks, such as personalized risk stratification. However, an overview of current applications and critical appraisal to peer-reviewed guidelines is lacking. The objectives of this study are to 1) provide an overview of current ML prediction models in orthopaedic trauma; 2) evaluate the completeness of reporting following the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement; and 3) assess the risk of bias following the Prediction model Risk Of Bias Assessment Tool (PROBAST) tool.
A systematic search screening 3,252 studies identified 45 ML-based prediction models in orthopaedic trauma up to January 2023. The TRIPOD statement assessed transparent reporting and the PROBAST tool the risk of bias.
A total of 40 studies reported on training and internal validation; four studies performed both development and external validation, and one study performed only external validation. The most commonly reported outcomes were mortality (33%, 15/45) and length of hospital stay (9%, 4/45), and the majority of prediction models were developed in the hip fracture population (60%, 27/45). The overall median completeness for the TRIPOD statement was 62% (interquartile range 30 to 81%). The overall risk of bias in the PROBAST tool was low in 24% (11/45), high in 69% (31/45), and unclear in 7% (3/45) of the studies. High risk of bias was mainly due to analysis domain concerns including small datasets with low number of outcomes, complete-case analysis in case of missing data, and no reporting of performance measures.
The results of this study showed that despite a myriad of potential clinically useful applications, a substantial part of ML studies in orthopaedic trauma lack transparent reporting, and are at high risk of bias. These problems must be resolved by following established guidelines to instil confidence in ML models among patients and clinicians. Otherwise, there will remain a sizeable gap between the development of ML prediction models and their clinical application in our day-to-day orthopaedic trauma practice.
骨科创伤中的机器学习（ML）预测模型在协助临床医生完成各种任务（如个性化风险分层）方面具有巨大潜力。然而，目前缺乏对其应用现状的概述以及对同行评审指南的批判性评价。本研究的目的是：1）概述骨科创伤中当前的ML预测模型；2）根据个体预后或诊断多变量预测模型的透明报告（TRIPOD）声明评估报告的完整性；3）根据预测模型偏倚风险评估工具（PROBAST）评估偏倚风险。
系统检索3252项研究，截至2023年1月，在骨科创伤中确定了45个基于ML的预测模型。TRIPOD声明评估报告的透明度，PROBAST工具评估偏倚风险。
共有40项研究报告了训练和内部验证；4项研究进行了模型开发和外部验证，1项研究仅进行了外部验证。最常报告的结局是死亡率（33%，15/45）和住院时间（9%，4/45），大多数预测模型是在髋部骨折人群中开发的（60%，27/45）。TRIPOD声明的总体中位完整性为62%（四分位间距30%至81%）。在PROBAST工具中，24%（11/45）的研究总体偏倚风险较低，69%（31/45）的研究偏倚风险较高，7%（3/45）的研究偏倚风险不明确。高偏倚风险主要归因于分析领域的问题，包括结局数量少的小数据集、数据缺失时的完全病例分析以及未报告性能指标。
本研究结果表明，尽管有大量潜在的临床有用应用，但骨科创伤中相当一部分ML研究缺乏透明报告，且偏倚风险高。必须遵循既定指南解决这些问题，以使患者和临床医生对ML模型有信心。否则，在我们日常的骨科创伤实践中，ML预测模型的开发与临床应用之间仍将存在相当大的差距。
