Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore.
Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore.
BMC Med Res Methodol. 2022 Nov 4;22(1):286. doi: 10.1186/s12874-022-01770-y.
Risk prediction models are useful tools in clinical decision-making which help with risk stratification and resource allocations and may lead to a better health care for patients. AutoScore is a machine learning-based automatic clinical score generator for binary outcomes. This study aims to expand the AutoScore framework to provide a tool for interpretable risk prediction for ordinal outcomes.
The AutoScore-Ordinal framework is generated using the same 6 modules of the original AutoScore algorithm including variable ranking, variable transformation, score derivation (from proportional odds models), model selection, score fine-tuning, and model evaluation. To illustrate the AutoScore-Ordinal performance, the method was conducted on electronic health records data from the emergency department at Singapore General Hospital over 2008 to 2017. The model was trained on 70% of the data, validated on 10% and tested on the remaining 20%.
This study included 445,989 inpatient cases, where the distribution of the ordinal outcome was 80.7% alive without 30-day readmission, 12.5% alive with 30-day readmission, and 6.8% died inpatient or by day 30 post discharge. Two point-based risk prediction models were developed using two sets of 8 predictor variables identified by the flexible variable selection procedure. The two models indicated reasonably good performance measured by mean area under the receiver operating characteristic curve (0.758 and 0.793) and generalized c-index (0.737 and 0.760), which were comparable to alternative models.
AutoScore-Ordinal provides an automated and easy-to-use framework for development and validation of risk prediction models for ordinal outcomes, which can systematically identify potential predictors from high-dimensional data.
风险预测模型是临床决策中的有用工具，有助于风险分层和资源分配，并可能为患者提供更好的医疗保健。AutoScore 是一种基于机器学习的二进制结果自动临床评分生成器。本研究旨在扩展 AutoScore 框架，为有序结果提供可解释的风险预测工具。
使用原始 AutoScore 算法的 6 个模块生成 AutoScore-Ordinal 框架，包括变量排名、变量转换、评分推导（来自比例优势模型）、模型选择、评分微调以及模型评估。为了说明 AutoScore-Ordinal 的性能，该方法在 2008 年至 2017 年期间使用新加坡综合医院急诊科的电子健康记录数据进行。该模型在 70%的数据上进行训练，在 10%的数据上进行验证，在其余 20%的数据上进行测试。
本研究包括 445,989 例住院患者，其中有序结果的分布为 80.7%无 30 天再入院存活，12.5%存活且 30 天内再入院，6.8%住院或出院后第 30 天内死亡。使用灵活变量选择过程确定的两组 8 个预测变量开发了两个基于两点的风险预测模型。这两个模型的表现相当不错，衡量指标为平均接收者操作特征曲线下面积（0.758 和 0.793）和广义 c 指数（0.737 和 0.760），与替代模型相当。
AutoScore-Ordinal 为有序结果的风险预测模型的开发和验证提供了一个自动化且易于使用的框架，可以系统地从高维数据中识别潜在的预测因素。
