Xie Feng, Chakraborty Bibhas, Ong Marcus Eng Hock, Goldstein Benjamin Alan, Liu Nan
Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore.
Department of Statistics and Applied Probability, National University of Singapore, Singapore, Singapore.
JMIR Med Inform. 2020 Oct 21;8(10):e21798. doi: 10.2196/21798.
Risk scores can be useful in clinical risk stratification and accurate allocations of medical resources, helping health providers improve patient care. Point-based scores are more understandable and explainable than other complex models and are now widely used in clinical decision making. However, the development of the risk scoring model is nontrivial and has not yet been systematically presented, with few studies investigating methods of clinical score generation using electronic health records.
This study aims to propose AutoScore, a machine learning-based automatic clinical score generator consisting of 6 modules for developing interpretable point-based scores. Future users can employ the AutoScore framework to create clinical scores effortlessly in various clinical applications.
We proposed the AutoScore framework comprising 6 modules that included variable ranking, variable transformation, score derivation, model selection, score fine-tuning, and model evaluation. To demonstrate the performance of AutoScore, we used data from the Beth Israel Deaconess Medical Center to build a scoring model for mortality prediction and then compared the data with other baseline models using the receiver operating characteristic analysis. A software package in R 3.5.3 (R Foundation) was also developed to demonstrate the implementation of AutoScore.
Implemented on the data set with 44,918 individual admission episodes of intensive care, the AutoScore-created scoring models performed comparably well as other standard methods (ie, logistic regression, stepwise regression, least absolute shrinkage and selection operator, and random forest) in terms of predictive accuracy and model calibration but required fewer predictors and presented high interpretability and accessibility. The nine-variable, AutoScore-created, point-based scoring model achieved an area under the curve (AUC) of 0.780 (95% CI 0.764-0.798), whereas the model of logistic regression with 24 variables had an AUC of 0.778 (95% CI 0.760-0.795). Moreover, the AutoScore framework also drives the clinical research continuum and automation with its integration of all necessary modules.
We developed an easy-to-use, machine learning-based automatic clinical score generator, AutoScore; systematically presented its structure; and demonstrated its superiority (predictive performance and interpretability) over other conventional methods using a benchmark database. AutoScore will emerge as a potential scoring tool in various medical applications.
风险评分在临床风险分层和医疗资源的准确分配中可能有用，有助于医疗服务提供者改善患者护理。基于点数的评分比其他复杂模型更易于理解和解释，目前已广泛应用于临床决策。然而，风险评分模型的开发并非易事，尚未得到系统介绍，很少有研究调查使用电子健康记录生成临床评分的方法。
本研究旨在提出AutoScore，这是一种基于机器学习的自动临床评分生成器，由6个模块组成，用于开发可解释的基于点数的评分。未来的用户可以使用AutoScore框架在各种临床应用中轻松创建临床评分。
我们提出了包含6个模块的AutoScore框架，这些模块包括变量排序、变量转换、评分推导、模型选择、评分微调以及模型评估。为了展示AutoScore的性能，我们使用贝斯以色列女执事医疗中心的数据构建了一个死亡率预测评分模型，然后使用受试者工作特征分析将该数据与其他基线模型进行比较。还开发了一个R 3.5.3（R基金会）软件包来展示AutoScore的实现。
在包含44918例重症监护个体入院病例的数据集上实施时，AutoScore创建的评分模型在预测准确性和模型校准方面与其他标准方法（即逻辑回归、逐步回归、最小绝对收缩和选择算子以及随机森林）表现相当，但所需的预测变量更少，具有较高的可解释性和易用性。由AutoScore创建的基于点数的九变量评分模型的曲线下面积（AUC）为0.780（95%CI 0.764 - 0.798），而包含24个变量的逻辑回归模型的AUC为0.778（95%CI 0.760 - 0.795）。此外，AutoScore框架通过整合所有必要模块，还推动了临床研究的连续性和自动化。
我们开发了一种易于使用的、基于机器学习 的自动临床评分生成器AutoScore；系统地介绍了其结构；并使用基准数据库证明了其相对于其他传统方法的优越性（预测性能和可解释性）。AutoScore将成为各种医疗应用中的一种潜在评分工具。
