Department of Computer Science, University of Toronto, Toronto, Ontario, Canada.
The Hospital for Sick Children, Toronto, Ontario, Canada.
JAMA Netw Open. 2022 Mar 1;5(3):e222599. doi: 10.1001/jamanetworkopen.2022.2599.
Increased wait times and long lengths of stay in emergency departments (EDs) are associated with poor patient outcomes. Systems to improve ED efficiency would be useful. Specifically, minimizing the time to diagnosis by developing novel workflows that expedite test ordering can help accelerate clinical decision-making.
To explore the use of machine learning-based medical directives (MLMDs) to automate diagnostic testing at triage for patients with common pediatric ED diagnoses.
DESIGN, SETTING, AND PARTICIPANTS: Machine learning models trained on retrospective electronic health record data were evaluated in a decision analytical model study conducted at the ED of the Hospital for Sick Children Toronto, Canada. Data were collected on all patients aged 0 to 18 years presenting to the ED from July 1, 2018, to June 30, 2019 (77 219 total patient visits).
Machine learning models were trained to predict the need for urinary dipstick testing, electrocardiogram, abdominal ultrasonography, testicular ultrasonography, bilirubin level testing, and forearm radiographs.
Models were evaluated using area under the receiver operator curve, true-positive rate, false-positive rate, and positive predictive values. Model decision thresholds were determined to limit the total number of false-positive results and achieve high positive predictive values. The time difference between patient triage completion and test ordering was assessed for each use of MLMD. Error rates were analyzed to assess model bias. In addition, model explainability was determined using Shapley Additive Explanations values.
There was a total of 42 238 boys (54.7%) included in model development; mean (SD) age of the children was 5.4 (4.8) years. Models obtained high area under the receiver operator curve (0.89-0.99) and positive predictive values (0.77-0.94) across each of the use cases. The proposed implementation of MLMDs would streamline care for 22.3% of all patient visits and make test results available earlier by 165 minutes (weighted mean) per affected patient. Model explainability for each MLMD demonstrated clinically relevant features having the most influence on model predictions. Models also performed with minimal to no sex bias.
The findings of this study suggest the potential for clinical automation using MLMDs. When integrated into clinical workflows, MLMDs may have the potential to autonomously order common ED tests early in a patient's visit with explainability provided to patients and clinicians.
急诊部（ED）的等待时间延长和住院时间延长与患者预后不良有关。改善 ED 效率的系统将是有用的。具体来说，通过开发加速测试订单的新工作流程来最小化诊断时间，可以帮助加速临床决策。
探索使用基于机器学习的医疗指令 (MLMD) 来自动化儿科 ED 常见诊断的分诊诊断测试。
设计、设置和参与者：在加拿大 SickKids 医院 ED 进行的决策分析模型研究中，评估了基于回顾性电子健康记录数据训练的机器学习模型。数据采集自 2018 年 7 月 1 日至 2019 年 6 月 30 日期间所有 0 至 18 岁就诊 ED 的患者（总共 77219 例患者就诊）。
机器学习模型被训练来预测尿试纸检测、心电图、腹部超声、睾丸超声、胆红素水平检测和前臂 X 光检查的需求。
使用接收者操作特征曲线下面积、真阳性率、假阳性率和阳性预测值评估模型。确定模型决策阈值以限制假阳性结果的总数并实现高阳性预测值。评估了每个 MLMD 使用的患者分诊完成和测试订单之间的时间差。分析错误率以评估模型偏差。此外，使用 Shapley Additive Explanations 值确定模型的可解释性。
模型开发中共有 42238 名男孩（54.7%）；儿童的平均（SD）年龄为 5.4（4.8）岁。在每个使用案例中，模型获得了高接收者操作特征曲线下面积（0.89-0.99）和阳性预测值（0.77-0.94）。拟议的 MLMD 实施将使 22.3%的所有患者就诊流程简化，并使受影响患者的测试结果提前 165 分钟（加权平均值）可用。每个 MLMD 的模型可解释性表明，对模型预测最有影响的是临床相关特征。模型还表现出最小的性别偏差或没有性别偏差。
这项研究的结果表明，使用 MLMD 进行临床自动化的潜力。当集成到临床工作流程中时，MLMD 可能有潜力在患者就诊早期自主安排常见的 ED 测试，并向患者和临床医生提供可解释性。
