John Radcliffe Hospital, Oxford University Hospitals NHS Foundation Trust, Oxford, UK; Division of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, Oxford, UK.
Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK.
Lancet Digit Health. 2022 Apr;4(4):e266-e278. doi: 10.1016/S2589-7500(21)00272-7. Epub 2022 Mar 9.
Uncertainty in patients' COVID-19 status contributes to treatment delays, nosocomial transmission, and operational pressures in hospitals. However, the typical turnaround time for laboratory PCR remains 12-24 h and lateral flow devices (LFDs) have limited sensitivity. Previously, we have shown that artificial intelligence-driven triage (CURIAL-1.0) can provide rapid COVID-19 screening using clinical data routinely available within 1 h of arrival to hospital. Here, we aimed to improve the time from arrival to the emergency department to the availability of a result, do external and prospective validation, and deploy a novel laboratory-free screening tool in a UK emergency department.
We optimised our previous model, removing less informative predictors to improve generalisability and speed, developing the CURIAL-Lab model with vital signs and readily available blood tests (full blood count [FBC]; urea, creatinine, and electrolytes; liver function tests; and C-reactive protein) and the CURIAL-Rapide model with vital signs and FBC alone. Models were validated externally for emergency admissions to University Hospitals Birmingham, Bedfordshire Hospitals, and Portsmouth Hospitals University National Health Service (NHS) trusts, and prospectively at Oxford University Hospitals, by comparison with PCR testing. Next, we compared model performance directly against LFDs and evaluated a combined pathway that triaged patients who had either a positive CURIAL model result or a positive LFD to a COVID-19-suspected clinical area. Lastly, we deployed CURIAL-Rapide alongside an approved point-of-care FBC analyser to provide laboratory-free COVID-19 screening at the John Radcliffe Hospital (Oxford, UK). Our primary improvement outcome was time-to-result, and our performance measures were sensitivity, specificity, positive and negative predictive values, and area under receiver operating characteristic curve (AUROC).
72 223 patients met eligibility criteria across the four validating hospital groups, in a total validation period spanning Dec 1, 2019, to March 31, 2021. CURIAL-Lab and CURIAL-Rapide performed consistently across trusts (AUROC range 0·858-0·881, 95% CI 0·838-0·912, for CURIAL-Lab and 0·836-0·854, 0·814-0·889, for CURIAL-Rapide), achieving highest sensitivity at Portsmouth Hospitals (84·1%, Wilson's 95% CI 82·5-85·7, for CURIAL-Lab and 83·5%, 81·8-85·1, for CURIAL-Rapide) at specificities of 71·3% (70·9-71·8) for CURIAL-Lab and 63·6% (63·1-64·1) for CURIAL-Rapide. When combined with LFDs, model predictions improved triage sensitivity from 56·9% (51·7-62·0) for LFDs alone to 85·6% with CURIAL-Lab (81·6-88·9; AUROC 0·925) and 88·2% with CURIAL-Rapide (84·4-91·1; AUROC 0·919), thereby reducing missed COVID-19 cases by 65% with CURIAL-Lab and 72% with CURIAL-Rapide. For the prospective deployment of CURIAL-Rapide, 520 patients were enrolled for point-of-care FBC analysis between Feb 18 and May 10, 2021, of whom 436 received confirmatory PCR testing and ten (2·3%) tested positive. Median time from arrival to a CURIAL-Rapide result was 45 min (IQR 32-64), 16 min (26·3%) sooner than with LFDs (61 min, 37-99; log-rank p<0·0001), and 6 h 52 min (90·2%) sooner than with PCR (7 h 37 min, 6 h 5 min to 15 h 39 min; p<0·0001). Classification performance was high, with sensitivity of 87·5% (95% CI 52·9-97·8), specificity of 85·4% (81·3-88·7), and negative predictive value of 99·7% (98·2-99·9). CURIAL-Rapide correctly excluded infection for 31 (58·5%) of 53 patients who were triaged by a physician to a COVID-19-suspected area but went on to test negative by PCR.
Our findings show the generalisability, performance, and real-world operational benefits of artificial intelligence-driven screening for COVID-19 over standard-of-care in emergency departments. CURIAL-Rapide provided rapid, laboratory-free screening when used with near-patient FBC analysis, and was able to reduce the number of patients who tested negative for COVID-19 but were triaged to COVID-19-suspected areas.
The Wellcome Trust, University of Oxford Medical and Life Sciences Translational Fund.
患者 COVID-19 状态的不确定性导致治疗延误、医院内的继发感染和运营压力。然而，实验室 PCR 的典型周转时间仍为 12-24 小时，而侧向流动设备（LFD）的灵敏度有限。之前，我们已经表明，人工智能驱动的分诊（CURIAL-1.0）可以使用患者到达医院后 1 小时内常规获得的临床数据提供快速的 COVID-19 筛查。在这里，我们旨在提高从到达急诊室到获得结果的时间，进行外部和前瞻性验证，并在英国急诊室部署一种新的无实验室筛查工具。
我们优化了之前的模型，删除了不太有信息的预测因素以提高通用性和速度，开发了包含生命体征和易于获得的血液测试（全血细胞计数[FBC]；尿素、肌酐和电解质；肝功能测试；C 反应蛋白）的 CURIAL-Lab 模型，以及包含生命体征和 FBC 的 CURIAL-Rapide 模型。通过与 PCR 检测比较，在英国伯明翰大学医院、贝德福德郡医院和朴茨茅斯医院 NHS 信托基金进行了外部验证，在牛津大学医院进行了前瞻性验证。接下来，我们直接比较了模型性能与 LFD，并评估了一种联合途径，该途径将 CURIAL 模型结果阳性或 LFD 阳性的患者分诊到 COVID-19 疑似临床区域。最后，我们在牛津大学约翰拉德克利夫医院（英国牛津）部署了 CURIAL-Rapide 与经批准的即时 FBC 分析仪一起提供无实验室 COVID-19 筛查。我们的主要改进结果是获得结果的时间，我们的性能指标包括敏感性、特异性、阳性和阴性预测值以及接收者操作特征曲线下的面积（AUROC）。
在总共涵盖 2019 年 12 月 1 日至 2021 年 3 月 31 日的四个验证医院组中，共有 72223 名患者符合纳入标准。CURIAL-Lab 和 CURIAL-Rapide 在各信托基金中表现一致（AUROC 范围为 0.858-0.881，95%CI 为 0.838-0.912，用于 CURIAL-Lab 和 0.836-0.854，95%CI 为 0.814-0.889，用于 CURIAL-Rapide），在朴茨茅斯医院达到最高敏感性（84.1%，Wilson 95%CI 为 82.5-85.7，用于 CURIAL-Lab 和 83.5%，81.8-85.1，用于 CURIAL-Rapide），特异性为 71.3%（70.9-71.8），用于 CURIAL-Lab 和 63.6%（63.1-64.1），用于 CURIAL-Rapide。当与 LFD 结合使用时，模型预测通过将 LFD 单独使用时的分诊敏感性从 56.9%（51.7-62.0）提高到 CURIAL-Lab 的 85.6%（81.6-88.9；AUROC 为 0.925）和 CURIAL-Rapide 的 88.2%（84.4-91.1；AUROC 为 0.919），从而使 CURIAL-Lab 减少了 65%的 COVID-19 漏诊病例，使 CURIAL-Rapide 减少了 72%的 COVID-19 漏诊病例。对于 CURIAL-Rapide 的前瞻性部署，在 2021 年 2 月 18 日至 5 月 10 日期间，有 520 名患者接受了即时 FBC 分析，其中 436 名患者接受了确认性 PCR 检测，10 名（2.3%）检测呈阳性。从到达到 CURIAL-Rapide 结果的中位时间为 45 分钟（IQR 32-64），比 LFD 快 16 分钟（26.3%）（61 分钟，37-99；对数秩检验 p<0.0001），比 PCR 快 6 小时 52 分钟（7 小时 37 分钟，6 小时 5 分钟至 15 小时 39 分钟；p<0.0001）。分类性能很高，敏感性为 87.5%（95%CI 为 52.9-97.8），特异性为 85.4%（81.3-88.7），阴性预测值为 99.7%（98.2-99.9）。CURIAL-Rapide 正确排除了 31 名（58.5%）医生分诊到 COVID-19 疑似区域但 PCR 检测结果为阴性的患者的感染。
我们的研究结果表明，人工智能驱动的 COVID-19 筛查在急诊室中具有普遍性、性能和实际操作优势，超过了标准护理。CURIAL-Rapide 与即时 FBC 分析结合使用时提供了快速的无实验室筛查，并且能够减少 PCR 检测为阴性但被分诊到 COVID-19 疑似区域的患者数量。
惠康信托基金，牛津大学医学和生命科学转化基金。
