移动人工智能症状检查器和基于网络的自我转诊工具在风湿病学中的诊断准确性：多中心随机对照试验。
Diagnostic Accuracy of a Mobile AI-Based Symptom Checker and a Web-Based Self-Referral Tool in Rheumatology: Multicenter Randomized Controlled Trial.
机构信息
Institute for Digital Medicine, University Hospital Giessen-Marburg, Philipps University Marburg, Marburg, Germany.
AGEIS, Université Grenoble Alpes, Grenoble, France.
出版信息
J Med Internet Res. 2024 Jul 23;26:e55542. doi: 10.2196/55542.
BACKGROUND
The diagnosis of inflammatory rheumatic diseases (IRDs) is often delayed due to unspecific symptoms and a shortage of rheumatologists. Digital diagnostic decision support systems (DDSSs) have the potential to expedite diagnosis and help patients navigate the health care system more efficiently.
OBJECTIVE
The aim of this study was to assess the diagnostic accuracy of a mobile artificial intelligence (AI)-based symptom checker (Ada) and a web-based self-referral tool (Rheport) regarding IRDs.
METHODS
A prospective, multicenter, open-label, crossover randomized controlled trial was conducted with patients newly presenting to 3 rheumatology centers. Participants were randomly assigned to complete a symptom assessment using either Ada or Rheport. The primary outcome was the correct identification of IRDs by the DDSSs, defined as the presence of any IRD in the list of suggested diagnoses by Ada or achieving a prespecified threshold score with Rheport. The gold standard was the diagnosis made by rheumatologists.
RESULTS
A total of 600 patients were included, among whom 214 (35.7%) were diagnosed with an IRD. Most frequent IRD was rheumatoid arthritis with 69 (11.5%) patients. Rheport's disease suggestion and Ada's top 1 (D1) and top 5 (D5) disease suggestions demonstrated overall diagnostic accuracies of 52%, 63%, and 58%, respectively, for IRDs. Rheport showed a sensitivity of 62% and a specificity of 47% for IRDs. Ada's D1 and D5 disease suggestions showed a sensitivity of 52% and 66%, respectively, and a specificity of 68% and 54%, respectively, concerning IRDs. Ada's diagnostic accuracy regarding individual diagnoses was heterogenous, and Ada performed considerably better in identifying rheumatoid arthritis in comparison to other diagnoses (D1: 42%; D5: 64%). The Cohen κ statistic of Rheport for agreement on any rheumatic disease diagnosis with Ada D1 was 0.15 (95% CI 0.08-0.18) and with Ada D5 was 0.08 (95% CI 0.00-0.16), indicating poor agreement for the presence of any rheumatic disease between the 2 DDSSs.
CONCLUSIONS
To our knowledge, this is the largest comparative DDSS trial with actual use of DDSSs by patients. The diagnostic accuracies of both DDSSs for IRDs were not promising in this high-prevalence patient population. DDSSs may lead to a misuse of scarce health care resources. Our results underscore the need for stringent regulation and drastic improvements to ensure the safety and efficacy of DDSSs.
TRIAL REGISTRATION
German Register of Clinical Trials DRKS00017642; https://drks.de/search/en/trial/DRKS00017642.
背景
由于症状不特异和风湿科医生短缺，炎症性风湿病（IRD）的诊断常常被延误。数字诊断决策支持系统（DDSS）有可能加快诊断速度，并帮助患者更有效地利用医疗保健系统。
目的
本研究旨在评估一种移动人工智能（AI）为基础的症状检查器（Ada）和一种基于网络的自我转诊工具（Rheport）在 IRD 方面的诊断准确性。
方法
这是一项前瞻性、多中心、开放标签、交叉随机对照试验，纳入了 3 家风湿病中心新就诊的患者。参与者被随机分配使用 Ada 或 Rheport 完成症状评估。主要结局是 DDSSs 正确识别 IRD，定义为 Ada 提示的任何 IRD 或 Rheport 达到预定的阈值分数。金标准是风湿病医生做出的诊断。
结果
共纳入 600 例患者，其中 214 例（35.7%）被诊断为 IRD。最常见的 IRD 是类风湿关节炎，有 69 例（11.5%）患者。Rheport 的疾病提示和 Ada 的前 1 名（D1）和前 5 名（D5）疾病提示对 IRD 的总体诊断准确性分别为 52%、63%和 58%。Rheport 对 IRD 的敏感性为 62%，特异性为 47%。Ada 的 D1 和 D5 疾病提示对 IRD 的敏感性分别为 52%和 66%，特异性分别为 68%和 54%。Ada 对个别诊断的诊断准确性存在差异，与其他诊断相比，Ada 对类风湿关节炎的识别能力要好得多（D1：42%；D5：64%）。Rheport 对 Ada D1 任何风湿性疾病诊断的 Cohen κ 统计量为 0.15（95%CI 0.08-0.18），对 Ada D5 的 Cohen κ 统计量为 0.08（95%CI 0.00-0.16），这表明这两种 DDSSs 对任何风湿性疾病的诊断一致性较差。
结论
据我们所知，这是最大的比较性 DDSS 试验，患者实际使用了 DDSS。在这一高患病率患者群体中，这两种 DDSS 对 IRD 的诊断准确性都不太理想。DDSS 可能导致稀缺医疗资源的滥用。我们的研究结果强调了需要严格监管和大幅度改进，以确保 DDSS 的安全性和有效性。
试验注册
德国临床试验注册处 DRKS00017642；https://drks.de/search/en/trial/DRKS00017642。
Zotero 插件