Patel Bhavik N, Rosenberg Louis, Willcox Gregg, Baltaxe David, Lyons Mimi, Irvin Jeremy, Rajpurkar Pranav, Amrhein Timothy, Gupta Rajan, Halabi Safwan, Langlotz Curtis, Lo Edward, Mammarappallil Joseph, Mariano A J, Riley Geoffrey, Seekins Jayne, Shen Luyao, Zucker Evan, Lungren Matthew
1Department of Radiology, Stanford University School of Medicine, 300 Pasteur Dr., H1307, Stanford, CA 94305 USA.
Unanimous AI, 2443 Fillmore Street #116, San Francisco, CA 94115-1814 USA.
NPJ Digit Med. 2019 Nov 18;2:111. doi: 10.1038/s41746-019-0189-7. eCollection 2019.
Human-in-the-loop (HITL) AI may enable an ideal symbiosis of human experts and AI models, harnessing the advantages of both while at the same time overcoming their respective limitations. The purpose of this study was to investigate a novel collective intelligence technology designed to amplify the diagnostic accuracy of networked human groups by forming real-time systems modeled on biological swarms. Using small groups of radiologists, the swarm-based technology was applied to the diagnosis of pneumonia on chest radiographs and compared against human experts alone, as well as two state-of-the-art deep learning AI models. Our work demonstrates that both the swarm-based technology and deep-learning technology achieved superior diagnostic accuracy than the human experts alone. Our work further demonstrates that when used in combination, the swarm-based technology and deep-learning technology outperformed either method alone. The superior diagnostic accuracy of the combined HITL AI solution compared to radiologists and AI alone has broad implications for the surging clinical AI deployment and implementation strategies in future practice.
人在回路（HITL）人工智能可以实现人类专家与人工智能模型的理想共生，利用两者的优势，同时克服它们各自的局限性。本研究的目的是调查一种新型集体智能技术，该技术旨在通过构建基于生物群体的实时系统来提高联网人类群体的诊断准确性。利用一小群放射科医生，将基于群体的技术应用于胸部X光片上肺炎的诊断，并与单独的人类专家以及两种最先进的深度学习人工智能模型进行比较。我们的工作表明，基于群体的技术和深度学习技术的诊断准确性均优于单独的人类专家。我们的工作进一步表明，当两者结合使用时，基于群体的技术和深度学习技术的表现优于单独使用任何一种方法。与放射科医生和单独的人工智能相比，组合式人在回路人工智能解决方案的卓越诊断准确性对未来临床人工智能的激增部署和实施策略具有广泛影响。
