Department of Computer Science, Systems and Communication, University of Milano-Bicocca, Milan, Italy; IRCCS Istituto Ortopedico Galeazzi, Milan, Italy.
IRCCS Istituto Ortopedico Galeazzi, Milan, Italy.
Artif Intell Med. 2023 Apr;138:102506. doi: 10.1016/j.artmed.2023.102506. Epub 2023 Feb 8.
In this paper, we study human-AI collaboration protocols, a design-oriented construct aimed at establishing and evaluating how humans and AI can collaborate in cognitive tasks. We applied this construct in two user studies involving 12 specialist radiologists (the knee MRI study) and 44 ECG readers of varying expertise (the ECG study), who evaluated 240 and 20 cases, respectively, in different collaboration configurations. We confirm the utility of AI support but find that XAI can be associated with a "white-box paradox", producing a null or detrimental effect. We also find that the order of presentation matters: AI-first protocols are associated with higher diagnostic accuracy than human-first protocols, and with higher accuracy than both humans and AI alone. Our findings identify the best conditions for AI to augment human diagnostic skills, rather than trigger dysfunctional responses and cognitive biases that can undermine decision effectiveness.
在本文中，我们研究了人机协作协议，这是一种面向设计的构建，旨在建立和评估人类和 AI 如何在认知任务中协作。我们在两项用户研究中应用了这一构建，涉及 12 名专业放射科医生（膝关节 MRI 研究）和 44 名不同专业水平的心电图阅读器（心电图研究），他们分别在不同的协作配置下评估了 240 个和 20 个病例。我们证实了 AI 支持的效用，但发现 XAI 可能与“白盒悖论”相关联，产生无效或有害的效果。我们还发现呈现顺序很重要：AI 优先协议与人类优先协议相比，与人类和 AI 单独相比，都与更高的诊断准确性相关。我们的研究结果确定了 AI 增强人类诊断技能的最佳条件，而不是触发可能破坏决策效果的功能失调反应和认知偏差。
