Senoner Julian, Schallmoser Simon, Kratzwald Bernhard, Feuerriegel Stefan, Netland Torbjørn
ETH Zurich, Zurich, Switzerland.
EthonAI, Zurich, Switzerland.
Sci Rep. 2024 Dec 28;14(1):31150. doi: 10.1038/s41598-024-82501-9.
Artificial intelligence (AI) provides considerable opportunities to assist human work. However, one crucial challenge of human-AI collaboration is that many AI algorithms operate in a black-box manner where the way how the AI makes predictions remains opaque. This makes it difficult for humans to validate a prediction made by AI against their own domain knowledge. For this reason, we hypothesize that augmenting humans with explainable AI improves task performance in human-AI collaboration. To test this hypothesis, we implement explainable AI in the form of visual heatmaps in inspection tasks conducted by domain experts. Visual heatmaps have the advantage that they are easy to understand and help to localize relevant parts of an image. We then compare participants that were either supported by (a) black-box AI or (b) explainable AI, where the latter supports them to follow AI predictions when the AI is accurate or overrule the AI when the AI predictions are wrong. We conducted two preregistered experiments with representative, real-world visual inspection tasks from manufacturing and medicine. The first experiment was conducted with factory workers from an electronics factory, who performed [Formula: see text] assessments of whether electronic products have defects. The second experiment was conducted with radiologists, who performed [Formula: see text] assessments of chest X-ray images to identify lung lesions. The results of our experiments with domain experts performing real-world tasks show that task performance improves when participants are supported by explainable AI with heatmaps instead of black-box AI. We find that explainable AI as a decision aid improved the task performance by 7.7 percentage points (95% confidence interval [CI]: 3.3% to 12.0%, [Formula: see text]) in the manufacturing experiment and by 4.7 percentage points (95% CI: 1.1% to 8.3%, [Formula: see text]) in the medical experiment compared to black-box AI. These gains represent a significant improvement in task performance.
人工智能（AI）为协助人类工作提供了大量机会。然而，人机协作面临的一个关键挑战是，许多AI算法以黑箱方式运行，AI做出预测的方式仍然不透明。这使得人类难以根据自身领域知识来验证AI做出的预测。因此，我们假设，在人机协作中，用可解释AI增强人类能力可提高任务绩效。为了验证这一假设，我们在领域专家进行的检查任务中，以视觉热图的形式实现了可解释AI。视觉热图具有易于理解且有助于定位图像相关部分的优点。然后，我们比较了分别由（a）黑箱AI或（b）可解释AI提供支持的参与者，其中后者在AI预测准确时帮助他们遵循AI的预测，而在AI预测错误时则允许他们推翻AI的预测。我们针对制造业和医学领域具有代表性的实际视觉检查任务进行了两项预先注册的实验。第一个实验是与一家电子厂的工人合作进行的，他们对电子产品是否有缺陷进行[公式：见正文]评估。第二个实验是与放射科医生合作进行的，他们对胸部X光图像进行[公式：见正文]评估以识别肺部病变。我们与执行实际任务的领域专家进行的实验结果表明，当参与者由带有热图的可解释AI而不是黑箱AI提供支持时，任务绩效会提高。我们发现，作为决策辅助工具的可解释AI在制造业实验中比黑箱AI将任务绩效提高了7.7个百分点（95%置信区间[CI]：3.3%至12.0%，[公式：见正文]），在医学实验中提高了4.7个百分点（95%CI：1.1%至8.3%，[公式：见正文]）。这些提升代表了任务绩效的显著改善。
