School of Computer Science and Information Systems, Pace University, New York, USA.
IBM Research, Cambridge, USA.
J Med Syst. 2021 May 4;45(6):64. doi: 10.1007/s10916-021-01743-6.
Ongoing research efforts have been examining how to utilize artificial intelligence technology to help healthcare consumers make sense of their clinical data, such as diagnostic radiology reports. How to promote the acceptance of such novel technology is a heated research topic. Recent studies highlight the importance of providing local explanations about AI prediction and model performance to help users determine whether to trust AI's predictions. Despite some efforts, limited empirical research has been conducted to quantitatively measure how AI explanations impact healthcare consumers' perceptions of using patient-facing, AI-powered healthcare systems. The aim of this study is to evaluate the effects of different AI explanations on people's perceptions of AI-powered healthcare system. In this work, we designed and deployed a large-scale experiment (N = 3,423) on Amazon Mechanical Turk (MTurk) to evaluate the effects of AI explanations on people's perceptions in the context of comprehending radiology reports. We created four groups based on two factors-the extent of explanations for the prediction (High vs. Low Transparency) and the model performance (Good vs. Weak AI Model)-and randomly assigned participants to one of the four conditions. Participants were instructed to classify a radiology report as describing a normal or abnormal finding, followed by completing a post-study survey to indicate their perceptions of the AI tool. We found that revealing model performance information can promote people's trust and perceived usefulness of system outputs, while providing local explanations for the rationale of a prediction can promote understandability but not necessarily trust. We also found that when model performance is low, the more information the AI system discloses, the less people would trust the system. Lastly, whether human agrees with AI predictions or not and whether the AI prediction is correct or not could also influence the effect of AI explanations. We conclude this paper by discussing implications for designing AI systems for healthcare consumers to interpret diagnostic report.
目前的研究工作旨在探讨如何利用人工智能技术帮助医疗保健消费者理解其临床数据，如诊断影像学报告。如何促进人们对这种新技术的接受是一个热门的研究课题。最近的研究强调了提供有关人工智能预测和模型性能的本地解释的重要性，以帮助用户确定是否信任人工智能的预测。尽管已经做了一些努力，但很少有实证研究来定量衡量人工智能解释如何影响医疗保健消费者对面向患者的、人工智能驱动的医疗保健系统的使用感知。本研究旨在评估不同人工智能解释对人们对人工智能驱动的医疗保健系统感知的影响。在这项工作中，我们在亚马逊 Mechanical Turk（MTurk）上设计并部署了一项大规模实验（N=3423），以评估在理解影像学报告的背景下，人工智能解释对人们感知的影响。我们根据两个因素（预测的解释程度（高透明度与低透明度）和模型性能（强人工智能模型与弱人工智能模型））创建了四个组，并随机将参与者分配到四个条件之一。参与者被指示对一份影像学报告进行分类，判断其描述的是正常还是异常结果，然后完成一项研究后调查，以表明他们对人工智能工具的感知。我们发现，揭示模型性能信息可以增强人们对系统输出的信任和感知有用性，而提供预测的本地解释可以提高可理解性，但不一定能增强信任。我们还发现，当模型性能较低时，人工智能系统披露的信息越多，人们对系统的信任度就越低。最后，无论人工智能是否与人类的判断一致，以及人工智能的预测是否正确，都可能影响人工智能解释的效果。我们通过讨论设计面向医疗保健消费者的人工智能系统以解释诊断报告的意义来结束本文。
