Rosenbacke Rikard, Melhus Åsa, McKee Martin, Stuckler David
Centre for Corporate Governance, Department of Accounting, Copenhagen Business School, Frederiksberg, Denmark.
Department of Medical Sciences, Clinical Microbiology, Uppsala University, Uppsala, Sweden.
JMIR AI. 2024 Oct 30;3:e53207. doi: 10.2196/53207.
Artificial intelligence (AI) has significant potential in clinical practice. However, its "black box" nature can lead clinicians to question its value. The challenge is to create sufficient trust for clinicians to feel comfortable using AI, but not so much that they defer to it even when it produces results that conflict with their clinical judgment in ways that lead to incorrect decisions. Explainable AI (XAI) aims to address this by providing explanations of how AI algorithms reach their conclusions. However, it remains unclear whether such explanations foster an appropriate degree of trust to ensure the optimal use of AI in clinical practice.
This study aims to systematically review and synthesize empirical evidence on the impact of XAI on clinicians' trust in AI-driven clinical decision-making.
A systematic review was conducted in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, searching PubMed and Web of Science databases. Studies were included if they empirically measured the impact of XAI on clinicians' trust using cognition- or affect-based measures. Out of 778 articles screened, 10 met the inclusion criteria. We assessed the risk of bias using standard tools appropriate to the methodology of each paper.
The risk of bias in all papers was moderate or moderate to high. All included studies operationalized trust primarily through cognitive-based definitions, with 2 also incorporating affect-based measures. Out of these, 5 studies reported that XAI increased clinicians' trust compared with standard AI, particularly when the explanations were clear, concise, and relevant to clinical practice. In addition, 3 studies found no significant effect of XAI on trust, and the presence of explanations does not automatically improve trust. Notably, 2 studies highlighted that XAI could either enhance or diminish trust, depending on the complexity and coherence of the provided explanations. The majority of studies suggest that XAI has the potential to enhance clinicians' trust in recommendations generated by AI. However, complex or contradictory explanations can undermine this trust. More critically, trust in AI is not inherently beneficial, as AI recommendations are not infallible. These findings underscore the nuanced role of explanation quality and suggest that trust can be modulated through the careful design of XAI systems.
Excessive trust in incorrect advice generated by AI can adversely impact clinical accuracy, just as can happen when correct advice is distrusted. Future research should focus on refining both cognitive and affect-based measures of trust and on developing strategies to achieve an appropriate balance in terms of trust, preventing both blind trust and undue skepticism. Optimizing trust in AI systems is essential for their effective integration into clinical practice.
人工智能（AI）在临床实践中具有巨大潜力。然而，其“黑箱”性质可能导致临床医生质疑其价值。挑战在于建立足够的信任，使临床医生在使用AI时感到安心，但又不能过度信任，以至于即使AI产生的结果与他们的临床判断相冲突并导致错误决策时，他们仍盲目听从。可解释人工智能（XAI）旨在通过解释AI算法如何得出结论来解决这一问题。然而，尚不清楚此类解释是否能培养出适当程度的信任，以确保AI在临床实践中的最佳应用。
本研究旨在系统回顾和综合关于XAI对临床医生对AI驱动的临床决策信任度影响的实证证据。
按照PRISMA（系统评价和Meta分析的首选报告项目）指南进行系统回顾，检索PubMed和Web of Science数据库。如果研究通过基于认知或情感的测量方法实证测量了XAI对临床医生信任度的影响，则纳入研究。在筛选的778篇文章中，有10篇符合纳入标准。我们使用适合每篇论文方法的标准工具评估偏倚风险。
所有论文的偏倚风险为中度或中度至高度。所有纳入研究主要通过基于认知的定义来衡量信任度，其中2项研究还纳入了基于情感的测量方法。其中，5项研究报告称，与标准AI相比，XAI提高了临床医生的信任度，特别是当解释清晰、简洁且与临床实践相关时。此外，3项研究发现XAI对信任度没有显著影响，解释的存在并不会自动提高信任度。值得注意的是，2项研究强调，XAI根据所提供解释的复杂性和连贯性，既可以增强也可以削弱信任度。大多数研究表明，XAI有潜力增强临床医生对AI生成的建议的信任。然而，复杂或矛盾的解释可能会破坏这种信任。更关键的是，对AI的信任本身并不一定有益，因为AI的建议并非绝对可靠。这些发现强调了解释质量的细微差别作用，并表明可以通过精心设计XAI系统来调节信任度。
对AI给出的错误建议过度信任可能会对临床准确性产生不利影响，就像不信任正确建议时一样。未来的研究应专注于完善基于认知和情感的信任度测量方法，并制定策略以在信任方面实现适当平衡，防止盲目信任和过度怀疑。优化对AI系统的信任对于它们有效融入临床实践至关重要。
