Liu Yushu, Liu Chenxi, Zheng Jianing, Xu Chang, Wang Dan
School of Medicine and Health Management, Huazhong University of Science and Technology, Wuhan, China.
Major Disciplinary Platform under Double First-Class Initiative for Liberal Arts at Huazhong University of Science and Technology (Research Center for High-Quality Development of Hospitals), Wuhan, China.
J Med Internet Res. 2025 Aug 7;27:e73374. doi: 10.2196/73374.
The integration of artificial intelligence (AI) in health care has significant potential, yet its acceptance by health care professionals (HCPs) is essential for successful implementation. Understanding HCPs' perspectives on the explainability and integrability of medical AI is crucial, as these factors influence their willingness to adopt and effectively use such technologies.
This study aims to improve the acceptance and use of medical AI. From a user perspective, it explores HCPs' understanding of the explainability and integrability of medical AI.
We performed a mixed systematic review by conducting a comprehensive search in the PubMed, Web of Science, Scopus, IEEE Xplore, and ACM Digital Library and arXiv databases for studies published between 2014 and 2024. Studies concerning an explanation or the integrability of medical AI were included. Study quality was assessed using the Joanna Briggs Institute critical appraisal checklist and Mixed Methods Appraisal Tool, with only medium- or high-quality studies included. Qualitative data were analyzed via thematic analysis, while quantitative findings were synthesized narratively.
Out of 11,888 records initially retrieved, 22 (0.19%) studies met the inclusion criteria. All selected studies were published from 2020 onward, reflecting the recency and relevance of the topic. The majority (18/22, 82%) originated from high-income countries, and most (17/22, 77%) adopted qualitative methodologies, with the remainder (5/22, 23%) using quantitative or mixed method approaches. From the included studies, a conceptual framework was developed that delineates HCPs' perceptions of explainability and integrability. Regarding explainability, HCPs predominantly emphasized postprocessing explanations, particularly aspects of local explainability such as feature relevance and case-specific outputs. Visual tools that enhance the explainability of AI decisions (eg, heat maps and feature attribution) were frequently mentioned as important enablers of trust and acceptance. For integrability, key concerns included workflow adaptation, system compatibility with electronic health records, and overall ease of use. These aspects were consistently identified as primary conditions for real-world adoption.
To foster wider adoption of AI in clinical settings, future system designs must center on the needs of HCPs. Enhancing post hoc explainability and ensuring seamless integration into existing workflows are critical to building trust and promoting sustained use. The proposed conceptual framework can serve as a practical guide for developers, researchers, and policy makers in aligning AI solutions with frontline user expectations.
PROSPERO CRD420250652253; https://www.crd.york.ac.uk/PROSPERO/view/CRD420250652253.
人工智能（AI）在医疗保健领域的整合具有巨大潜力，然而医疗保健专业人员（HCPs）对其的接受对于成功实施至关重要。了解HCPs对医疗AI的可解释性和可整合性的看法至关重要，因为这些因素会影响他们采用和有效使用此类技术的意愿。
本研究旨在提高医疗AI的接受度和使用率。从用户角度出发，探讨HCPs对医疗AI的可解释性和可整合性的理解。
我们进行了一项混合系统评价，在PubMed、科学网、Scopus、IEEE Xplore、ACM数字图书馆和arXiv数据库中全面检索2014年至2024年发表的研究。纳入有关医疗AI的解释或可整合性的研究。使用乔安娜·布里格斯研究所的批判性评价清单和混合方法评价工具评估研究质量，仅纳入中高质量研究。定性数据通过主题分析进行分析，定量结果进行叙述性综合。
在最初检索的11,888条记录中，22项（0.19%）研究符合纳入标准。所有选定研究均为2020年以后发表，反映了该主题的时效性和相关性。大多数（18/22，82%）来自高收入国家，大多数（17/22，77%）采用定性方法，其余（5/22，23%）采用定量或混合方法。从纳入的研究中，开发了一个概念框架，描绘了HCPs对可解释性和可整合性的看法。关于可解释性，HCPs主要强调后处理解释，特别是局部可解释性的方面，如特征相关性和特定案例输出。增强AI决策可解释性的视觉工具（如热图和特征归因）经常被提及是信任和接受的重要促成因素。对于可整合性，关键问题包括工作流程调整、系统与电子健康记录的兼容性以及总体易用性。这些方面一直被确定为实际应用的主要条件。
为了促进AI在临床环境中的更广泛应用，未来的系统设计必须以HCPs的需求为中心。增强事后可解释性并确保无缝集成到现有工作流程中对于建立信任和促进持续使用至关重要。所提出的概念框架可为开发者、研究人员和政策制定者使AI解决方案符合一线用户期望提供实用指南。
PROSPERO CRD420250652253；https://www.crd.york.ac.uk/PROSPERO/view/CRD420250652253
