医学影像中人工智能的风险评估与缓解措施——一项关于实施乳腺钼靶筛查独立人工智能的定性研究

Risk inventory and mitigation actions for AI in medical imaging-a qualitative study of implementing standalone AI for screening mammography.

作者信息

Gerigoorian Annika, Kloub Maha, Dembrower Karin, Engwall Mats, Strand Fredrik

机构信息

Department of Industrial Economics and Management, KTH Royal Institute of Technology, Stockholm, Sweden.

Department of Oncology-Pathology, Karolinska Institutet, Anna Steckséns gata 30A, Stockholm, SE 171 64, Sweden.

出版信息

BMC Health Serv Res. 2025 Jul 30;25(1):998. doi: 10.1186/s12913-025-13176-9.

DOI:10.1186/s12913-025-13176-9

PMID:40731348

原文链接:https://pmc.ncbi.nlm.nih.gov/articles/PMC12309172/

Abstract

BACKGROUND

Recent prospective studies have shown that AI may be integrated in double-reader settings to increase cancer detection. The ScreenTrustCAD study was conducted at the breast radiology department at the Capio S:t Göran Hospital where AI is now implemented in clinical practice. This study reports on how the hospital prepared by exploring risks from an enterprise risk management perspective, i.e., applying a holistic and proactive perspective, and developed risk mitigation actions.

METHOD

The study was conducted as an integral part of the preparations before implementing AI in a breast imaging department. Collaborative ideation sessions were conducted with personnel at the hospital, either directly or indirectly involved with AI, to identify risks. Two external experts with competencies in cybersecurity, machine learning, and the ethical aspects of AI, were interviewed as a complement. The risks identified were analyzed according to an Enterprise Risk Management framework, adopted for healthcare, that assumes risks to be emerging from eight different domains. Finally, appropriate risk mitigation actions were identified and discussed.

FINDINGS

Twenty-three risks were identified covering seven of eight risk domains, in turn generating 51 suggested risk mitigation actions. Not only does the study indicate the emergence of patient safety risks, but it also shows that there are operational, strategic, financial, human capital, legal, and technological risks. The risks with most suggested mitigation actions were ‘Radiographers unable to answer difficult questions from patients’, ‘Increased risk that patient-reported symptoms are missed by the single radiologist’, ‘Increased pressure on the single reader knowing they are the only radiologist to catch a mistake by AI’, and ‘The performance of the AI algorithm might deteriorate’.

CONCLUSION

Before a clinical integration of AI, hospitals should expand, identify, and address risks beyond immediate patient safety by applying comprehensive and proactive risk management.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1186/s12913-025-13176-9.

摘要

背景

近期的前瞻性研究表明，人工智能（AI）可整合到双读模式中以提高癌症检测率。ScreenTrustCAD研究在卡皮奥圣戈兰医院的乳腺放射科开展，目前AI已在该医院的临床实践中得到应用。本研究报告了该医院如何从企业风险管理角度，即运用整体和前瞻性视角，探索风险并制定风险缓解措施。

方法

该研究是在乳腺影像科室实施AI之前准备工作的一个组成部分。与医院中直接或间接参与AI工作的人员进行协作构思会议，以识别风险。作为补充，还采访了两位在网络安全、机器学习以及AI伦理方面具有专业能力的外部专家。根据适用于医疗保健领域的企业风险管理框架对识别出的风险进行分析，该框架假定风险源自八个不同领域。最后，确定并讨论了适当的风险缓解措施。

结果

共识别出23项风险，涵盖八个风险领域中的七个，进而产生了51项建议的风险缓解措施。该研究不仅表明出现了患者安全风险，还显示存在运营、战略、财务、人力资本、法律和技术风险。建议采取缓解措施最多的风险包括“放射技师无法回答患者的难题”、“单一放射科医生漏诊患者报告症状的风险增加”、“单一读片医生知道自己是唯一能发现AI错误的人而面临的压力增加”以及“AI算法的性能可能会下降”。