文献检索文档翻译深度研究
Suppr Zotero 插件Zotero 插件
邀请有礼套餐&价格历史记录

新学期,新优惠

限时优惠:9月1日-9月22日

30天高级会员仅需29元

1天体验卡首发特惠仅需5.99元

了解详情
不再提醒
插件&应用
Suppr Zotero 插件Zotero 插件浏览器插件Mac 客户端Windows 客户端微信小程序
高级版
套餐订阅购买积分包
AI 工具
文献检索文档翻译深度研究
关于我们
关于 Suppr公司介绍联系我们用户协议隐私条款
关注我们

Suppr 超能文献

核心技术专利:CN118964589B侵权必究
粤ICP备2023148730 号-1Suppr @ 2025

眼动引导的多模态融合:迈向使用可解释人工智能的自适应学习框架

Eye-Guided Multimodal Fusion: Toward an Adaptive Learning Framework Using Explainable Artificial Intelligence.

作者信息

Moradizeyveh Sahar, Hanif Ambreen, Liu Sidong, Qi Yuankai, Beheshti Amin, Di Ieva Antonio

机构信息

Computational NeuroSurgery (CNS) Lab, Macquarie Medical School, Faculty of Medicine, Health and Human Sciences, Macquarie University, Sydney 2113, Australia.

Centre for Applied Artificial Intelligence, School of Computing, Faculty of Science and Engineering, Macquarie University, Sydney 2113, Australia.

出版信息

Sensors (Basel). 2025 Jul 24;25(15):4575. doi: 10.3390/s25154575.


DOI:10.3390/s25154575
PMID:40807742
原文链接:https://pmc.ncbi.nlm.nih.gov/articles/PMC12349219/
Abstract

Interpreting diagnostic imaging and identifying clinically relevant features remain challenging tasks, particularly for novice radiologists who often lack structured guidance and expert feedback. To bridge this gap, we propose an Eye-Gaze Guided Multimodal Fusion framework that leverages expert eye-tracking data to enhance learning and decision-making in medical image interpretation. By integrating chest X-ray (CXR) images with expert fixation maps, our approach captures radiologists' visual attention patterns and highlights regions of interest (ROIs) critical for accurate diagnosis. The fusion model utilizes a shared backbone architecture to jointly process image and gaze modalities, thereby minimizing the impact of noise in fixation data. We validate the system's interpretability using Gradient-weighted Class Activation Mapping (Grad-CAM) and assess both classification performance and explanation alignment with expert annotations. Comprehensive evaluations, including robustness under gaze noise and expert clinical review, demonstrate the framework's effectiveness in improving model reliability and interpretability. This work offers a promising pathway toward intelligent, human-centered AI systems that support both diagnostic accuracy and medical training.

摘要

解读诊断成像并识别临床相关特征仍然是具有挑战性的任务,特别是对于那些常常缺乏结构化指导和专家反馈的新手放射科医生而言。为了弥补这一差距,我们提出了一种眼动引导的多模态融合框架,该框架利用专家眼动追踪数据来加强医学图像解读中的学习和决策。通过将胸部X光(CXR)图像与专家注视图相结合,我们的方法捕捉放射科医生的视觉注意力模式,并突出显示对准确诊断至关重要的感兴趣区域(ROI)。融合模型利用共享骨干架构来联合处理图像和注视模态,从而将注视数据中的噪声影响降至最低。我们使用梯度加权类激活映射(Grad-CAM)验证系统的可解释性,并评估分类性能以及与专家注释的解释一致性。包括注视噪声下的稳健性和专家临床审查在内的综合评估证明了该框架在提高模型可靠性和可解释性方面的有效性。这项工作为支持诊断准确性和医学培训的智能、以人为本的人工智能系统提供了一条充满希望的途径。

https://cdn.ncbi.nlm.nih.gov/pmc/blobs/af38/12349219/b886397b40c5/sensors-25-04575-g007.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/af38/12349219/78f5b720cd22/sensors-25-04575-g001.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/af38/12349219/1826de99c4c0/sensors-25-04575-g002.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/af38/12349219/2b1dea418105/sensors-25-04575-g003.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/af38/12349219/f6e3fede11f3/sensors-25-04575-g004.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/af38/12349219/d2748506e2af/sensors-25-04575-g005.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/af38/12349219/9153071f5b02/sensors-25-04575-g006.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/af38/12349219/b886397b40c5/sensors-25-04575-g007.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/af38/12349219/78f5b720cd22/sensors-25-04575-g001.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/af38/12349219/1826de99c4c0/sensors-25-04575-g002.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/af38/12349219/2b1dea418105/sensors-25-04575-g003.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/af38/12349219/f6e3fede11f3/sensors-25-04575-g004.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/af38/12349219/d2748506e2af/sensors-25-04575-g005.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/af38/12349219/9153071f5b02/sensors-25-04575-g006.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/af38/12349219/b886397b40c5/sensors-25-04575-g007.jpg

相似文献

[1]
Eye-Guided Multimodal Fusion: Toward an Adaptive Learning Framework Using Explainable Artificial Intelligence.

Sensors (Basel). 2025-7-24

[2]
CXR-MultiTaskNet a unified deep learning framework for joint disease localization and classification in chest radiographs.

Sci Rep. 2025-8-31

[3]
Shedding light on ai in radiology: A systematic review and taxonomy of eye gaze-driven interpretability in deep learning.

Eur J Radiol. 2024-3

[4]
A deep learning approach to direct immunofluorescence pattern recognition in autoimmune bullous diseases.

Br J Dermatol. 2024-7-16

[5]
Are Artificial Intelligence Models Listening Like Cardiologists? Bridging the Gap Between Artificial Intelligence and Clinical Reasoning in Heart-Sound Classification Using Explainable Artificial Intelligence.

Bioengineering (Basel). 2025-5-22

[6]
Deep Learning and Image Generator Health Tabular Data (IGHT) for Predicting Overall Survival in Patients With Colorectal Cancer: Retrospective Study.

JMIR Med Inform. 2025-8-19

[7]
ItpCtrl-AI: End-to-end interpretable and controllable artificial intelligence by modeling radiologists' intentions.

Artif Intell Med. 2025-2

[8]
Novel Artificial Intelligence-Driven Infant Meningitis Screening From High-Resolution Ultrasound Imaging.

Ultrasound Med Biol. 2025-6-28

[9]
Explainable Artificial Intelligence (XAI) in the Era of Large Language Models: Applying an XAI Framework in Pediatric Ophthalmology Diagnosis using the Gemini Model.

AMIA Jt Summits Transl Sci Proc. 2025-6-10

[10]
Deep learning-based image classification for AI-assisted integration of pathology and radiology in medical imaging.

Front Med (Lausanne). 2025-6-2

本文引用的文献

[1]
Eye Gaze Guided Cross-Modal Alignment Network for Radiology Report Generation.

IEEE J Biomed Health Inform. 2024-12

[2]
Analyzing Eye Paths Using Fractals.

Adv Neurobiol. 2024

[3]
Eye-Gaze-Guided Vision Transformer for Rectifying Shortcut Learning.

IEEE Trans Med Imaging. 2023-11

[4]
Explainable AI in medical imaging: An overview for clinical practitioners - Beyond saliency-based XAI approaches.

Eur J Radiol. 2023-5

[5]
Skill Characterisation of Sonographer Gaze Patterns during Second Trimester Clinical Fetal Ultrasounds using Time Curves.

Proc Eye Track Res Appl Symp. 2022-6

[6]
Changes in Radiologists' Gaze Patterns Against Lung X-rays with Different Abnormalities: a Randomized Experiment.

J Digit Imaging. 2023-6

[7]
REFLACX, a dataset of reports and eye-tracking data for localization of abnormalities in chest x-rays.

Sci Data. 2022-6-18

[8]
Explainable artificial intelligence (XAI) in deep learning-based medical image analysis.

Med Image Anal. 2022-7

[9]
Recent advances and clinical applications of deep learning in medical image analysis.

Med Image Anal. 2022-7

[10]
On Smart Gaze Based Annotation of Histopathology Images for Training of Deep Convolutional Neural Networks.

IEEE J Biomed Health Inform. 2022-7

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

推荐工具

医学文档翻译智能文献检索