Bai Qi, Zou Xiaodi, Alhaskawi Ahmad, Dong Yanzhao, Zhou Haiying, Ezzi Sohaib Hasan Abdullah, Kota Vishnu Goutham, AbdullaAbdulla Mohamed Hasan Hasan, Abdalbary Sahar Ahmed, Hu Xianliang, Lu Hui
Department of Orthopedics, The First Affiliated Hospital, Zhejiang University, #79 Qingchun Road, Hangzhou, Zhejiang Province, 310003, People's Republic of China.
School of Mathematical Sciences, Zhejiang University, # 866 Yuhangtang Road, Hangzhou, Zhejiang Province, 310058, People's Republic of China.
Sci Rep. 2025 May 23;15(1):17991. doi: 10.1038/s41598-025-00570-w.
The task of generating medical reports automatically is of paramount importance in modern healthcare, offering a substantial reduction in the workload of radiologists and accelerating the processes of clinical diagnosis and treatment. Current challenges include handling limited sample sizes and interpreting intricate multi-modal and multi-view medical data. In order to improve the accuracy and efficiency for radiologists, we conducted this investigation. This study aims to present a novel methodology for medical report generation that leverages Multi-View Contrastive Learning (MVCL) applied to MRI data, combined with a Symptom Consultant (SC) for extracting medical insights, to improve the quality and efficiency of automated medical report generation. We introduce an advanced MVCL framework that maximizes the potential of multi-view MRI data to enhance visual feature extraction. Alongside, the SC component is employed to distill critical medical insights from symptom descriptions. These components are integrated within a transformer decoder architecture, which is then applied to the Deep Wrist dataset for model training and evaluation. Our experimental analysis on the Deep Wrist dataset reveals that our proposed integration of MVCL and SC significantly outperforms the baseline model in terms of accuracy and relevance of the generated medical reports. The results indicate that our approach is particularly effective in capturing and utilizing the complex information inherent in multi-modal and multi-view medical datasets. The combination of MVCL and SC constitutes a powerful approach to medical report generation, addressing the existing challenges in the field. The demonstrated superiority of our model over traditional methods holds promise for substantial improvements in clinical diagnosis and automated report generation, indicating a significant stride forward in medical technology.
在现代医疗保健中，自动生成医学报告的任务至关重要，它能大幅减轻放射科医生的工作量，并加速临床诊断和治疗过程。当前的挑战包括处理有限的样本量以及解读复杂的多模态和多视图医学数据。为了提高放射科医生的准确性和效率，我们进行了这项调查。本研究旨在提出一种新颖的医学报告生成方法，该方法利用应用于MRI数据的多视图对比学习（MVCL），并结合症状顾问（SC）来提取医学见解，以提高自动医学报告生成的质量和效率。我们引入了一个先进的MVCL框架，该框架最大限度地发挥多视图MRI数据的潜力，以增强视觉特征提取。同时，采用SC组件从症状描述中提炼关键的医学见解。这些组件被集成到一个Transformer解码器架构中，然后应用于深度手腕数据集进行模型训练和评估。我们在深度手腕数据集上的实验分析表明，我们提出的MVCL和SC的集成在生成的医学报告的准确性和相关性方面明显优于基线模型。结果表明，我们的方法在捕获和利用多模态和多视图医学数据集中固有的复杂信息方面特别有效。MVCL和SC的结合构成了一种强大的医学报告生成方法，解决了该领域现有的挑战。我们的模型相对于传统方法所展示的优越性有望在临床诊断和自动报告生成方面取得实质性改进，表明医学技术向前迈出了重要一步。
