Lei Jiayu, Dai Lisong, Jiang Haoyun, Wu Chaoyi, Zhang Xiaoman, Zhang Yao, Yao Jiangchao, Xie Weidi, Zhang Yanyong, Li Yuehua, Zhang Ya, Wang Yanfeng
School of Computer Science and Technology, University of Science and Technology of China, Hefei, Anhui, 230026, China; Shanghai Artificial Intelligence Laboratory, Shanghai, 200232, China.
Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, 200233, China.
Comput Med Imaging Graph. 2025 Jun;122:102516. doi: 10.1016/j.compmedimag.2025.102516. Epub 2025 Mar 7.
Magnetic Resonance Imaging (MRI) has become a pivotal tool in diagnosing brain diseases, with a wide array of computer-aided artificial intelligence methods being proposed to enhance diagnostic accuracy. However, early studies were often limited by small-scale datasets and a narrow range of disease types, which posed challenges in model generalization. This study presents UniBrain, a hierarchical knowledge-enhanced pre-training framework designed for universal brain MRI diagnosis. UniBrain leverages a large-scale dataset comprising 24,770 imaging-report pairs from routine diagnostics for pre-training. Unlike previous approaches that either focused solely on visual representation learning or used brute-force alignment between vision and language, the framework introduces a hierarchical alignment mechanism. This mechanism extracts structured knowledge from free-text clinical reports at multiple granularities, enabling vision-language alignment at both the sequence and case levels, thereby significantly improving feature learning efficiency. A coupled vision-language perception module is further employed for text-guided multi-label classification, which facilitates zero-shot evaluation and fine-tuning of downstream tasks without modifying the model architecture. UniBrain is validated on both in-domain and out-of-domain datasets, consistently surpassing existing state-of-the-art diagnostic models and demonstrating performance on par with radiologists in specific disease categories. It shows strong generalization capabilities across diverse tasks, highlighting its potential for broad clinical application. The code is available at https://github.com/ljy19970415/UniBrain.
磁共振成像（MRI）已成为诊断脑部疾病的关键工具，人们提出了各种各样的计算机辅助人工智能方法来提高诊断准确性。然而，早期研究往往受限于小规模数据集和狭窄的疾病类型范围，这给模型泛化带来了挑战。本研究提出了UniBrain，这是一个为通用脑部MRI诊断设计的分层知识增强预训练框架。UniBrain利用一个包含来自常规诊断的24770个影像-报告对的大规模数据集进行预训练。与以往要么只专注于视觉表征学习，要么在视觉和语言之间使用蛮力对齐的方法不同，该框架引入了一种分层对齐机制。这种机制从自由文本临床报告中以多种粒度提取结构化知识，实现序列和病例级别的视觉-语言对齐，从而显著提高特征学习效率。还采用了一个耦合的视觉-语言感知模块进行文本引导的多标签分类，这有助于在不修改模型架构的情况下对下游任务进行零样本评估和微调。UniBrain在域内和域外数据集上均得到验证，始终超越现有的最先进诊断模型，并在特定疾病类别中表现出与放射科医生相当的性能。它在各种任务中都显示出强大的泛化能力，突出了其广泛临床应用的潜力。代码可在https://github.com/ljy19970415/UniBrain获取。
