Fujita Nana, Yasaka Koichiro, Kiryu Shigeru, Abe Osamu
Department of Radiology, National Center for Global Health and Medicine, Japan Institute for Health Security, Tokyo, Japan.
Department of Radiology, The University of Tokyo, Tokyo, Japan.
Emerg Radiol. 2025 Jun 2. doi: 10.1007/s10140-025-02354-1.
This study aimed to develop an automated early warning system using a large language model (LLM) to identify acute to subacute brain infarction from free-text computed tomography (CT) or magnetic resonance imaging (MRI) radiology reports.
In this retrospective study, 5,573, 1,883, and 834 patients were included in the training (mean age, 67.5 ± 17.2 years; 2,831 males), validation (mean age, 61.5 ± 18.3 years; 994 males), and test (mean age, 66.5 ± 16.1 years; 488 males) datasets. An LLM (Japanese Bidirectional Encoder Representations from Transformers model) was fine-tuned to classify the CT and MRI reports into three groups (group 0, newly identified acute to subacute infarction; group 1, known acute to subacute infarction or old infarction; group 2, without infarction). The training and validation processes were repeated 15 times, and the best-performing model on the validation dataset was selected to further evaluate its performance on the test dataset.
The best fine-tuned model exhibited sensitivities of 0.891, 0.905, and 0.959 for groups 0, 1, and 2, respectively, in the test dataset. The macrosensitivity (the average of sensitivity for all groups) and accuracy were 0.918 and 0.923, respectively. The model's performance in extracting newly identified acute brain infarcts was high, with an area under the receiver operating characteristic curve of 0.979 (95% confidence interval, 0.956-1.000). The average prediction time was 0.115 ± 0.037 s per patient.
A fine-tuned LLM could extract newly identified acute to subacute brain infarcts based on CT or MRI findings with high performance.
本研究旨在开发一种使用大语言模型（LLM）的自动预警系统，以从自由文本计算机断层扫描（CT）或磁共振成像（MRI）放射学报告中识别急性至亚急性脑梗死。
在这项回顾性研究中，5573例、1883例和834例患者分别纳入训练（平均年龄67.5±17.2岁；男性2831例）、验证（平均年龄61.5±18.3岁；男性994例）和测试（平均年龄66.5±16.1岁；男性488例）数据集。对一个LLM（日语双向编码器表征来自变压器模型）进行微调，以将CT和MRI报告分为三组（0组，新发现的急性至亚急性梗死；1组，已知急性至亚急性梗死或陈旧性梗死；2组，无梗死）。训练和验证过程重复15次，选择在验证数据集上表现最佳的模型，以进一步评估其在测试数据集上的性能。
在测试数据集中，最佳微调模型对0组、1组和2组的敏感度分别为0.891、0.905和0.959。宏敏感度（所有组敏感度的平均值）和准确率分别为0.918和0.923。该模型在提取新发现的急性脑梗死方面表现出色，受试者操作特征曲线下面积为0.979（95%置信区间，0.956 - 1.000）。平均预测时间为每位患者0.115±0.037秒。
经过微调的LLM可以根据CT或MRI结果高效提取新发现的急性至亚急性脑梗死。
