Kaya Kenan, Gietzen Carsten, Hahnfeldt Robert, Zoubi Maher, Emrich Tilman, Halfmann Moritz C, Sieren Malte Maria, Elser Yannic, Krumm Patrick, Brendel Jan M, Nikolaou Konstantin, Haag Nina, Borggrefe Jan, Krüchten Ricarda von, Müller-Peltzer Katharina, Ehrengut Constantin, Denecke Timm, Hagendorff Andreas, Goertz Lukas, Gertz Roman J, Bunck Alexander Christian, Maintz David, Persigehl Thorsten, Lennartz Simon, Luetkens Julian A, Jaiswal Astha, Iuga Andra Iza, Pennig Lenhard, Kottlors Jonathan
Institute for Diagnostic and Interventional Radiology, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany.
Institute for Diagnostic and Interventional Radiology, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany.
J Cardiovasc Magn Reson. 2024;26(2):101068. doi: 10.1016/j.jocmr.2024.101068. Epub 2024 Jul 28.
Diagnosing myocarditis relies on multimodal data, including cardiovascular magnetic resonance (CMR), clinical symptoms, and blood values. The correct interpretation and integration of CMR findings require radiological expertise and knowledge. We aimed to investigate the performance of Generative Pre-trained Transformer 4 (GPT-4), a large language model, for report-based medical decision-making in the context of cardiac MRI for suspected myocarditis.
This retrospective study includes CMR reports from 396 patients with suspected myocarditis and eight centers, respectively. CMR reports and patient data including blood values, age, and further clinical information were provided to GPT-4 and radiologists with 1 (resident 1), 2 (resident 2), and 4 years (resident 3) of experience in CMR and knowledge of the 2018 Lake Louise Criteria. The final impression of the report regarding the radiological assessment of whether myocarditis is present or not was not provided. The performance of Generative pre-trained transformer 4 (GPT-4) and the human readers were compared to a consensus reading (two board-certified radiologists with 8 and 10 years of experience in CMR). Sensitivity, specificity, and accuracy were calculated.
GPT-4 yielded an accuracy of 83%, sensitivity of 90%, and specificity of 78%, which was comparable to the physician with 1 year of experience (R1: 86%, 90%, 84%, p = 0.14) and lower than that of more experienced physicians (R2: 89%, 86%, 91%, p = 0.007 and R3: 91%, 85%, 96%, p < 0.001). GPT-4 and human readers showed a higher diagnostic performance when results from T1- and T2-mapping sequences were part of the reports, for residents 1 and 3 with statistical significance (p = 0.004 and p = 0.02, respectively).
GPT-4 yielded good accuracy for diagnosing myocarditis based on CMR reports in a large dataset from multiple centers and therefore holds the potential to serve as a diagnostic decision-supporting tool in this capacity, particularly for less experienced physicians. Further studies are required to explore the full potential and elucidate educational aspects of the integration of large language models in medical decision-making.
心肌炎的诊断依赖于多模态数据，包括心血管磁共振成像（CMR）、临床症状和血液指标。CMR检查结果的正确解读和整合需要放射学专业知识。我们旨在研究生成式预训练变换器4（GPT-4）这一大型语言模型在疑似心肌炎心脏磁共振成像背景下基于报告的医疗决策中的表现。
这项回顾性研究分别纳入了来自8个中心的396例疑似心肌炎患者的CMR报告。将CMR报告以及包括血液指标、年龄和其他临床信息在内的患者数据提供给GPT-4以及具有1年（住院医师1）、2年（住院医师2）和4年（住院医师3）CMR经验且了解2018年路易斯湖标准的放射科医生。报告中未给出关于是否存在心肌炎的放射学评估的最终结论。将生成式预训练变换器4（GPT-4）和人类读者的表现与一致性解读（两位具有8年和10年CMR经验的经委员会认证的放射科医生）进行比较。计算敏感性、特异性和准确性。
GPT-4的准确率为83%，敏感性为90%，特异性为78%，与有1年经验的医生（住院医师1：86%、90%、84%，p = 0.14）相当，但低于经验更丰富的医生（住院医师2：89%、86%、91%，p = 0.007；住院医师3：91%、85%、96%，p < 0.001）。当T1和T2映射序列的结果包含在报告中时，GPT-4和人类读者的诊断表现更高，住院医师1和住院医师3的情况具有统计学意义（分别为p = 0.004和p = 0.02）。
在来自多个中心的大型数据集中，GPT-4基于CMR报告诊断心肌炎具有良好的准确率，因此有潜力作为一种诊断决策支持工具，特别是对于经验较少的医生。需要进一步研究以探索其全部潜力，并阐明大型语言模型在医疗决策中整合的教育意义。
