Department of Diagnostic and Interventional Radiology, University Hospital Bonn, Venusberg-Campus 1, 53127 Bonn, Germany; Quantitative Imaging Lab Bonn (QILaB), University Hospital Bonn, Venusberg-Campus 1, 53127 Bonn, Germany.
University Hospital Bonn, Department of Medical Biometry, Informatics, and Epidemiology, Venusberg-Campus 1, 53127 Bonn, Germany.
J Cardiovasc Magn Reson. 2024 Summer;26(1):101035. doi: 10.1016/j.jocmr.2024.101035. Epub 2024 Mar 7.
Patients are increasingly using Generative Pre-trained Transformer 4 (GPT-4) to better understand their own radiology findings.
To evaluate the performance of GPT-4 in transforming cardiovascular magnetic resonance (CMR) reports into text that is comprehensible to medical laypersons.
ChatGPT with GPT-4 architecture was used to generate three different explained versions of 20 various CMR reports (n = 60) using the same prompt: "Explain the radiology report in a language understandable to a medical layperson". Two cardiovascular radiologists evaluated understandability, factual correctness, completeness of relevant findings, and lack of potential harm, while 13 medical laypersons evaluated the understandability of the original and the GPT-4 reports on a Likert scale (1 "strongly disagree", 5 "strongly agree"). Readability was measured using the Automated Readability Index (ARI). Linear mixed-effects models (values given as median [interquartile range]) and intraclass correlation coefficient (ICC) were used for statistical analysis.
GPT-4 reports were generated on average in 52 s ± 13. GPT-4 reports achieved a lower ARI score (10 [9-12] vs 5 [4-6]; p < 0.001) and were subjectively easier to understand for laypersons than original reports (1 [1] vs 4 [4,5]; p < 0.001). Eighteen out of 20 (90%) standard CMR reports and 2/60 (3%) GPT-generated reports had an ARI score corresponding to the 8th grade level or higher. Radiologists' ratings of the GPT-4 reports reached high levels for correctness (5 [4, 5]), completeness (5 [5]), and lack of potential harm (5 [5]); with "strong agreement" for factual correctness in 94% (113/120) and completeness of relevant findings in 81% (97/120) of reports. Test-retest agreement for layperson understandability ratings between the three simplified reports generated from the same original report was substantial (ICC: 0.62; p < 0.001). Interrater agreement between radiologists was almost perfect for lack of potential harm (ICC: 0.93, p < 0.001) and moderate to substantial for completeness (ICC: 0.76, p < 0.001) and factual correctness (ICC: 0.55, p < 0.001).
GPT-4 can reliably transform complex CMR reports into more understandable, layperson-friendly language while largely maintaining factual correctness and completeness, and can thus help convey patient-relevant radiology information in an easy-to-understand manner.
患者越来越多地使用生成式预训练转换器 4（GPT-4）来更好地了解自己的放射学发现。
评估 GPT-4 将心血管磁共振（CMR）报告转换为医学外行易懂的文本的性能。
使用具有 GPT-4 架构的 ChatGPT 生成了 20 种不同 CMR 报告的三种不同解释版本（n=60），使用相同的提示：“用医学外行易懂的语言解释放射学报告”。两位心血管放射科医生评估了易懂性、事实正确性、相关发现的完整性以及潜在危害的缺失，而 13 名医学外行则使用李克特量表（1“强烈不同意”，5“强烈同意”）对原始报告和 GPT-4 报告的易懂性进行了评估。使用自动化可读性指数（ARI）测量可读性。使用线性混合效应模型（给出中位数[四分位数范围]的值）和组内相关系数（ICC）进行统计分析。
GPT-4 报告的生成平均用时 52 秒±13 秒。GPT-4 报告的 ARI 得分较低（10 [9-12] 与 5 [4-6]；p<0.001），并且外行比原始报告更容易理解（1 [1] 与 4 [4,5]；p<0.001）。20 份标准 CMR 报告中的 18 份（90%）和 60 份 GPT 生成报告中的 2 份（3%）的 ARI 得分对应 8 年级或更高年级的水平。放射科医生对 GPT-4 报告的正确性（5 [4,5]）、完整性（5 [5]）和潜在危害缺失（5 [5]）的评分很高；94%（113/120）的报告和 81%（97/120）的报告对事实正确性的评价为“强烈同意”。对同一原始报告生成的三个简化报告，外行易懂性评分的测试-再测试一致性为中等至高（ICC：0.62；p<0.001）。放射科医生之间在缺乏潜在危害方面的一致性几乎是完美的（ICC：0.93，p<0.001），在完整性（ICC：0.76，p<0.001）和事实正确性（ICC：0.55，p<0.001）方面的一致性为中度到高度。
GPT-4 可以可靠地将复杂的 CMR 报告转换为更易于理解的、面向医学外行的语言，同时在很大程度上保持事实正确性和完整性，从而帮助以易于理解的方式传达与患者相关的放射学信息。
