Chung Eric M, Zhang Samuel C, Nguyen Anthony T, Atkins Katelyn M, Sandler Howard M, Kamrava Mitchell
Department of Radiation Oncology, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
Samuel Oschin Comprehensive Cancer Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
Digit Health. 2023 Dec 19;9:20552076231221620. doi: 10.1177/20552076231221620. eCollection 2023 Jan-Dec.
Patients now have direct access to their radiology reports, which can include complex terminology and be difficult to understand. We assessed ChatGPT's ability to generate summarized MRI reports for patients with prostate cancer and evaluated physician satisfaction with the artificial intelligence (AI)-summarized report.
We used ChatGPT to summarize five full MRI reports for patients with prostate cancer performed at a single institution from 2021 to 2022. Three summarized reports were generated for each full MRI report. Full MRI and summarized reports were assessed for readability using Flesch-Kincaid Grade Level (FK) score. Radiation oncologists were asked to evaluate the AI-summarized reports via an anonymous questionnaire. Qualitative responses were given on a 1-5 Likert-type scale. Fifty newly diagnosed prostate cancer patient MRIs performed at a single institution were additionally assessed for physician online portal response rates.
Fifteen summarized reports were generated from five full MRI reports using ChatGPT. The median FK score for the full MRI reports and summarized reports was 9.6 vs. 5.0, ( < 0.05), respectively. Twelve radiation oncologists responded to our questionnaire. The mean [SD] ratings for summarized reports were factual correctness (4.0 [0.6], understanding 4.0 [0.7]), completeness (4.1 [0.5]), potential for harm (3.5 [0.9]), overall quality (3.4 [0.9]), and likelihood to send to patient (3.1 [1.1]). Current physician online portal response rates were 14/50 (28%) at our institution.
We demonstrate a novel application of ChatGPT to summarize MRI reports at a reading level appropriate for patients. Physicians were likely to be satisfied with the summarized reports with respect to factual correctness, ease of understanding, and completeness. Physicians were less likely to be satisfied with respect to potential for harm, overall quality, and likelihood to send to patients. Further research is needed to optimize ChatGPT's ability to summarize radiology reports and understand what factors influence physician trust in AI-summarized reports.
患者现在可以直接获取他们的放射学报告，这些报告可能包含复杂的术语，难以理解。我们评估了ChatGPT为前列腺癌患者生成MRI报告摘要的能力，并评估了医生对人工智能（AI）摘要报告的满意度。
我们使用ChatGPT对2021年至2022年在单一机构为前列腺癌患者进行的五份完整MRI报告进行总结。每份完整的MRI报告生成三份总结报告。使用弗莱什-金凯德年级水平（FK）评分评估完整MRI报告和总结报告的可读性。放射肿瘤学家被要求通过匿名问卷评估AI摘要报告。定性回答采用1-5李克特量表。另外对在单一机构进行的50例新诊断前列腺癌患者的MRI进行了医生在线门户响应率评估。
使用ChatGPT从五份完整的MRI报告中生成了15份总结报告。完整MRI报告和总结报告的中位数FK评分分别为9.6和5.0（P<0.05）。12名放射肿瘤学家回复了我们的问卷。总结报告的平均[标准差]评分在事实准确性（4.0[0.6]）、易懂性（4.0[0.7]）、完整性（4.1[0.5]）、潜在危害（3.5[0.9]）、总体质量（3.4[0.9]）以及发送给患者的可能性（3.1[1.1]）方面。我们机构目前医生在线门户响应率为14/50（28%）。
我们展示了ChatGPT在以适合患者的阅读水平总结MRI报告方面的新应用。医生可能对总结报告在事实准确性、易于理解和完整性方面感到满意。医生在潜在危害、总体质量以及发送给患者的可能性方面不太可能满意。需要进一步研究以优化ChatGPT总结放射学报告的能力，并了解哪些因素影响医生对AI摘要报告的信任。
