PURPOSE

The 21st Century Cures Act final rule requires that patients have real-time access to their radiology reports, which contain technical language. The objective of this study to was to use a novel system called RadGPT, which integrates concept extraction and a large language model (LLM), to help patients understand their radiology reports.

METHODS

RadGPT generated 150 concept explanations and 390 question-and-answer pairs from 30 radiology report impressions from between 2012 and 2020. The extracted concepts were used to create concept-based explanations, as well as concept-based question-and-answer pairs for which questions were generated using either a fixed template or an LLM. Additionally, report-based question-and-answer pairs were generated directly from the impression using an LLM without concept extraction. One board-certified radiologist and four radiology residents rated the material quality using a standardized rubric.

RESULTS

Concept-based LLM-generated questions were of significantly higher quality than concept-based template-generated questions (P < .001). Excluding those template-based question-and-answer pairs from further analysis, nearly all (>95%) of RadGPT-generated materials were rated highly, with at least 50% receiving the highest possible ranking from all five raters. No answers or explanations were rated as likely to affect the safety or effectiveness of patient care. Report-level LLM-based questions and answers were rated particularly highly, with 92% of report-level LLM-based questions and 61% of the corresponding report-level answers receiving the highest rating from all raters.

CONCLUSIONS

The educational tool RadGPT generated high-quality explanations and question-and-answer pairs that were personalized for each radiology report, unlikely to produce harmful explanations, and likely to enhance patient understanding of radiology information.