Education Research: Creating Online Interactive Case-Based Learning Experiences From Educational Case Reports With Large Language Models: A Feasibility Study.

BACKGROUND AND OBJECTIVES

Case reports are a fundamental part of medical literature and education. Artificial intelligence (AI) is increasingly influencing medical education and can potentially augment the delivery of the educational content in case reports. The aim of this study was to evaluate the feasibility of using AI, namely large language models (LLMs), to convert previously published into an interactive online format to facilitate case-based learning.

METHODS

Three were converted into a free-text "screenplay" using the LLM Claude 3.5 Sonnet. These "screenplays" were then delivered in an interactive format through an online platform using GPT-4o. Two neurology fellows interrogated (prompted) the cases delivered by the online platform in a question-and-answer manner, seeking history, examination findings, and investigation results to arrive at a diagnosis and plan. These neurology fellows were not aware of the case report or screenplay content and asked questions in a manner that they would when evaluating a patient. A neurologist then reviewed each question-and-answer exchange for "screenplay" adherence and medical appropriateness. Results were analyzed with descriptive statistics.

RESULTS

The overall number of appropriate responses generated by the LLM was 206 of 210 (98.1%). There were 26 of 210 responses in which additional content was generated, all of which were medically plausible or consistent with the context of the case. The 4 errors that occurred were omissions of investigation results at the "screenplay" stage, which are amenable to manual correction. The omissions were the results of 3 unrevealing blood tests and 1 electroencephalogram result. None of these errors precluded the establishment of the diagnosis and completion of the case.

DISCUSSION

It is feasible to convert into an interactive question-and-answer format using LLMs. It should be noted that the nondeterministic nature of frontier LLMs and the potential for such LLM versions to change frequently are relevant considerations in making estimates of performance. Further studies investigating the impacts of this educational innovation are required.