Lyell McEwin Hospital, Elizabeth Vale, SA 5112, Australia; SA Health, Adelaide, SA 5000, Australia; University of Adelaide, Adelaide, SA 5005, Australia; Royal Adelaide Hospital, Adelaide, SA 5000, Australia.
Lyell McEwin Hospital, Elizabeth Vale, SA 5112, Australia; University of Adelaide, Adelaide, SA 5005, Australia.
J Clin Neurosci. 2024 Nov;129:110847. doi: 10.1016/j.jocn.2024.110847. Epub 2024 Sep 20.
Audits are an integral part of effective modern healthcare. The collection of data for audits can be resource intensive. Large language models (LLM) may be able to assist. This pilot study aimed to assess the feasibility of using a LLM to extract stroke audit data from free-text medical documentation.
Discharge summaries from a one-month retrospective cohort of stroke admissions at a tertiary hospital were collected. A locally-deployed LLM, LLaMA3, was then used to extract a variety of routine stroke audit data from free-text discharge summaries. These data were compared to the previously collected human audit data in the statewide registry. Manual case note review was undertaken in cases of discordance.
Overall, there was a total of 144 data points that were extracted (9 data points for each of the 16 patients). The LLM was correct in 135/144 (93.8%) of individual datapoints. This performance included binary categorical, multiple-option categorical, datetime, and free-text extraction fields.
LLM may be able to assist with the efficient collection of stroke audit data. Such approaches may be pursued in other specialties. Future studies should seek to examine the most effective way to deploy such approaches in conjunction with human auditors and researchers.
审核是现代医疗保健的重要组成部分。审核数据的收集可能需要大量资源。大型语言模型（LLM）可能会有所帮助。这项试点研究旨在评估使用 LLM 从文本记录中提取中风审核数据的可行性。
收集了一家三级医院中风入院一个月的回顾性队列的出院记录。然后，使用本地部署的 LLM，即 LLaMA3，从自由文本的出院记录中提取各种常规中风审核数据。将这些数据与全州注册中心中之前收集的人工审核数据进行比较。对于不一致的情况，进行手动病历审查。
总体而言，共提取了 144 个数据点（16 名患者的每个数据点 9 个）。LLM 在 135/144（93.8%）个单独的数据点上是正确的。这种性能包括二进制分类、多选项分类、日期时间和自由文本提取字段。
LLM 可能能够帮助高效地收集中风审核数据。这种方法可能会在其他专业领域中得到应用。未来的研究应探讨在人工审核员和研究人员的配合下，以最有效的方式部署这种方法。
