Bridging the Gap: Can Large Language Models Match Human Expertise in Writing Neurosurgical Operative Notes?

BACKGROUND

Proper documentation is essential for patient care. The popularity of artificial intelligence (AI) offers the potential for improvements in neurosurgical note-writing. This study aimed to assess how AI can optimize documentation in neurosurgical procedures.

METHODS

Thirty-six operative notes were included. All identifiable data were removed. Essential information, such as perioperative data and diagnosis, was sourced from these notes. ChatGPT 4.0 was trained to draft notes from surgical vignettes using each surgeon's note template. One hundred forty-four surveys with a surgeon or AI note were shared with 3 surgeons to evaluate accuracy, content, and organization using a 5-point scale. Accuracy was defined as the factual correctness; content, as the comprehensiveness; and organization, as the arrangement of the note. Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) scores quantified each note's readability.

RESULTS

The mean AI accuracy was not different from the mean surgeon accuracy (4.44 vs. 4.33; P = 0.512), the mean AI content was lower than the mean surgeon content (3.73 vs. 4.42; P < 0.001). The mean AI note FKGL was greater than the mean surgeon FKGL (13.13 vs. 9.99; P < 0.001) and the mean AI FRE was lower than the mean surgeon FRE (21.42 vs. 41.70; P < 0.001).

CONCLUSIONS

AI notes were on par with surgeon notes in terms of accuracy and organization but lacked in content. Additionally, AI notes used language at an advanced reading level. These findings support the potential for ChatGPT to enhance the efficiency of neurosurgery documentation.