Department of Anesthesiology, El Camino Health, 2500 Grant Road, Mountain View, California 94040, USA.
Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, 300 Pasteur Drive, Room H3580, MC 5640, Stanford 94305, CA, USA.
J Clin Anesth. 2024 Nov;98:111582. doi: 10.1016/j.jclinane.2024.111582. Epub 2024 Aug 20.
ChatGPT4 is a leading large language model (LLM) chatbot released by OpenAI in 2023. ChatGPT4 can respond to free-text queries, answer questions and make suggestions regarding virtually any topic. ChatGPT4 has successfully answered anesthesia and even obstetric anesthesia knowledge-based questions with reasonable accuracy. However, ChatGPT4 has yet to be challenged in obstetric anesthesia clinical decision-making.
In this study, we evaluated the performance of ChatGPT4 in the management of clinical labor analgesia scenarios compared to expert obstetric anesthesiologists.
Eight clinical questions with progressively increasing medical complexity were posed to ChatGPT4.
The ChatGPT4 responses were rated by seven expert obstetric anesthesiologists based on safety, accuracy and completeness of each response using a five-point Likert rating scale.
ChatGPT4 was deemed safe in 73% of responses to the presented obstetric anesthesia clinical scenarios (27% of responses were deemed unsafe). None of the ChatGPT4 responses were unanimously deemed to be safe by all seven expert obstetric anesthesiologists. Moreover, ChatGPT4 responses were overall partly accurate (score 4 out of 5) and somewhat incomplete (score 3.5 out of 5).
In summary, approximately one quarter of all responses by ChatGPT4 were deemed unsafe by expert obstetric anesthesiologists. These findings may suggest the need for more fine-tuning and training of LLMs such as ChatGPT4 specifically for clinical decision making in obstetric anesthesia or other specialized medical fields. These LLMs may come to play an important future role in assisting obstetric anesthesiologists in clinical decision making and enhancing overall patient care.
ChatGPT4 是 OpenAI 于 2023 年发布的一款领先的大型语言模型（LLM）聊天机器人。ChatGPT4 可以对自由文本查询做出响应，回答关于几乎任何主题的问题并提供建议。ChatGPT4 已经成功地以合理的准确性回答了麻醉学，甚至产科麻醉学的基础知识问题。然而，ChatGPT4 在产科麻醉临床决策方面尚未受到挑战。
在这项研究中，我们评估了 ChatGPT4 在管理临床分娩镇痛场景方面的表现，与专家产科麻醉师进行了比较。
向 ChatGPT4 提出了八个具有逐渐增加医学复杂性的临床问题。
七位专家产科麻醉师根据每个回答的安全性、准确性和完整性，使用五点 Likert 评分量表对 ChatGPT4 的回答进行评分。
ChatGPT4 被认为在呈现的产科麻醉临床场景的 73%的回答中是安全的（27%的回答被认为是不安全的）。在所有七个专家产科麻醉师中，没有一个人一致认为 ChatGPT4 的所有回答都是安全的。此外，ChatGPT4 的回答总体上是部分准确的（得分为 5 分中的 4 分），并且有些不完整（得分为 5 分中的 3.5 分）。
总之，ChatGPT4 的大约四分之一的回答被专家产科麻醉师认为是不安全的。这些发现可能表明需要对像 ChatGPT4 这样的大型语言模型进行更精细的调整和培训，特别是在产科麻醉或其他专业医学领域的临床决策中。这些大型语言模型可能会在未来在协助产科麻醉师进行临床决策和增强整体患者护理方面发挥重要作用。
