Aesthet Surg J. 2023 Nov 16;43(12):NP1078-NP1082. doi: 10.1093/asj/sjad128.
BACKGROUND: Developed originally as a tool for resident self-evaluation, the Plastic Surgery Inservice Training Examination (PSITE) has become a standardized tool adopted by Plastic Surgery residency programs. The introduction of large language models (LLMs), such as ChatGPT (OpenAI, San Francisco, CA), has demonstrated the potential to help propel the field of Plastic Surgery. OBJECTIVES: The authors of this study wanted to assess whether or not ChatGPT could be utilized as a tool in resident education by assessing its accuracy on the PSITE. METHODS: Questions were obtained from the 2022 PSITE, which was present on the American Council of Academic Plastic Surgeons (ACAPS) website. Questions containing images or tables were carefully inspected and flagged before being inputted into ChatGPT. All responses by ChatGPT were qualified utilizing the properties of natural coherence. Responses that were found to be incorrect were divided into the following categories: logical, informational, or explicit fallacy. RESULTS: ChatGPT answered a total of 242 questions with an accuracy of 54.96%. The software incorporated logical reasoning in 88.8% of questions, internal information in 95.5% of questions, and external information in 92.1% of questions. When stratified by correct and incorrect responses, we determined that there was a statistically significant difference in ChatGPT's use of external information (P < .05). CONCLUSIONS: ChatGPT is a versatile tool that has the potential to impact resident education by providing general knowledge, clarifying information, providing case-based learning, and promoting evidence-based medicine. With advancements in LLM and artificial intelligence (AI), it is possible that ChatGPT may be an impactful tool for resident education within Plastic Surgery.
背景:整形外科住院医师培训考试(PSITE)最初是作为住院医师自我评价的工具而开发的,现已成为整形外科住院医师项目采用的标准化工具。大型语言模型(LLM)的引入,如 ChatGPT(OpenAI,旧金山,加利福尼亚州),已经证明了其有潜力帮助推动整形外科领域的发展。
目的:本研究的作者希望通过评估 ChatGPT 在 PSITE 上的准确性,来确定它是否可以作为住院医师教育的工具。
方法:从美国整形外科学会学术委员会(ACAPS)网站上的 2022 年 PSITE 中获取问题。对包含图像或表格的问题进行仔细检查和标记,然后输入 ChatGPT。利用自然连贯性的特性对 ChatGPT 的所有回复进行了限定。将发现不正确的回复分为逻辑、信息或明确错误。
结果:ChatGPT 共回答了 242 个问题,准确率为 54.96%。该软件在 88.8%的问题中纳入了逻辑推理,在 95.5%的问题中纳入了内部信息,在 92.1%的问题中纳入了外部信息。在正确和错误回答分层后,我们确定 ChatGPT 在使用外部信息方面存在统计学上的显著差异(P <.05)。
结论:ChatGPT 是一种多功能工具,它有潜力通过提供一般知识、澄清信息、提供基于案例的学习以及促进循证医学来影响住院医师教育。随着大型语言模型和人工智能(AI)的进步,ChatGPT 可能成为整形外科住院医师教育的一个有影响力的工具。
Aesthet Surg J. 2023-11-16
J Plast Reconstr Aesthet Surg. 2023-12
Aesthet Surg J. 2023-7-15
J Med Internet Res. 2023-8-22
Plast Reconstr Surg Glob Open. 2025-4-10
J Pain Res. 2025-3-19
J Exp Orthop. 2025-1-2
BMC Med Inform Decis Mak. 2024-11-29