文献检索文档翻译深度研究
Suppr Zotero 插件Zotero 插件
邀请有礼套餐&价格历史记录

新学期,新优惠

限时优惠:9月1日-9月22日

30天高级会员仅需29元

1天体验卡首发特惠仅需5.99元

了解详情
不再提醒
插件&应用
Suppr Zotero 插件Zotero 插件浏览器插件Mac 客户端Windows 客户端微信小程序
高级版
套餐订阅购买积分包
AI 工具
文献检索文档翻译深度研究
关于我们
关于 Suppr公司介绍联系我们用户协议隐私条款
关注我们

Suppr 超能文献

核心技术专利:CN118964589B侵权必究
粤ICP备2023148730 号-1Suppr @ 2025

人工智能与住院医师在上肢骨科培训考试问题上表现的比较。

Comparison of Artificial Intelligence to Resident Performance on Upper-Extremity Orthopaedic In-Training Examination Questions.

作者信息

Ozdag Yagiz, Hayes Daniel S, Makar Gabriel S, Manzar Shahid, Foster Brian K, Shultz Mason J, Klena Joel C, Grandizio Louis C

机构信息

Department of Orthopaedic Surgery, Geisinger Musculoskeletal Institute, Geisinger Commonwealth School of Medicine, Danville, PA.

出版信息

J Hand Surg Glob Online. 2023 Dec 11;6(2):164-168. doi: 10.1016/j.jhsg.2023.10.013. eCollection 2024 Mar.


DOI:10.1016/j.jhsg.2023.10.013
PMID:38903829
原文链接:https://pmc.ncbi.nlm.nih.gov/articles/PMC11185884/
Abstract

PURPOSE: Currently, there is a paucity of prior investigations and studies examining applications for artificial intelligence (AI) in upper-extremity (UE) surgical education. The purpose of this investigation was to assess the performance of a novel AI tool (ChatGPT) on UE questions on the Orthopaedic In-Training Examination (OITE). We aimed to compare the performance of ChatGPT to the examination performance of hand surgery residents. METHODS: We selected questions from the 2020-2022 OITEs that focused on both the hand and UE as well as the shoulder and elbow content domains. These questions were divided into two categories: those with text-only prompts (text-only questions) and those that included supplementary images or videos (media questions). Two authors (B.K.F. and G.S.M.) converted the accompanying media into text-based descriptions. Included questions were inputted into ChatGPT (version 3.5) to generate responses. Each OITE question was entered into ChatGPT three times: (1) open-ended response, which requested a free-text response; (2) multiple-choice responses without asking for justification; and (3) multiple-choice response with justification. We referred to the OITE scoring guide for each year in order to compare the percentage of correct AI responses to correct resident responses. RESULTS: A total of 102 UE OITE questions were included; 59 were text-only questions, and 43 were media-based. ChatGPT correctly answered 46 (45%) of 102 questions using the Multiple Choice No Justification prompt requirement (42% for text-based and 44% for media questions). Compared to ChatGPT, postgraduate year 1 orthopaedic residents achieved an average score of 51% correct. Postgraduate year 5 residents answered 76% of the same questions correctly. CONCLUSIONS: ChatGPT answered fewer UE OITE questions correctly compared to hand surgery residents of all training levels. CLINICAL RELEVANCE: Further development of novel AI tools may be necessary if this technology is going to have a role in UE education.

摘要

目的:目前,针对人工智能(AI)在上肢(UE)外科教育中的应用进行的前期调查和研究较少。本研究的目的是评估一种新型人工智能工具(ChatGPT)在骨科住院医师培训考试(OITE)中关于上肢问题的表现。我们旨在将ChatGPT的表现与手外科住院医师的考试表现进行比较。 方法:我们从2020 - 2022年的OITE中选取了聚焦于手部和上肢以及肩部和肘部内容领域的问题。这些问题分为两类:仅带有文本提示的问题(纯文本问题)和包含补充图像或视频的问题(多媒体问题)。两位作者(B.K.F.和G.S.M.)将附带的多媒体内容转换为基于文本的描述。将纳入的问题输入到ChatGPT(3.5版本)中以生成回答。每个OITE问题在ChatGPT中输入三次:(1)开放式回答,要求自由文本回答;(2)不要求说明理由的多项选择题回答;(3)要求说明理由的多项选择题回答。我们参考每年的OITE评分指南,以便比较人工智能正确回答的百分比与住院医师正确回答的百分比。 结果:总共纳入了102个上肢OITE问题;59个是纯文本问题,43个是基于多媒体的问题。ChatGPT使用“不要求说明理由的多项选择题”提示要求正确回答了102个问题中的46个(45%)(基于文本的问题为42%,基于多媒体的问题为44%)。与ChatGPT相比,一年级骨科住院医师的平均正确得分率为51%。五年级住院医师正确回答了相同问题的76%。 结论:与所有培训水平的手外科住院医师相比,ChatGPT正确回答的上肢OITE问题较少。 临床意义:如果这项技术要在上肢教育中发挥作用,可能需要进一步开发新型人工智能工具。

https://cdn.ncbi.nlm.nih.gov/pmc/blobs/a3e4/11185884/5085e4bf2ccc/gr2.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/a3e4/11185884/9c519dfa1f38/gr1.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/a3e4/11185884/5085e4bf2ccc/gr2.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/a3e4/11185884/9c519dfa1f38/gr1.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/a3e4/11185884/5085e4bf2ccc/gr2.jpg

相似文献

[1]
Comparison of Artificial Intelligence to Resident Performance on Upper-Extremity Orthopaedic In-Training Examination Questions.

J Hand Surg Glob Online. 2023-12-11

[2]
Artificial Intelligence in Orthopaedics: Performance of ChatGPT on Text and Image Questions on a Complete AAOS Orthopaedic In-Training Examination (OITE).

J Surg Educ. 2024-11

[3]
ChatGPT Performs at the Level of a Third-Year Orthopaedic Surgery Resident on the Orthopaedic In-Training Examination.

JB JS Open Access. 2023-12-11

[4]
ChatGPT, Bard, and Bing Chat Are Large Language Processing Models That Answered Orthopaedic In-Training Examination Questions With Similar Accuracy to First-Year Orthopaedic Surgery Residents.

Arthroscopy. 2025-3

[5]
Can Artificial Intelligence Pass the American Board of Orthopaedic Surgery Examination? Orthopaedic Residents Versus ChatGPT.

Clin Orthop Relat Res. 2023-8-1

[6]
Evaluating ChatGPT Performance on the Orthopaedic In-Training Examination.

JB JS Open Access. 2023-9-8

[7]
Generative Artificial Intelligence Performs at a Second-Year Orthopedic Resident Level.

Cureus. 2024-3-13

[8]
Performance of Two Artificial Intelligence Generative Language Models on the Orthopaedic In-Training Examination.

Orthopedics. 2024

[9]
Inadequate Performance of ChatGPT on Orthopedic Board-Style Written Exams.

Cureus. 2024-6-18

[10]
Can generative artificial intelligence pass the orthopaedic board examination?

J Orthop. 2023-11-5

引用本文的文献

[1]
Exploring the Current Applications of Artificial Intelligence in Orthopaedic Surgical Training: A Systematic Scoping Review.

Cureus. 2025-4-3

[2]
Examining the Role of Large Language Models in Orthopedics: Systematic Review.

J Med Internet Res. 2024-11-15

本文引用的文献

[1]
Can Artificial Intelligence Pass the American Board of Orthopaedic Surgery Examination? Orthopaedic Residents Versus ChatGPT.

Clin Orthop Relat Res. 2023-8-1

[2]
Performance of ChatGPT on a Radiology Board-style Examination: Insights into Current Strengths and Limitations.

Radiology. 2023-6

[3]
Performance of an Artificial Intelligence Chatbot in Ophthalmic Knowledge Assessment.

JAMA Ophthalmol. 2023-6-1

[4]
ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns.

Healthcare (Basel). 2023-3-19

[5]
Evaluating the Feasibility of ChatGPT in Healthcare: An Analysis of Multiple Clinical and Research Scenarios.

J Med Syst. 2023-3-4

[6]
Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models.

PLOS Digit Health. 2023-2-9

[7]
How Does ChatGPT Perform on the United States Medical Licensing Examination (USMLE)? The Implications of Large Language Models for Medical Education and Knowledge Assessment.

JMIR Med Educ. 2023-2-8

[8]
Design and Application of Artificial Intelligence Technology-Driven Education and Teaching System in Universities.

Comput Math Methods Med. 2022

[9]
An updated analysis of shoulder and elbow questions on the Orthopedic In-Training Examination.

J Shoulder Elbow Surg. 2022-11

[10]
American Board of Orthopaedic Surgery's Initiatives Toward Competency-Based Education.

JB JS Open Access. 2022-5-19

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

推荐工具

医学文档翻译智能文献检索