Maksimoski Matthew, Noble Anisha Rhea, Smith David F
Division of Pediatric Otolaryngology, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, U.S.A.
Department of Otolaryngology - Head and Neck Surgery, University of Cincinnati, 231 Albert Sabin Way, Cincinnati, USA.
Laryngoscope. 2024 Sep;134(9):4011-4015. doi: 10.1002/lary.31410. Epub 2024 Mar 28.
Investigate the accuracy of ChatGPT in the manner of medical questions related to otolaryngology.
A ChatGPT session was opened within which 93 questions were asked related to otolaryngology topics. Questions were drawn from all major domains within otolaryngology and based upon key action statements (KAS) from clinical practice guidelines (CPGs). Twenty-one "patient-level" questions were also asked of the program. Answers were graded as either "correct," "partially correct," "incorrect," or "non-answer."
Correct answers were given at a rate of 45.5% (71.4% correct in patient-level, 37.3% CPG); partially correct answers at 31.8% (28.6% patient-level, 32.8% CPG); incorrect at 21.6% (0% patient-level, 28.4% CPG); and 1.1% non-answers (% patient-level, 1.5% CPG). There was no difference in the rate of correct answers between CPGs published before or after the period of data collection cited by ChatGPT. CPG-based questions were less likely to be correct than patient-level questions (p = 0.003).
Publicly available artificial intelligence software has become increasingly popular with consumers for everything from story-telling to data collection. In this study, we examined the accuracy of ChatGPT responses to questions related to otolaryngology over 7 domains and 21 published CPGs. Physicians and patients should understand the limitations of this software as it applies to otolaryngology, and programmers in future iterations should consider giving greater weight to information published by well-established journals and written by national content experts.
N/A Laryngoscope, 134:4011-4015, 2024.
以与耳鼻喉科相关的医学问题的方式研究ChatGPT的准确性。
开启一个ChatGPT会话，其中提出了93个与耳鼻喉科主题相关的问题。问题取自耳鼻喉科的所有主要领域，并基于临床实践指南（CPG）中的关键行动声明（KAS）。还向该程序提出了21个“患者层面”的问题。答案被评为“正确”“部分正确”“错误”或“无答案”。
正确答案的给出率为45.5%（患者层面为71.4%正确，CPG为37.3%）；部分正确答案为31.8%（患者层面为28.6%，CPG为32.8%）；错误答案为21.6%（患者层面为0%，CPG为28.4%）；无答案为1.1%（患者层面为%，CPG为1.5%）。ChatGPT引用的数据收集期之前或之后发布的CPG之间，正确答案的给出率没有差异。基于CPG的问题比患者层面的问题正确的可能性更小（p = 0.003）。
公开可用的人工智能软件在从讲故事到数据收集等各个方面越来越受消费者欢迎。在本研究中，我们检查了ChatGPT对7个领域和21个已发表的CPG中与耳鼻喉科相关问题的回答准确性。医生和患者应该了解该软件在应用于耳鼻喉科时的局限性，并且未来版本的程序员应该考虑更重视权威期刊发表的、由国家内容专家撰写的信息。
无 《喉镜》，134:4011 - 4015，2024年。
