Boston University Chobanian & Avedisian School of Medicine, Boston, MA, USA.
Boston University School of Public Health, Boston, MA, USA.
Eur Arch Otorhinolaryngol. 2024 Nov;281(11):6141-6146. doi: 10.1007/s00405-024-08834-3. Epub 2024 Jul 17.
This study delves into the broader implications of artificial intelligence (AI) text generation technologies, including large language models (LLMs) and chatbots, on the scientific literature of otolaryngology. By observing trends in AI-generated text within published otolaryngology studies, this investigation aims to contextualize the impact of AI-driven tools that are reshaping scientific writing and communication.
Text from 143 original articles published in JAMA Otolaryngology - Head and Neck Surgery was collected, representing periods before and after ChatGPT's release in November 2022. The text from each article's abstract, introduction, methods, results, and discussion were entered into ZeroGPT.com to estimate the percentage of AI-generated content. Statistical analyses, including T-Tests and Fligner-Killeen's tests, were conducted using R.
A significant increase was observed in the mean percentage of AI-generated text post-ChatGPT release, especially in the abstract (from 34.36 to 46.53%, p = 0.004), introduction (from 32.43 to 45.08%, p = 0.010), and discussion sections (from 15.73 to 25.03%, p = 0.015). Publications of authors from non-English speaking countries demonstrated a higher percentage of AI-generated text.
This study found that the advent of ChatGPT has significantly impacted writing practices among researchers publishing in JAMA Otolaryngology - Head and Neck Surgery, raising concerns over the accuracy of AI-created content and potential misinformation risks. This manuscript highlights the evolving dynamics between AI technologies, scientific communication, and publication integrity, emphasizing the urgent need for continued research in this dynamic field. The findings also suggest an increasing reliance on AI tools like ChatGPT, raising questions about their broader implications for scientific publishing.
本研究深入探讨了人工智能（AI）文本生成技术，包括大型语言模型（LLM）和聊天机器人，对耳鼻喉科学科文献的广泛影响。通过观察 AI 生成文本在已发表的耳鼻喉科学研究中的趋势，本研究旨在从 AI 驱动工具正在重塑科学写作和交流的角度来分析其影响。
从《美国医学会耳鼻喉科学杂志-头颈部外科》发表的 143 篇原始文章中收集文本，这些文章分别代表了 ChatGPT 于 2022 年 11 月发布前后的时期。将每篇文章的摘要、引言、方法、结果和讨论部分的文本输入到 ZeroGPT.com 中，以估计 AI 生成内容的百分比。使用 R 进行统计分析，包括 T 检验和 Fligner-Killeen 检验。
在 ChatGPT 发布后，AI 生成文本的平均百分比显著增加，尤其是在摘要（从 34.36%增加到 46.53%，p=0.004）、引言（从 32.43%增加到 45.08%，p=0.010）和讨论部分（从 15.73%增加到 25.03%，p=0.015）。来自非英语国家的作者的出版物显示出更高比例的 AI 生成文本。
本研究发现，ChatGPT 的出现显著影响了在《美国医学会耳鼻喉科学杂志-头颈部外科》上发表的研究人员的写作实践，这引发了对 AI 生成内容准确性和潜在错误信息风险的担忧。本研究稿突出了 AI 技术、科学传播和出版诚信之间不断变化的动态，强调了在这一动态领域持续研究的迫切需要。研究结果还表明对 ChatGPT 等 AI 工具的依赖日益增加，这引发了对其对科学出版更广泛影响的质疑。
