Department of Economics, University of California, Santa Barbara, Santa Barbara, California, United States of America.
Kellogg School of Management, Northwestern University, Evanston, Illinois, United States of America.
PLoS One. 2024 Jul 12;19(7):e0304807. doi: 10.1371/journal.pone.0304807. eCollection 2024.
The rapid advances in Generative AI tools have produced both excitement and worry about how AI will impact academic writing. However, little is known about what norms are emerging around AI use in manuscript preparation or how these norms might be enforced. We address both gaps in the literature by conducting a survey of 271 academics about whether it is necessary to report ChatGPT use in manuscript preparation and by running GPT-modified abstracts from 2,716 published papers through a leading AI detection software to see if these detectors can detect different AI uses in manuscript preparation. We find that most academics do not think that using ChatGPT to fix grammar needs to be reported, but detection software did not always draw this distinction, as abstracts for which GPT was used to fix grammar were often flagged as having a high chance of being written by AI. We also find disagreements among academics on whether more substantial use of ChatGPT to rewrite text needs to be reported, and these differences were related to perceptions of ethics, academic role, and English language background. Finally, we found little difference in their perceptions about reporting ChatGPT and research assistant help, but significant differences in reporting perceptions between these sources of assistance and paid proofreading and other AI assistant tools (Grammarly and Word). Our results suggest that there might be challenges in getting authors to report AI use in manuscript preparation because (i) there is not uniform agreement about what uses of AI should be reported and (ii) journals might have trouble enforcing nuanced reporting requirements using AI detection tools.
生成式人工智能工具的快速发展既让人兴奋，也让人担心人工智能将如何影响学术写作。然而，人们对在 manuscript preparation 中使用人工智能出现的规范知之甚少，也不知道这些规范将如何得到执行。我们通过对 271 名学者进行调查，了解他们是否有必要报告在 manuscript preparation 中使用 ChatGPT 的情况，以及通过将 2716 篇已发表论文的 GPT 修改摘要提交给领先的 AI 检测软件，来了解这些检测器是否可以检测到 manuscript preparation 中的不同 AI 使用情况，从而填补了文献中的这两个空白。我们发现，大多数学者认为使用 ChatGPT 来修正语法不需要报告，但检测软件并不总是能区分这一点，因为使用 GPT 来修正语法的摘要经常被标记为高度可能是由 AI 编写的。我们还发现，学者们对于是否需要报告更大量地使用 ChatGPT 来重写文本存在分歧，这些分歧与对道德、学术角色和英语语言背景的看法有关。最后，我们发现他们对报告 ChatGPT 和研究助理帮助的看法差异不大，但这些来源的帮助与付费校对和其他 AI 助手工具（Grammarly 和 Word）之间的报告看法存在显著差异。我们的研究结果表明，让作者报告在 manuscript preparation 中使用人工智能可能会面临挑战，原因是（i）对于应该报告哪些人工智能的使用还没有统一的意见，（ii）期刊可能难以使用 AI 检测工具来执行细微的报告要求。
