Anil Abhishek, Saravanan Aswini, Singh Surjit, Shamim Muhammad Aaqib, Tiwari Krishna, Lal Hina, Seshatri Shanmugapriya, Gomaz Simi Bridjit, Karat Thoyyib P, Dwivedi Pradeep, Varthya Shoban Babu, Kaur Rimple Jeet, Satapathy Prakasini, Padhi Bijaya Kumar, Gaidhane Shilpa, Patil Manoj, Khatib Mahalaqua Nazli, Barboza Joshuan J, Sah Ranjit
Department of Pharmacology, All India Institute of Medical Sciences, Jodhpur, Rajasthan-342005, India.
Department of Dermatology, Venereology and Leprosy, All India Institute of Medical Sciences, Jodhpur, Rajasthan-342005, India.
Heliyon. 2023 Aug 24;9(9):e19194. doi: 10.1016/j.heliyon.2023.e19194. eCollection 2023 Sep.
The increasing pressure to publish research has led to a rise in plagiarism incidents, creating a need for effective plagiarism detection software. The importance of this study lies in the high cost variation amongst the available options for plagiarism detection. By uncovering the advantages of these low-cost or free alternatives, researchers could access the appropriate tools for plagiarism detection. This is the first study to compare four plagiarism detection tools and assess factors impacting their effectiveness in identifying plagiarism in AI-generated articles.
A prospective cross-over study was conducted with the primary objective to compare Overall Similarity Index(OSI) of four plagiarism detection software(iThenticate, Grammarly, Small SEO Tools, and DupliChecker) on AI-generated articles. ChatGPT was used to generate 100 articles, ten from each of ten general domains affecting various aspects of life. These were run through four software, recording the OSI. Flesch Reading Ease Score(FRES), Gunning Fog Index(GFI), and Flesch-Kincaid Grade Level(FKGL) were used to assess how factors, such as article length and language complexity, impact plagiarism detection.
The study found significant variation in OSI(p < 0.001) among the four software, with Grammarly having the highest mean rank(3.56) and Small SEO Tools having the lowest(1.67). Pairwise analyses revealed significant differences(p < 0.001) between all pairs except for Small SEO Tools-DupliChecker. Number of words showed a significant correlation with OSI for iThenticate(p < 0.05) but not for the other three. FRES had a positive correlation, and GFI had a negative correlation with OSI by DupliChecker. FKGL negatively correlated with OSI by Small SEO Tools and DupliChecker.
Grammarly is unexpectedly most effective in detecting plagiarism in AI-generated articles compared to the other tools. This could be due to different softwares using diverse data sources. This highlights the potential for lower-cost plagiarism detection tools to be utilized by researchers.
发表研究成果的压力不断增加，导致抄袭事件增多，因此需要有效的抄袭检测软件。本研究的重要性在于，现有抄袭检测工具的成本差异很大。通过揭示这些低成本或免费替代品的优势，研究人员可以获得合适的抄袭检测工具。这是第一项比较四种抄袭检测工具并评估影响它们在识别人工智能生成文章中的抄袭有效性的因素的研究。
进行了一项前瞻性交叉研究，主要目的是比较四种抄袭检测软件（iThenticate、Grammarly、Small SEO Tools和DupliChecker）在人工智能生成文章上的总体相似度指数（OSI）。使用ChatGPT生成100篇文章，从影响生活各个方面的十个一般领域中各生成十篇。这些文章通过四种软件运行，记录OSI。使用弗莱什易读性分数（FRES）、冈宁雾度指数（GFI）和弗莱什-金凯德年级水平（FKGL）来评估文章长度和语言复杂度等因素如何影响抄袭检测。
研究发现四种软件之间的OSI存在显著差异（p < 0.001），Grammarly的平均排名最高（3.56），Small SEO Tools的平均排名最低（1.67）。成对分析显示，除了Small SEO Tools-DupliChecker之外，所有成对之间都存在显著差异（p < 0.001）。单词数量与iThenticate的OSI呈显著正相关（p < 0.05），但与其他三种软件无关。FRES与DupliChecker的OSI呈正相关，GFI与DupliChecker的OSI呈负相关。FKGL与Small SEO Tools和DupliChecker的OSI呈负相关。
与其他工具相比，Grammarly在检测人工智能生成文章中的抄袭方面出人意料地最有效。这可能是由于不同软件使用了不同的数据源。这凸显了研究人员使用低成本抄袭检测工具的潜力。
