Department of Medical Humanities, School of Medicine, University of Split, Split, Croatia.
Department of Research in Biomedicine and Health, School of Medicine, University of Split, Split, Croatia.
Sci Rep. 2024 Oct 30;14(1):26046. doi: 10.1038/s41598-024-77576-3.
Recent advancements in artificial intelligence (AI) have led to concerns about its potential misuse in education. As large language models (LLMs) such as ChatGPT and Bard can generate human-like text, researchers and educators noted the potential redundancy of tasking students with writing academic essays. We aimed to explore if the two LLMs could generate unstructured essays on medical students' personal experiences of challenges and ethical dilemmas that are indistinguishable from human-written texts. We collected 47 original student-written essays from which we extracted keywords to develop prompts for the LLMs. We then used these prompts to generate an equivalent number of essays using ChatGPT and Bard. We analysed the essays using the Language Inquiry and Word Count (LIWC) 22 software, extracting the main LIWC summary measures and variables related to social and psychological processes. We also conducted sub-analyses for sixteen student essays that were presumably written entirely or in part by AI, according to two AI detectors. We found that AI-written essays used more language related to affect, authenticity, and analytical thinking compared to student-written essays after we removed AI-co-written student essays from the analysis. We observed that, despite the differences in language characteristics compared to student-written essays, both LLMs are highly effective in generating essays on students' personal experiences and opinions regarding real-life ethical dilemmas.
最近人工智能（AI）的进步引发了人们对其在教育中被滥用的担忧。由于 ChatGPT 和 Bard 等大型语言模型（LLM）可以生成类似人类的文本，研究人员和教育工作者注意到让学生撰写学术论文可能存在冗余。我们旨在探讨这两个 LLM 是否可以生成关于医学生个人经历挑战和伦理困境的非结构化论文，且这些论文与人类撰写的文本无法区分。我们从 47 篇原始学生撰写的论文中收集了关键词，以开发 LLM 的提示。然后，我们使用这些提示使用 ChatGPT 和 Bard 生成了数量相等的论文。我们使用语言查询和词汇计数（LIWC）22 软件分析了这些论文，提取了主要的 LIWC 总结措施和与社会及心理过程相关的变量。我们还根据两个 AI 探测器对十六篇据称完全或部分由 AI 撰写的学生论文进行了子分析。我们发现，在从分析中删除由 AI 合著的学生论文后，与学生撰写的论文相比，AI 撰写的论文在使用与情感、真实性和分析性思维相关的语言方面更多。我们观察到，尽管与学生撰写的论文相比，语言特征存在差异，但这两个 LLM 都非常有效地生成关于学生个人经历和对现实生活中伦理困境的看法的论文。
