Brondani Mario, Alves Claudia, Ribeiro Cecilia, Braga Mariana M, Garcia Renata C Mathes, Ardenghi Thiago, Pattanaporn Komkham
Faculty of Dentistry, Department of Oral Health Sciences, University of British Columbia, Vancouver, Canada.
Faculty of Dentistry, Department of Dentistry II, Federal University of Maranhão, Sao Luis-Maranhao, Brazil.
J Dent Educ. 2024 Dec;88(12):1671-1680. doi: 10.1002/jdd.13663. Epub 2024 Jul 7.
Reflections enable students to gain additional value from a given experience. The use of Chat Generative Pre-training Transformer (ChatGPT, OpenAI Incorporated) has gained momentum, but its impact on dental education is understudied.
To assess whether or not university instructors can differentiate reflections generated by ChatGPT from those generated by students, and to assess whether or not the content of a thematic analysis generated by ChatGPT differs from that generated by qualitative researchers on the same reflections.
Hardcopies of 20 reflections (10 generated by undergraduate dental students and 10 generated by ChatGPT) were distributed to three instructors who had at least 5 years of teaching experience. Instructors were asked to assign either 'ChatGPT' or 'student' to each reflection. Ten of these reflections (five generated by undergraduate dental students and five generated by ChatGPT) were randomly selected and distributed to two qualitative researchers who were asked to perform a brief thematic analysis with codes and themes. The same ten reflections were also thematically analyzed by ChatGPT.
The three instructors correctly determined whether the reflections were student or ChatGPT generated 85% of the time. Most disagreements (40%) happened with the reflections generated by ChatGPT, as the instructors thought to be generated by students. The thematic analyses did not differ substantially when comparing the codes and themes produced by the two researchers with those generated by ChatGPT.
Instructors could differentiate between reflections generated by ChatGPT or by students most of the time. The overall content of a thematic analysis generated by the artificial intelligence program ChatGPT did not differ from that generated by qualitative researchers. Overall, the promising applications of ChatGPT will likely generate a paradigm shift in (dental) health education, research, and practice.
反思能使学生从特定经历中获得额外价值。聊天生成预训练变换器（ChatGPT，OpenAI公司）的使用势头渐盛，但其对牙科教育的影响尚未得到充分研究。
评估大学教师能否区分ChatGPT生成的反思与学生生成的反思，以及评估ChatGPT生成的主题分析内容与定性研究人员对相同反思生成的内容是否存在差异。
将20篇反思的硬拷贝（10篇由牙科本科生生成，10篇由ChatGPT生成）分发给三位具有至少5年教学经验的教师。要求教师为每篇反思标注“ChatGPT”或“学生”。从这些反思中随机选取10篇（5篇由牙科本科生生成，5篇由ChatGPT生成）分发给两位定性研究人员，要求他们用代码和主题进行简要的主题分析。同样的10篇反思也由ChatGPT进行主题分析。
三位教师在85%的情况下能正确判断反思是由学生还是ChatGPT生成的。大多数分歧（40%）发生在ChatGPT生成的反思上，教师误以为是学生生成的。比较两位研究人员生成的代码和主题与ChatGPT生成的代码和主题时，主题分析没有实质性差异。
教师在大多数情况下能够区分ChatGPT生成的反思和学生生成的反思。人工智能程序ChatGPT生成的主题分析的总体内容与定性研究人员生成的内容没有差异。总体而言，ChatGPT的前景应用可能会给（牙科）健康教育、研究和实践带来范式转变。
