Stern Jonathan M, Fernandez-Perez Antonio, Cruz-Ossa Natalia, Hernandez Victor H, McNamara Colin A, D'Apuzzo Michele R
Department of Orthopaedic Surgery, University of Miami Miller School of Medicine, Miami, FL, USA.
Department of Orthopaedic Surgery, University of Miami Miller School of Medicine, Miami, FL, USA.
J Arthroplasty. 2025 Aug 11. doi: 10.1016/j.arth.2025.07.072.
Artificial intelligence (AI), particularly large language models (LLMs) such as ChatGPT, has expanded across various fields, including medical education and professional applications. However, the extent to which AI is utilized in writing personal statements (PSs) for adult reconstruction fellowship applications remains unclear. This study aimed to analyze the prevalence of AI-generated text in PSs submitted to our institution before and after the release of ChatGPT.
We retrospectively reviewed PSs submitted to our institution's adult reconstruction fellowship from 2021 to 2025. The PSs were divided into two cohorts: Pre-PS (2021 to 2022) and Post-PS (2024 to 2025). The PSs from 2023 were excluded due to uncertainty in AI adoption. All PSs were analyzed using GPTZero, an AI detection software, to determine the proportion of AI-generated versus human-generated text. Descriptive statistics and comparative analyses were conducted.
A total of 421 PSs were analyzed. The Pre-PS cohort had an average GPTZero score of 99.5% (SD 1.9) human, 0.4% (SD 0.8) AI, and 0.1% (SD 1.8) mixed, while the Post-PS cohort had scores of 83.8% (SD 29.9) human, 15.1% (SD 28.9) AI, and 1.1% mixed (SD 5.2) (P < 0.001). The AI-generated text was significantly more prevalent in the Post-PS cohort compared to the Pre-PS cohort. Additionally, international medical graduates (IMGs) and applicants from non-U.S. residencies demonstrated a higher proportion of AI-generated text in their PSs compared to U.S. applicants (P < 0.001).
The use of AI in PS writing has increased significantly since the release of ChatGPT. Given the role of PSs in candidate selection, these findings highlight the need for transparency, standardized guidelines regarding AI-assisted writing, and re-evaluation of the importance placed on personal statements in candidate selection. Further research should expand to other subspecialties and institutions to assess the broader implications of AI in postgraduate medical education.
人工智能（AI），尤其是诸如ChatGPT之类的大语言模型（LLMs），已在包括医学教育和专业应用在内的各个领域得到广泛应用。然而，在成人重建 fellowship 申请的个人陈述（PSs）撰写中，AI 的使用程度仍不明确。本研究旨在分析在ChatGPT发布前后提交至我们机构的PSs中AI生成文本的流行情况。
我们回顾性审查了2021年至2025年提交至我们机构成人重建 fellowship 的PSs。PSs被分为两个队列：PS前（2021年至2022年）和PS后（2024年至2025年）。由于AI采用情况的不确定性，2023年的PSs被排除。使用AI检测软件GPTZero对所有PSs进行分析，以确定AI生成文本与人工生成文本的比例。进行了描述性统计和比较分析。
共分析了421份PSs。PS前队列的GPTZero平均得分是人工生成占99.5%（标准差1.9），AI生成占0.4%（标准差0.8），混合生成占0.1%（标准差1.8）；而PS后队列的得分是人工生成占83.8%（标准差29.9），AI生成占15.1%（标准差28.9），混合生成占1.1%（标准差5.2）（P<0.001）。与PS前队列相比，PS后队列中AI生成的文本明显更为普遍。此外，与美国申请人相比，国际医学毕业生（IMGs）和来自非美国住院医师培训项目的申请人在其PSs中AI生成文本的比例更高（P<0.001）。
自ChatGPT发布以来，PS撰写中AI的使用显著增加。鉴于PSs在候选人选拔中的作用，这些发现凸显了透明度的必要性、关于AI辅助写作的标准化指南，以及重新评估个人陈述在候选人选拔中的重要性。进一步的研究应扩展到其他亚专业和机构，以评估AI在研究生医学教育中的更广泛影响。
