现代马基雅维利主义者？基于 9000 条评论分类的整形和美容外科中 ChatGPT 生成的患者评论的假象。
Modern Machiavelli? The illusion of ChatGPT-generated patient reviews in plastic and aesthetic surgery based on 9000 review classifications.
机构信息
Department of Plastic Surgery and Hand Surgery, Klinikum Rechts der Isar, Technical University of Munich, Munich, Germany; Division of Plastic Surgery, Department of Surgery, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Instituto Ivo Pitanguy, Hospital Santa Casa de Misericórdia, Pontifícia Universidade Católica do Rio de Janeiro, Rio de Janeiro, Brazil.
Instituto Ivo Pitanguy, Hospital Santa Casa de Misericórdia, Pontifícia Universidade Católica do Rio de Janeiro, Rio de Janeiro, Brazil.
出版信息
J Plast Reconstr Aesthet Surg. 2024 Jan;88:99-108. doi: 10.1016/j.bjps.2023.10.119. Epub 2023 Oct 29.
BACKGROUND
Online patient reviews are crucial in guiding individuals who seek plastic surgery, but artificial chatbots pose a threat of disseminating fake reviews. This study aimed to compare real patient feedback with ChatGPT-generated reviews for the top five US plastic surgery procedures.
METHODS
Thirty real patient reviews on rhinoplasty, blepharoplasty, facelift, liposuction, and breast augmentation were collected from RealSelf and used as templates for ChatGPT to generate matching patient reviews. Prolific users (n = 30) assessed 150 pairs of reviews to identify human-written and artificial intelligence (AI)-generated reviews. Patient reviews were further assessed using AI content detector software (Copyleaks AI).
RESULTS
Among the 9000 classification tasks, 64.3% and 35.7% of reviews were classified as authentic and fake, respectively. On an average, the author (human versus machine) was correctly identified in 59.6% of cases, and this poor classification performance was consistent across all procedures. Patients with prior aesthetic treatment showed poorer classification performance than those without (p < 0.05). The mean character count in human-written reviews was significantly higher (p < 0.001) that that in AI-generated reviews, with a significant correlation between character count and participants' accuracy rate (p < 0.001). Emotional timbre of reviews differed significantly with "happiness" being more prevalent in human-written reviews (p < 0.001), and "disappointment" being more prevalent in AI reviews (p = 0.005). Copyleaks AI correctly classified 96.7% and 69.3% of human-written and ChatGPT-generated reviews, respectively.
CONCLUSION
ChatGPT convincingly replicates authentic patient reviews, even deceiving commercial AI detection software. Analyzing emotional tone and review length can help differentiate real from fake reviews, underscoring the need to educate both patients and physicians to prevent misinformation and mistrust.
背景
在线患者评价对于指导寻求整形手术的个体至关重要，但人工智能聊天机器人存在传播虚假评价的威胁。本研究旨在比较美国排名前五的整形手术的真实患者反馈与 ChatGPT 生成的评价。
方法
从 RealSelf 收集了 30 份关于鼻整形术、双眼皮手术、面部提升术、抽脂术和隆胸术的真实患者评价，并将其用作 ChatGPT 生成匹配患者评价的模板。高产用户（n=30）评估了 150 对评价，以识别人工撰写和人工智能（AI）生成的评价。使用 AI 内容检测软件（Copyleaks AI）进一步评估患者评价。
结果
在 9000 项分类任务中，分别有 64.3%和 35.7%的评价被分类为真实和虚假。平均而言，作者（人类与机器）在 59.6%的情况下被正确识别，这种较差的分类性能在所有手术中都一致。有先前美容治疗的患者比没有的患者表现出较差的分类性能（p<0.05）。人工撰写的评价中字符数的平均值明显更高（p<0.001），且字符数与参与者准确率之间存在显著相关性（p<0.001）。评价的情感音色差异显著，人工撰写的评价中更常见“快乐”（p<0.001），而 AI 评价中更常见“失望”（p=0.005）。Copyleaks AI 正确分类了 96.7%的人工撰写和 69.3%的 ChatGPT 生成的评价。
结论
ChatGPT 令人信服地复制了真实的患者评价，甚至欺骗了商业 AI 检测软件。分析情感色调和评价长度可以帮助区分真实和虚假评价，强调需要教育患者和医生以防止错误信息和不信任。
Zotero 插件