Zusman Natalie L, Bauer Matthew, Mann Jennah, Goldstein Rachel Y
Jackie and Gene Autry Orthopedic Center, Children's Hospital Los Angeles, Los Angeles, CA.
J Pediatr Soc North Am. 2024 Feb 5;5(4):762. doi: 10.55275/JPOSNA-2023-762. eCollection 2023 Nov.
Artificial intelligence services, such as ChatGPT (generative pre-trained transformer), can provide parents with tailored responses to their pediatric orthopaedic concerns. We undertook a qualitative study to assess the accuracy of the answer provided by ChatGPT in comparison to OrthoKids ("OK"), a patient-facing educational platform governed by the Pediatric Orthopaedic Society of North America (POSNA) for common pediatric orthopaedic conditions. A cross-sectional study was performed from May 26 to June 18, 2023. OK website (orthokids.org) was reviewed and 30 existing questions were collected. The corresponding OK and ChatGPT responses were recorded. Two pediatric orthopaedic surgeons assessed the answer provided by ChatGPT against the OK response. Answers were graded as: AGREE (accurate information; question addressed in full), NEUTRAL (accurate information; question not answered), DISAGREE (information was inaccurate or could be detrimental to patients' health). The evaluators' responses were compiled; discrepancies were adjudicated by a third pediatric orthopaedist. Additional chatbot answer characteristics such as unprompted treatment recommendations, bias, and referral to a healthcare provider were recorded. Data was analyzed using descriptive statistics. The chatbot's answers were agreed upon in 93% of questions. Two responses were felt to be neutral. No responses met disagreement. Unprompted treatment recommendations were included in 55% of its responses (excluding treatment-specific questions). The chatbot encouraged users to "consult with a healthcare professional" in all responses. It was nearly an equal split between recommending a generic provider (46%) in contrast to specifically stating a pediatric orthopaedist (54%). The chatbot was inconsistent in related topics in its provider recommendations, such as recommending a pediatric orthopaedist in 3 of 5 spine conditions. Questions pertaining to common pediatric orthopaedic conditions were accurately represented by a chatbot in comparison to a specialty society-governed website. The knowledge that chatbots deliver appropriate responses is reassuring. However, the chatbot frequently offered unsolicited treatment recommendations whilst simultaneously inconsistently recommending an orthopaedic consultation. We urge caution to parents utilizing artificial intelligence without also consulting a healthcare professional. IV •Artificial intelligence chatbots are becoming increasingly popular, as demonstrated by the rapid rise of publications on the topic in the last 3 months, and they represent a novel patient education online platform.•In comparing 30 common pediatric orthopaedic conditions, >90% of the chatbot's responses were felt to be in agreement with a specialty society's parent-patient-facing education platform.•The chatbot's responses were largely unbiased and referred patients to a healthcare professional. However, the responses lacked references or citing sources for the provided information.
人工智能服务,如ChatGPT(生成式预训练变换器),可以为家长提供针对其小儿骨科问题的定制化回复。我们进行了一项定性研究,以评估ChatGPT提供的答案与OrthoKids(“OK”)相比的准确性,OrthoKids是一个由北美小儿骨科学会(POSNA)管理的面向患者的常见小儿骨科疾病教育平台。2023年5月26日至6月18日进行了一项横断面研究。对OK网站(orthokids.org)进行了审查,并收集了30个现有问题。记录了相应的OK和ChatGPT回复。两名小儿骨科外科医生根据OK回复评估ChatGPT提供的答案。答案分为:同意(信息准确;问题得到全面解答)、中立(信息准确;问题未得到解答)、不同意(信息不准确或可能对患者健康有害)。汇总评估者的回复;差异由第三位小儿骨科医生裁决。记录了聊天机器人答案的其他特征,如主动提供的治疗建议、偏差以及转介给医疗服务提供者。使用描述性统计分析数据。聊天机器人在93%的问题上答案得到认可。有两个回复被认为是中立的。没有回复被判定为不同意。其回复中有55%包含主动提供的治疗建议(不包括特定治疗问题)。聊天机器人在所有回复中都鼓励用户“咨询医疗专业人员”。在推荐普通医疗服务提供者(46%)与明确推荐小儿骨科医生(54%)之间几乎平分秋色。聊天机器人在其推荐医疗服务提供者的相关主题上不一致,例如在5种脊柱疾病中的3种中推荐小儿骨科医生。与一个由专业学会管理的网站相比,聊天机器人对常见小儿骨科疾病相关问题的回答较为准确。知道聊天机器人能提供恰当回复令人安心。然而,聊天机器人经常主动提供治疗建议,同时在推荐骨科会诊方面也不一致。我们敦促家长在使用人工智能时谨慎行事,同时也要咨询医疗专业人员。四、人工智能聊天机器人越来越受欢迎,过去3个月关于该主题的出版物迅速增加就证明了这一点,它们代表了一个新型的在线患者教育平台。•在比较30种常见小儿骨科疾病时,超过90%的聊天机器人回复被认为与一个专业学会面向家长和患者的教育平台一致。•聊天机器人的回复基本无偏差,并将患者转介给医疗专业人员。然而,回复中缺乏对所提供信息的参考文献或引用来源。