Department of Surgery, Breast Surgical Oncology, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA.
School of Medicine and Dentistry, University of Rochester, Rochester, New York, USA.
J Surg Oncol. 2024 Aug;130(2):188-203. doi: 10.1002/jso.27715. Epub 2024 Jun 4.
Artificial intelligence (AI)-driven chatbots, capable of simulating human-like conversations, are becoming more prevalent in healthcare. While this technology offers potential benefits in patient engagement and information accessibility, it raises concerns about potential misuse, misinformation, inaccuracies, and ethical challenges.
This study evaluated a publicly available AI chatbot, ChatGPT, in its responses to nine questions related to breast cancer surgery selected from the American Society of Breast Surgeons' frequently asked questions (FAQ) patient education website. Four breast surgical oncologists assessed the responses for accuracy and reliability using a five-point Likert scale and the Patient Education Materials Assessment (PEMAT) Tool.
The average reliability score for ChatGPT in answering breast cancer surgery questions was 3.98 out of 5.00. Surgeons unanimously found the responses understandable and actionable per the PEMAT criteria. The consensus found ChatGPT's overall performance was appropriate, with minor or no inaccuracies.
ChatGPT demonstrates good reliability in responding to breast cancer surgery queries, with minor, nonharmful inaccuracies. Its answers are accurate, clear, and easy to comprehend. Notably, ChatGPT acknowledged its informational role and did not attempt to replace medical advice or discourage users from seeking input from a healthcare professional.
人工智能（AI）驱动的聊天机器人能够模拟类似人类的对话，在医疗保健领域越来越普及。虽然这项技术在患者参与和信息获取方面具有潜在的好处，但它也引发了对潜在滥用、错误信息、不准确和道德挑战的担忧。
本研究评估了一个名为 ChatGPT 的公共可用 AI 聊天机器人，它对从美国乳腺外科学会（ASBrS）常见问题解答（FAQ）患者教育网站中选择的九个与乳腺癌手术相关的问题的回答。四位乳腺外科肿瘤学家使用五点李克特量表和患者教育材料评估（PEMAT）工具评估了这些回答的准确性和可靠性。
ChatGPT 在回答乳腺癌手术问题方面的平均可靠性得分为 3.98 分（满分 5.00 分）。根据 PEMAT 标准，外科医生一致认为回答易于理解且可操作。共识认为 ChatGPT 的整体表现是合适的，只有轻微或没有不准确的地方。
ChatGPT 在回答乳腺癌手术查询方面表现出良好的可靠性，只有轻微的、无害的不准确之处。它的答案准确、清晰且易于理解。值得注意的是，ChatGPT 承认其信息角色，并未试图替代医疗建议或劝阻用户不向医疗保健专业人员寻求意见。
