New York University Langone Health, Department of Obstetrics and Gynecology, Division of Gynecologic Oncology, New York, NY, United States of America.
New York University Langone Health, Department of Obstetrics and Gynecology, Division of Gynecologic Oncology, New York, NY, United States of America.
Gynecol Oncol. 2024 Apr;183:115-119. doi: 10.1016/j.ygyno.2024.04.006. Epub 2024 Apr 26.
Artificial Intelligence (AI) systems such as ChatGPT can take medical examinations and counsel patients regarding medical diagnosis. We aim to quantify the accuracy of the ChatGPT V3.4 in answering commonly asked questions pertaining to genetic testing and counseling for gynecologic cancers.
Forty questions were formulated in conjunction with gynecologic oncologists and adapted from professional society guidelines and ChatGPT version 3.5 was queried, the version that is readily available to the public. The two categories of questions were genetic counseling guidelines and questions pertaining to specific genetic disorders. The answers were scored by two attending Gynecologic Oncologists according to the following scale: 1) correct and comprehensive, 2) correct but not comprehensive, 3) some correct, some incorrect, and 4) completely incorrect. Scoring discrepancies were resolved by additional third reviewer. The proportion of responses earning each score were calculated overall and within each question category.
ChatGPT provided correct and comprehensive answers to 33/40 (82.5%) questions, correct but not comprehensive answers to 6/40 (15%) questions, partially incorrect answers to 1/40 (2.5%) questions, and completely incorrect answers to 0/40 (0%) questions. The genetic counseling category of questions had the highest proportion of answers that were both correct and comprehensive with ChatGPT answering all 20/20 questions with 100% accuracy and were comprehensive in responses. ChatGPT performed equally in the specific genetic disorders category, with 88.2% (15/17) and 66.6% (2/3) correct and comprehensive answers to questions pertaining to hereditary breast and ovarian cancer and Lynch syndrome questions respectively.
ChatGPT accurately answers questions about genetic syndromes, genetic testing, and counseling in majority of the studied questions. These data suggest this powerful tool can be utilized as a patient resource for genetic counseling questions, though more data input from gynecologic oncologists would be needed to educate patients on genetic syndromes.
人工智能（AI）系统，如 ChatGPT，可以进行医学检查并为患者提供医疗诊断方面的建议。我们旨在量化 ChatGPT V3.4 回答与妇科癌症相关的基因检测和咨询的常见问题的准确性。
与妇科肿瘤学家合作制定了 40 个问题，并改编自专业协会的指南，同时查询了公众可获得的 ChatGPT 版本 3.5。这两个类别的问题是遗传咨询指南和特定遗传疾病相关的问题。两位主治妇科肿瘤学家根据以下标准对答案进行评分：1）正确且全面，2）正确但不全面，3）部分正确，部分错误，4）完全错误。评分差异由额外的第三位评审员解决。计算了每个得分的回答比例，包括总体和每个问题类别。
ChatGPT 正确且全面地回答了 40 个问题中的 33 个（82.5%），正确但不全面地回答了 6 个（15%），部分正确地回答了 1 个（2.5%），完全错误地回答了 0 个（0%）。遗传咨询类问题的答案中，正确且全面的比例最高，ChatGPT 正确回答了 20 个中的 20 个问题，准确率为 100%，且答案全面。在特定遗传疾病类别中，ChatGPT 的表现同样出色，正确且全面地回答了 17 个中的 15 个（88.2%）和 3 个中的 2 个（66.6%）与遗传性乳腺癌和卵巢癌以及林奇综合征相关的问题。
ChatGPT 准确回答了研究中大部分问题关于遗传综合征、基因检测和咨询的问题。这些数据表明，这种强大的工具可以作为患者遗传咨询的资源，但需要更多来自妇科肿瘤学家的数据输入，以便为患者提供遗传综合征方面的教育。
