Department of Obstetrics and Gynecology, Biruni University, Istanbul, Turkey.
Endometriosis Research and Support Organization (Endo Türkiye), Istanbul, Turkey.
Int J Gynaecol Obstet. 2024 May;165(2):691-695. doi: 10.1002/ijgo.15309. Epub 2023 Dec 18.
To evaluate the accuracy and reproducibility of ChatGPT's free version answers about endometriosis for the first time.
Detailed internet searches to identify frequently asked questions (FAQs) about endometriosis have been performed. Scientific questions were prepared in accordance with the European Society of Human Reproduction and Embryology (ESHRE) endometriosis guidelines. An experienced gynecologist gave a score of 1-4 for each ChatGPT answer. The repeatability of ChatGPT answers about endometriosis was analyzed by asking each question twice, and the reproducibility of ChatGPT was accepted as scoring the answer to the same question in the same score category.
A total of 91.4% (n = 71) of all FAQs were answered completely, accurately, and sufficiently. ChatGPT had the highest accuracy in the symptom and diagnosis category (94.1%, 16/17 questions) and the lowest accuracy in the treatment category (81.3%, 13/16 questions). Furthermore, of the 40 questions based on the ESHRE endometriosis guidelines, 27 (67.5%) were classified as grade 1, seven (17.5%) as grade 2, and six (15.0%) as grade 3. The reproducibility rate of FAQs in the prevention, symptoms, and diagnosis, and complications categories was the highest (100% for all categories). The reproducibility rate was the lowest for questions based on the ESHRE endometriosis guidelines (70.0%).
ChatGPT accurately and satisfactorily responded to more than 90% of the questions about endometriosis, but to only 67.5% of questions based on the ESHRE endometriosis guidelines.
首次评估 ChatGPT 免费版回答子宫内膜异位症相关问题的准确性和可重复性。
详细进行了互联网搜索，以确定子宫内膜异位症的常见问题 (FAQ)。根据欧洲人类生殖与胚胎学会 (ESHRE) 的子宫内膜异位症指南准备了科学问题。一位经验丰富的妇科医生对每个 ChatGPT 回答进行了 1-4 分的评分。通过两次询问每个问题来分析 ChatGPT 回答子宫内膜异位症的可重复性，并且将 ChatGPT 的可重复性定义为对相同问题给出相同评分类别的答案。
所有 FAQ 的 91.4%（n=71）被完全、准确和充分地回答。ChatGPT 在症状和诊断类别中的准确性最高（94.1%，17 个问题中的 16 个），在治疗类别中的准确性最低（81.3%，16 个问题中的 13 个）。此外，在基于 ESHRE 子宫内膜异位症指南的 40 个问题中，27 个（67.5%）被归类为 1 级，7 个（17.5%）为 2 级，6 个（15.0%）为 3 级。在预防、症状和诊断以及并发症类别中，FAQ 的可重复性最高（所有类别均为 100%）。基于 ESHRE 子宫内膜异位症指南的问题的可重复性最低（70.0%）。
ChatGPT 准确且满意地回答了超过 90%的子宫内膜异位症相关问题，但仅回答了基于 ESHRE 子宫内膜异位症指南的问题的 67.5%。
