Maida Marcello, Mori Yuichi, Fuccio Lorenzo, Sferrazza Sandro, Vitello Alessandro, Facciorusso Antonio, Hassan Cesare
Department of Medicine and Surgery, Kore University of Enna, Enna, Italy.
Gastroenterology Unit, Umberto I Hospital, Enna, Italy.
Endosc Int Open. 2025 May 12;13:a25689416. doi: 10.1055/a-2568-9416. eCollection 2025.
Recent studies showed that large language models (LLMs) could enhance understanding of colorectal cancer (CRC) screening, potentially increasing participation rates. However, a limitation of these studies is that questions posed to LLMs are generated by experts. This study aimed to investigate ChatGPT-4o effectiveness in answering CRC screening queries directly generated by patients.
Ten consecutive subjects aged 50 to 69 years who were eligible for the Italian national CRC screening program but not actively involved were enrolled. Four possible scenarios for CRC screening were presented to each participant and they were asked to formulate one question per scenario to gather additional information. These questions were then posed to ChatGPT in two separate sessions. The responses were evaluated by five senior experts, who rated each answer based on three criteria: accuracy, completeness, and comprehensibility, using a 5-point Likert scale. In addition, the same 10 patients who created the questions assessed the answers, rating each response as complete, understandable, and trustworthy on a dichotomous scale (yes/no).
Experts rated the responses with mean scores of 4.1 ± 1.0 for accuracy, 4.2 ± 1.0 for completeness, and 4.3 ± 1.0 for comprehensibility. Patients rated responses as complete in 97.5%, understandable in 95%, and trustworthy in 100% of cases. Consistency over time was confirmed by an 86.8% similarity between session responses.
Despite variability in questions and answers, ChatGPT confirmed good performances in answering CRC screening queries, even when used directly by patients.
近期研究表明，大语言模型（LLMs）能够增进对结直肠癌（CRC）筛查的理解，可能会提高参与率。然而，这些研究的一个局限性在于向大语言模型提出的问题是由专家生成的。本研究旨在调查ChatGPT - 4o在回答患者直接提出的CRC筛查相关问题方面的有效性。
连续纳入10名年龄在50至69岁之间、符合意大利国家CRC筛查计划条件但未积极参与的受试者。向每位参与者呈现四种可能的CRC筛查场景，并要求他们针对每个场景提出一个问题以获取更多信息。然后，在两个不同的环节中将这些问题提交给ChatGPT。由五位资深专家对回答进行评估，他们根据准确性、完整性和可理解性这三个标准，使用5分制李克特量表对每个答案进行评分。此外，提出问题的这10名患者对答案进行评估，在二分制量表（是/否）上对每个回答的完整性、可理解性和可信度进行评分。
专家对回答的评分如下：准确性平均得分为4.1±1.0，完整性平均得分为4.2±1.0，可理解性平均得分为4.3±1.0。患者对回答的评分结果为：97.5%的回答完整，95%的回答可理解，100%的回答可信。不同环节回答之间86.8%的相似度证实了随时间的一致性。
尽管问题和答案存在差异，但ChatGPT在回答CRC筛查相关问题方面表现良好，即使是患者直接使用时也是如此。
