Gao Zejun, Ge Jinlin, Xu Ruoshi, Chen Xiaoyan, Cai Zhenzhai
Department of Gastroenterology, Second Affiliated Hospital and Yuying Children's Hospital of Wenzhou Medical University, Wenzhou, China.
Front Med (Lausanne). 2024 Oct 10;11:1489117. doi: 10.3389/fmed.2024.1489117. eCollection 2024.
Advances in artificial intelligence are gradually transforming various fields, but its applicability among ordinary people is unknown. This study aims to explore the ability of a large language model to address related questions.
We created several prompts on the basis of guidelines and the clinical concerns of patients. The capacity of ChatGPT on queries was evaluated by experts. Ordinary people assessed the applicability.
The responses to each prompt in ChatGPT-4 were good in terms of response length and repeatability. There was good agreement in each dimension (Fleiss' kappa ranged from 0.302 to 0.690, < 0.05). The accuracy, completeness, usefulness, comprehension and satisfaction scores of the experts were generally high. Rated usefulness and comprehension among ordinary people were significantly lower than expert, while medical students gave a relatively positive evaluation.
ChatGPT-4 performs well in resolving related questions. Large language models may become an excellent tool for medical students in the future, but still requires further research and validation.
人工智能的进步正在逐渐改变各个领域，但其在普通人中的适用性尚不清楚。本研究旨在探讨大型语言模型解决相关问题的能力。
我们根据指南和患者的临床关注点创建了几个提示。专家评估了ChatGPT对查询的处理能力。普通人评估了其适用性。
ChatGPT-4对每个提示的回答在回答长度和可重复性方面都很好。在每个维度上都有很好的一致性（Fleiss' kappa范围从0.302到0.690，P < 0.05）。专家的准确性、完整性、有用性、理解度和满意度得分普遍较高。普通人的有用性评分和理解度明显低于专家，而医学生给出了相对积极的评价。
ChatGPT-4在解决相关问题方面表现良好。大型语言模型未来可能会成为医学生的优秀工具，但仍需要进一步的研究和验证。
