Alabdulmohsen Dalal M, Almahmudi Mesa A, Alhashim Jehad N, Almahdi Mohammed H, Alkishy Eman F, Almossabeh Modhahir J, Alkhalifah Saleh A
Internal Medicine, College of Medicine, King Faisal University, Hofuf, SAU.
Hematology and Oncology, King Saud Medical City, Riyadh, SAU.
Cureus. 2024 Jul 8;16(7):e64114. doi: 10.7759/cureus.64114. eCollection 2024 Jul.
ChatGPT (OpenAI, San Francisco, CA, USA) is a novel artificial intelligence (AI) application that is used by millions of people, and the numbers are growing by the day. Because it has the potential to be a source of patient information, the study aimed to evaluate the ability of ChatGPT to answer frequently asked questions (FAQs) about asthma with consistent reliability, acceptability, and easy readability.
We collected 30 FAQs about asthma from the Global Initiative for Asthma website. ChatGPT was asked each question twice, by two different users, to assess for consistency. The responses were evaluated by five board-certified internal medicine physicians for reliability and acceptability. The consistency of responses was determined by the differences in evaluation between the two answers to the same question. The readability of all responses was measured using the Flesch Reading Ease Scale (FRES), the Flesch-Kincaid Grade Level (FKGL), and the Simple Measure of Gobbledygook (SMOG).
Sixty responses were collected for evaluation. Fifty-six (93.33%) of the responses were of good reliability. The average rating of the responses was 3.65 out of 4 total points. 78.3% (n=47) of the responses were found acceptable by the evaluators to be the only answer for an asthmatic patient. Only two (6.67%) of the 30 questions had inconsistent answers. The average readability of all responses was determined to be 33.50±14.37 on the FRES, 12.79±2.89 on the FKGL, and 13.47±2.38 on the SMOG.
Compared to online websites, we found that ChatGPT can be a reliable and acceptable source of information for asthma patients in terms of information quality. However, all responses were of difficult readability, and none followed the recommended readability levels. Therefore, the readability of this AI application requires improvement to be more suitable for patients.
ChatGPT（美国加利福尼亚州旧金山的OpenAI公司）是一种新型人工智能应用程序，数百万用户都在使用它，而且其用户数量还在与日俱增。鉴于它有可能成为患者信息的来源，本研究旨在评估ChatGPT回答有关哮喘的常见问题的能力，确保回答具有可靠的一致性、可接受性和易读性。
我们从全球哮喘防治创议组织的网站收集了30个有关哮喘的常见问题。由两名不同的用户向ChatGPT各提出每个问题两次，以评估回答的一致性。五名获得内科专科认证的医生对回答进行可靠性和可接受性评估。回答的一致性由同一问题的两个答案之间的评估差异确定。所有回答的易读性使用弗莱什易读性量表（FRES）、弗莱什-金凯德年级水平量表（FKGL）和简明晦涩度测量法（SMOG）进行测量。
共收集到60个回答用于评估。其中56个（93.33%）回答具有良好的可靠性。回答的平均评分在满分4分中为3.65分。评估人员认为78.3%（n=47）的回答可作为哮喘患者的唯一答案接受。30个问题中只有两个（6.67%）的答案不一致。所有回答的平均易读性在FRES量表上为33.50±14.37，在FKGL量表上为12.79±2.89，在SMOG量表上为13.47±2.38。
与在线网站相比，我们发现就信息质量而言，ChatGPT可以成为哮喘患者可靠且可接受的信息来源。然而，所有回答的易读性都较差，没有一个符合推荐的易读性水平。因此，这种人工智能应用程序的易读性需要改进，以更适合患者。
