Faculty of Medicine and Health, University of Sydney, Sydney, New South Wales, Australia.
Faculty of Science, University of Sydney, Sydney, New South Wales, Australia.
Br J Clin Pharmacol. 2024 Dec;90(12):3361-3366. doi: 10.1111/bcp.16275. Epub 2024 Oct 2.
Drug-drug interactions (DDIs) present a significant health burden, compounded by clinician time constraints and poor patient health literacy. We assessed the ability of ChatGPT (generative artificial intelligence-based large language model) to predict DDIs in a real-world setting. Demographics, diagnoses and prescribed medicines for 120 hospitalized patients were input through three standardized prompts to ChatGPT version 3.5 and compared against pharmacist DDI evaluation to estimate diagnostic accuracy. Area under receiver operating characteristic and inter-rater reliability (Cohen's and Fleiss' kappa coefficients) were calculated. ChatGPT's responses differed based on prompt wording style, with higher sensitivity for prompts mentioning 'drug interaction'. Confusion matrices displayed low true positive and high true negative rates, and there was minimal agreement between ChatGPT and pharmacists (Cohen's kappa values 0.077-0.143). Low sensitivity values suggest a lack of success in identifying DDIs by ChatGPT, and further development is required before it can reliably assess potential DDIs in real-world scenarios.
药物-药物相互作用（DDI）对健康造成了重大负担，加上临床医生时间有限和患者健康素养较差，情况更加复杂。我们评估了 ChatGPT（基于生成式人工智能的大型语言模型）在真实环境中预测 DDI 的能力。通过三个标准化提示，将 120 名住院患者的人口统计学数据、诊断和处方药物输入到 ChatGPT 版本 3.5 中，并与药剂师的 DDI 评估进行比较，以估计诊断准确性。计算了受试者工作特征曲线下的面积和组内相关系数（Cohen's 和 Fleiss' kappa 系数）。ChatGPT 的反应因提示措辞风格而异，提及“药物相互作用”的提示具有更高的敏感性。混淆矩阵显示出低真阳性和高真阴性率，并且 ChatGPT 和药剂师之间的一致性很小（Cohen's kappa 值为 0.077-0.143）。低灵敏度值表明 ChatGPT 在识别 DDI 方面的成功率较低，在能够可靠地评估真实场景中的潜在 DDI 之前，还需要进一步开发。
