Montiel-Romero Santiago, Rajme-López Sandra, Román-Montes Carla Marina, López-Iñiguez Alvaro, Rivera-Villegas Héctor Orlando, Ochoa-Hein Eric, González-Lara María Fernanda, Ponce-de-León Alfredo, Tamez-Torres Karla María, Martinez-Guerra Bernardo Alfonso
Department of Infectious Diseases, Instituto Nacional de Ciencias Médicas y Nutrición Salvador Zubirán, Mexico City, Mexico, 15 Vasco de Quiroga, Belisario Domínguez Secc 16 Tlalpan, Mexico City, 14080.
Clinical Microbiology Laboratory, Department of Infectious Diseases, Instituto Nacional de Ciencias Médicas y Nutrición Salvador Zubirán, Mexico City, Mexico, 15 Vasco de Quiroga, Belisario Domínguez Secc 16 Tlalpan, Mexico City, 14080.
BMC Infect Dis. 2025 Jan 7;25(1):38. doi: 10.1186/s12879-024-10426-9.
Antimicrobial resistance is a global threat to public health. Chat Generative Pre-trained Transformer (ChatGPT) is a language model tool based on artificial intelligence. ChatGPT could analyze data from antimicrobial susceptibility tests in real time, especially in places where infectious diseases (ID) specialists are not available. We aimed to evaluate the agreement between ChatGPT and ID specialists regarding appropriate antibiotic prescription in simulated cases.
Using data from microbiological isolates recovered in our center, we fabricated 100 cases of patients with different infections. Each case included age, infectious syndrome, isolated organism and complete antibiogram. Considering a precise set of instructions, the cases were introduced into ChatGPT and presented to five ID specialists. For each case, we asked, (1) "What is the most appropriate antibiotic that should be prescribed to the patient in the clinical case?" and (2) "According to the interpretation of the antibiogram, what is the most probable mechanism of resistance?". We then calculated the agreement between ID specialists and ChatGPT, as well as Cohen's kappa coefficient.
Regarding the recommended antibiotic prescription, agreement between ID specialists and ChatGPT was observed in 51/100 cases. The calculated kappa coefficient was 0.48. Agreement on antimicrobial resistance mechanisms was observed in 42/100 cases. The calculated kappa coefficient was 0.39. In a subanalysis according to infectious syndromes and microorganisms, Agreement (range 25 - 80%) and kappa coefficients (range 0.21-0.79) varied.
We found poor agreement between ID specialists and ChatGPT regarding the recommended antibiotic management in simulated clinical cases.
抗菌药物耐药性是对公共卫生的全球性威胁。聊天生成预训练变换器（ChatGPT）是一种基于人工智能的语言模型工具。ChatGPT可以实时分析抗菌药物敏感性试验的数据，尤其是在没有传染病（ID）专家的地方。我们旨在评估在模拟病例中ChatGPT与ID专家在适当抗生素处方方面的一致性。
利用我们中心回收的微生物分离株数据，编造了100例不同感染患者的病例。每个病例包括年龄、感染综合征、分离出的病原体和完整的抗菌谱。按照一组精确的说明，将这些病例输入ChatGPT并呈现给五位ID专家。对于每个病例，我们询问：（1）“在这个临床病例中，应该给患者开的最合适的抗生素是什么？”以及（2）“根据抗菌谱的解读，最可能的耐药机制是什么？”然后我们计算了ID专家与ChatGPT之间的一致性以及科恩kappa系数。
关于推荐的抗生素处方，在100例病例中有51例观察到ID专家与ChatGPT之间存在一致性。计算出的kappa系数为0.48。在100例病例中有42例观察到在抗菌药物耐药机制方面存在一致性。计算出的kappa系数为0.39。在根据感染综合征和微生物进行的亚分析中，一致性（范围为25%-80%）和kappa系数（范围为0.21-0.79）各不相同。
我们发现在模拟临床病例中，ID专家与ChatGPT在推荐的抗生素管理方面一致性较差。
