Department of Pharmaceutical Sciences, University at Buffalo, The State University of New York, Buffalo, NY, 14214-8033, USA.
J Pharmacokinet Pharmacodyn. 2024 Jun;51(3):187-197. doi: 10.1007/s10928-024-09921-y. Epub 2024 Apr 24.
To assess ChatGPT 4.0 (ChatGPT) and Gemini Ultra 1.0 (Gemini) large language models on NONMEM coding tasks relevant to pharmacometrics and clinical pharmacology. ChatGPT and Gemini were assessed on tasks mimicking real-world applications of NONMEM. The tasks ranged from providing a curriculum for learning NONMEM, an overview of NONMEM code structure to generating code. Prompts in lay language to elicit NONMEM code for a linear pharmacokinetic (PK) model with oral administration and a more complex model with two parallel first-order absorption mechanisms were investigated. Reproducibility and the impact of "temperature" hyperparameter settings were assessed. The code was reviewed by two NONMEM experts. ChatGPT and Gemini provided NONMEM curriculum structures combining foundational knowledge with advanced concepts (e.g., covariate modeling and Bayesian approaches) and practical skills including NONMEM code structure and syntax. ChatGPT provided an informative summary of the NONMEM control stream structure and outlined the key NONMEM Translator (NM-TRAN) records needed. ChatGPT and Gemini were able to generate code blocks for the NONMEM control stream from the lay language prompts for the two coding tasks. The control streams contained focal structural and syntax errors that required revision before they could be executed without errors and warnings. The code output from ChatGPT and Gemini was not reproducible, and varying the temperature hyperparameter did not reduce the errors and omissions substantively. Large language models may be useful in pharmacometrics for efficiently generating an initial coding template for modeling projects. However, the output can contain errors and omissions that require correction.
评估 ChatGPT 4.0 (ChatGPT) 和 Gemini Ultra 1.0 (Gemini) 大型语言模型在与药物代谢动力学和临床药理学相关的 NONMEM 编码任务上的表现。ChatGPT 和 Gemini 被评估在模拟 NONMEM 实际应用的任务上的表现。这些任务从提供 NONMEM 学习课程、NONMEM 代码结构概述到生成代码不等。研究了用外行语言提出的、用于具有口服给药的线性药代动力学 (PK) 模型和具有两个平行一阶吸收机制的更复杂模型的 NONMEM 代码生成任务。评估了可重复性和“温度”超参数设置的影响。代码由两位 NONMEM 专家进行了审查。ChatGPT 和 Gemini 提供了结合基础知识和高级概念(例如,协变量建模和贝叶斯方法)以及实用技能(包括 NONMEM 代码结构和语法)的 NONMEM 课程结构。ChatGPT 提供了 NONMEM 控制流结构的信息性摘要,并概述了所需的关键 NONMEM 翻译器 (NM-TRAN) 记录。ChatGPT 和 Gemini 能够根据这两个编码任务的外行语言提示生成 NONMEM 控制流的代码块。控制流包含焦点结构和语法错误,需要在没有错误和警告的情况下执行之前进行修订。ChatGPT 和 Gemini 生成的代码不可重复,并且改变温度超参数并不能实质性地减少错误和遗漏。大型语言模型在药物代谢动力学中可能有助于高效生成建模项目的初始编码模板。然而,输出可能包含需要纠正的错误和遗漏。