Medical University of Vienna, Institute of Artificial Intelligence, Center for Medical Data Science, Währingerstraße 25a, 1090, Vienna, Austria; CeMM Research Center for Molecular Medicine of the Austrian Academy of Sciences, Lazarettgasse 14, 1090, Vienna, Austria.
CeMM Research Center for Molecular Medicine of the Austrian Academy of Sciences, Lazarettgasse 14, 1090, Vienna, Austria.
Comput Biol Med. 2024 Aug;178:108796. doi: 10.1016/j.compbiomed.2024.108796. Epub 2024 Jun 22.
Computational simulation of biological processes can be a valuable tool for accelerating biomedical research, but usually requires extensive domain knowledge and manual adaptation. Large language models (LLMs) such as GPT-4 have proven surprisingly successful for a wide range of tasks. This study provides proof-of-concept for the use of GPT-4 as a versatile simulator of biological systems.
We introduce SimulateGPT, a proof-of-concept for knowledge-driven simulation across levels of biological organization through structured prompting of GPT-4. We benchmarked our approach against direct GPT-4 inference in blinded qualitative evaluations by domain experts in four scenarios and in two quantitative scenarios with experimental ground truth. The qualitative scenarios included mouse experiments with known outcomes and treatment decision support in sepsis. The quantitative scenarios included prediction of gene essentiality in cancer cells and progression-free survival in cancer patients.
In qualitative experiments, biomedical scientists rated SimulateGPT's predictions favorably over direct GPT-4 inference. In quantitative experiments, SimulateGPT substantially improved classification accuracy for predicting the essentiality of individual genes and increased correlation coefficients and precision in the regression task of predicting progression-free survival.
This proof-of-concept study suggests that LLMs may enable a new class of biomedical simulators. Such text-based simulations appear well suited for modeling and understanding complex living systems that are difficult to describe with physics-based first-principles simulations, but for which extensive knowledge is available as written text. Finally, we propose several directions for further development of LLM-based biomedical simulators, including augmentation through web search retrieval, integrated mathematical modeling, and fine-tuning on experimental data.
计算生物学过程的模拟可以成为加速生物医学研究的有价值的工具，但通常需要广泛的领域知识和手动适应。大型语言模型（LLM），如 GPT-4，已经在广泛的任务中取得了惊人的成功。本研究为使用 GPT-4 作为生物系统的通用模拟器提供了概念验证。
我们引入了 SimulateGPT，这是一种通过对 GPT-4 进行结构化提示来实现跨生物学组织层次知识驱动模拟的概念验证。我们通过四个场景中的领域专家进行的盲态定性评估和两个具有实验基准的定量场景，对我们的方法进行了基准测试。定性场景包括具有已知结果的小鼠实验和脓毒症中的治疗决策支持。定量场景包括预测癌细胞中的基因必需性和癌症患者的无进展生存期。
在定性实验中，生物医学科学家对 SimulateGPT 的预测评价优于直接 GPT-4 推断。在定量实验中，SimulateGPT 大大提高了预测个别基因必需性的分类准确性，并增加了预测无进展生存期的回归任务中的相关系数和精度。
这项概念验证研究表明，LLM 可能能够实现一类新的生物医学模拟器。这种基于文本的模拟似乎非常适合对难以用基于物理原理的第一性原理模拟来描述的复杂生命系统进行建模和理解，但这些系统有大量的文字知识可供使用。最后，我们提出了几种进一步开发基于 LLM 的生物医学模拟器的方向，包括通过网络搜索检索、集成数学建模和对实验数据进行微调来进行增强。
