Gisladottir Undina, Zietz Michael, Kivelson Sophia, Tanaka Yutaro, Sirdeshmukh Gaurav, Brown Kathleen LaRow, Tatonetti Nicholas P
Department of Biomedical Informatics, Columbia University, New York, NY, USA.
Department of Computational Biomedicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
Drug Saf. 2025 Sep 2. doi: 10.1007/s40264-025-01594-x.
Adverse drug reactions (ADRs), including those resulting from drug interactions, remain a leading cause of morbidity and mortality. Structured product labels (SPLs) serve as a primary source for drug safety information. Having machine-readable product labels, including adverse reactions (ARs) and drug interactions, readily available would allow researchers to streamline medication safety studies. However, extracting this information is complex and requires the use of natural language processing (NLP) methods.
In this study, we explored the application of generative language models in the extraction of drug safety information from SPLs.
We compared multiple generative LLMs (GPT, Llama, and Mixtral) to two baseline methods in the task of extracting adverse reactions (ARs) from SPLs. We explored various factors, such as prompting strategies and term complexity, that impact the performance of these models in the extraction of ARs. Finally, we explored the generative models' capacity to extract drug interactions from a separate section of SPLs without additional fine-tuning or training, demonstrating their flexibility and adaptability for information retrieval.
We found that generative language models, specifically GPT-4, are able to match or exceed the performance of previous state-of-the-art models without additional training or fine-tuning. Additionally, we found that the specific SPL section, surrounding context, and complexity of the AR term impacted the extraction performance. Finally, we demonstrated the generalizability of these models by applying them to a separate task of extracting drug names from the drug interaction section where curated training data are not available.
Generative language models demonstrate significant potential for automating drug safety information extraction from SPLs, offering a promising avenue for improving post-market surveillance and reducing ADRs. Future work should focus on refining prompting strategies and expanding the models' capabilities to handle increasingly complex and nuanced drug safety information.
药物不良反应（ADR），包括药物相互作用导致的不良反应，仍然是发病和死亡的主要原因。结构化产品标签（SPL）是药物安全信息的主要来源。拥有机器可读的产品标签，包括不良反应（AR）和药物相互作用信息，将使研究人员能够简化药物安全性研究。然而，提取这些信息很复杂，需要使用自然语言处理（NLP）方法。
在本研究中，我们探索了生成式语言模型在从SPL中提取药物安全信息方面的应用。
我们在从SPL中提取不良反应（AR）的任务中，将多个生成式语言模型（GPT、Llama和Mixtral）与两种基线方法进行了比较。我们探讨了各种因素，如提示策略和术语复杂性，这些因素会影响这些模型在提取AR方面的性能。最后，我们探索了生成式模型在无需额外微调或训练的情况下，从SPL的单独部分提取药物相互作用的能力，展示了它们在信息检索方面的灵活性和适应性。
我们发现，生成式语言模型，特别是GPT-4，能够在无需额外训练或微调的情况下，达到或超过先前最先进模型的性能。此外，我们发现AR术语的特定SPL部分、周围上下文和复杂性会影响提取性能。最后，我们通过将这些模型应用于从药物相互作用部分提取药物名称的单独任务（此处没有经过整理的训练数据），证明了这些模型的通用性。
生成式语言模型在自动从SPL中提取药物安全信息方面显示出巨大潜力，为改善上市后监测和减少ADR提供了一条有前景的途径。未来的工作应集中在完善提示策略和扩展模型处理日益复杂和细微的药物安全信息的能力上。
