Han Tianyu, Nebelung Sven, Khader Firas, Wang Tianci, Müller-Franzes Gustav, Kuhl Christiane, Försch Sebastian, Kleesiek Jens, Haarburger Christoph, Bressem Keno K, Kather Jakob Nikolas, Truhn Daniel
Department of Diagnostic and Interventional Radiology, University Hospital Aachen, Aachen, Germany.
Institute of Pathology, University Medical Center of the Johannes Gutenberg-University, Mainz, Germany.
NPJ Digit Med. 2024 Oct 23;7(1):288. doi: 10.1038/s41746-024-01282-7.
Large language models (LLMs) have broad medical knowledge and can reason about medical information across many domains, holding promising potential for diverse medical applications in the near future. In this study, we demonstrate a concerning vulnerability of LLMs in medicine. Through targeted manipulation of just 1.1% of the weights of the LLM, we can deliberately inject incorrect biomedical facts. The erroneous information is then propagated in the model's output while maintaining performance on other biomedical tasks. We validate our findings in a set of 1025 incorrect biomedical facts. This peculiar susceptibility raises serious security and trustworthiness concerns for the application of LLMs in healthcare settings. It accentuates the need for robust protective measures, thorough verification mechanisms, and stringent management of access to these models, ensuring their reliable and safe use in medical practice.
大语言模型（LLMs）拥有广泛的医学知识，能够对多个领域的医学信息进行推理，在不久的将来具有在各种医学应用中展现出广阔前景的潜力。在本研究中，我们展示了大语言模型在医学领域存在的一个令人担忧的漏洞。通过仅对大语言模型1.1%的权重进行有针对性的操纵，我们就能故意注入错误的生物医学事实。然后，错误信息会在模型输出中传播，同时在其他生物医学任务上保持性能。我们在一组1025个错误的生物医学事实中验证了我们的发现。这种特殊的易感性引发了对大语言模型在医疗环境中应用的严重安全和可信度担忧。它凸显了采取强有力的保护措施、完善的验证机制以及严格管理对这些模型的访问权限的必要性，以确保它们在医学实践中可靠且安全地使用。
