Omiye Jesutofunmi A, Lester Jenna C, Spichak Simon, Rotemberg Veronica, Daneshjou Roxana
Department of Dermatology, Stanford School of Medicine, Stanford, CA, USA.
Department of Biomedical Data Science, Stanford School of Medicine, Stanford, CA, USA.
NPJ Digit Med. 2023 Oct 20;6(1):195. doi: 10.1038/s41746-023-00939-z.
Large language models (LLMs) are being integrated into healthcare systems; but these models may recapitulate harmful, race-based medicine. The objective of this study is to assess whether four commercially available large language models (LLMs) propagate harmful, inaccurate, race-based content when responding to eight different scenarios that check for race-based medicine or widespread misconceptions around race. Questions were derived from discussions among four physician experts and prior work on race-based medical misconceptions believed by medical trainees. We assessed four large language models with nine different questions that were interrogated five times each with a total of 45 responses per model. All models had examples of perpetuating race-based medicine in their responses. Models were not always consistent in their responses when asked the same question repeatedly. LLMs are being proposed for use in the healthcare setting, with some models already connecting to electronic health record systems. However, this study shows that based on our findings, these LLMs could potentially cause harm by perpetuating debunked, racist ideas.
大语言模型（LLMs）正在被整合到医疗系统中；但这些模型可能会重现有害的、基于种族的医学观念。本研究的目的是评估四个商用大语言模型在回应八个不同场景时是否会传播有害的、不准确的、基于种族的内容，这些场景用于检查基于种族的医学观念或围绕种族的普遍误解。问题源自四位医师专家的讨论以及医学实习生所相信的关于基于种族的医学误解的先前研究。我们用九个不同的问题评估了四个大语言模型，每个问题被询问五次，每个模型总共得到45个回答。所有模型的回答中都有延续基于种族的医学观念的例子。当被反复问到相同问题时，模型的回答并不总是一致。有人提议在医疗环境中使用大语言模型，一些模型已经连接到电子健康记录系统。然而，这项研究表明，基于我们的发现，这些大语言模型可能会因延续已被揭穿的种族主义观念而潜在地造成伤害。
