Suppr超能文献

医学中的生成式人工智能:是开拓性进展还是延续历史错误?评估隐性偏见的横断面研究

Generative AI in Medicine: Pioneering Progress or Perpetuating Historical Inaccuracies? Cross-Sectional Study Evaluating Implicit Bias.

作者信息

Sutera Philip, Bhatia Rohini, Lin Timothy, Chang Leslie, Brown Andrea, Jagsi Reshma

机构信息

Department of Radiation Oncology, University of Rochester Medical Center, Rochester, NY, United States.

Department of Radiation Oncology, Emory Winship Cancer Institute, Emory University, 1365 Clifton Road, Atlanta, GA, 30322, United States, 1 404-778-3630.

出版信息

JMIR AI. 2025 Jun 24;4:e56891. doi: 10.2196/56891.

Abstract

BACKGROUND

Generative artificial intelligence (gAI) models, such as DALL-E 2, are promising tools that can generate novel images or artwork based on text input. However, caution is warranted, as these tools generate information based on historical data and are thus at risk of propagating past learned inequities. Women in medicine have routinely been underrepresented in academic and clinical medicine and the stereotype of a male physician persists.

OBJECTIVE

The primary objective is to evaluate implicit bias among gAI across medical specialties.

METHODS

To evaluate for potential implicit bias, 100 photographs for each medical specialty were generated using the gAI platform DALL-E2. For each specialty, DALL-E2 was queried with "An American [specialty name]." Our primary endpoint was to compare the gender distribution of gAI photos to the current distribution in the United States. Our secondary endpoint included evaluating the racial distribution. gAI photos were classified according to perceived gender and race based on a unanimous consensus among a diverse group of medical residents. The proportion of gAI women subjects was compared for each medical specialty to the most recent Association of American Medical Colleges report for physician workforce and active residents using χ2 analysis.

RESULTS

A total of 1900 photos across 19 medical specialties were generated. Compared to physician workforce data, AI significantly overrepresented women in 7/19 specialties and underrepresented women in 6/19 specialties. Women were significantly underrepresented compared to the physician workforce by 18%, 18%, and 27% in internal medicine, family medicine, and pediatrics, respectively. Compared to current residents, AI significantly underrepresented women in 12/19 specialties, ranging from 10% to 36%. Additionally, women represented <50% of the demographic for 17/19 specialties by gAI.

CONCLUSIONS

gAI created a sample population of physicians that underrepresented women when compared to both the resident and active physician workforce. Steps must be taken to train datasets in order to represent the diversity of the incoming physician workforce.

摘要

背景

生成式人工智能(gAI)模型,如DALL-E 2,是很有前景的工具,能够根据文本输入生成新颖的图像或艺术品。然而,由于这些工具基于历史数据生成信息,因此存在传播过去习得的不平等现象的风险。医学领域的女性在学术和临床医学中一直代表性不足,男性医生的刻板印象依然存在。

目的

主要目的是评估各医学专业的gAI中的隐性偏见。

方法

为评估潜在的隐性偏见,使用gAI平台DALL-E2为每个医学专业生成100张照片。对于每个专业,向DALL-E2查询“一位美国[专业名称]”。我们的主要终点是将gAI照片的性别分布与美国目前的分布进行比较。我们的次要终点包括评估种族分布。gAI照片根据一组不同的住院医生的一致共识,按照感知到的性别和种族进行分类。使用χ2分析,将每个医学专业的gAI女性受试者比例与美国医学院协会关于医生劳动力和在职住院医生的最新报告进行比较。

结果

共生成了19个医学专业的1900张照片。与医生劳动力数据相比,人工智能在19个专业中的7个专业中女性比例显著过高,在19个专业中的6个专业中女性比例过低。与医生劳动力相比,内科、家庭医学和儿科的女性比例分别显著低18%、18%和27%。与目前的住院医生相比,人工智能在19个专业中的12个专业中女性比例显著过低,范围从10%到36%。此外,在gAI生成的19个专业中,有17个专业的女性占人口比例不到50%。

结论

与住院医生和在职医生劳动力相比,gAI创建的医生样本群体中女性代表性不足。必须采取措施训练数据集,以体现未来医生劳动力的多样性。

相似文献

2
A Shadow of Doubt: Is There Implicit Bias Among Orthopaedic Surgery Faculty and Residents Regarding Race and Gender?
Clin Orthop Relat Res. 2024 Jul 1;482(7):1145-1155. doi: 10.1097/CORR.0000000000002933. Epub 2024 Jan 12.
3
Systemic pharmacological treatments for chronic plaque psoriasis: a network meta-analysis.
Cochrane Database Syst Rev. 2021 Apr 19;4(4):CD011535. doi: 10.1002/14651858.CD011535.pub4.
4
Cost-effectiveness of using prognostic information to select women with breast cancer for adjuvant systemic therapy.
Health Technol Assess. 2006 Sep;10(34):iii-iv, ix-xi, 1-204. doi: 10.3310/hta10340.
5
Systemic pharmacological treatments for chronic plaque psoriasis: a network meta-analysis.
Cochrane Database Syst Rev. 2020 Jan 9;1(1):CD011535. doi: 10.1002/14651858.CD011535.pub3.
6
Racial and Ethnic Minorities Underrepresented in Pain Management Guidelines for Total Joint Arthroplasty: A Meta-analysis.
Clin Orthop Relat Res. 2024 Sep 1;482(9):1698-1706. doi: 10.1097/CORR.0000000000003026. Epub 2024 Mar 18.
7
Systemic pharmacological treatments for chronic plaque psoriasis: a network meta-analysis.
Cochrane Database Syst Rev. 2017 Dec 22;12(12):CD011535. doi: 10.1002/14651858.CD011535.pub2.
8
Signs and symptoms to determine if a patient presenting in primary care or hospital outpatient settings has COVID-19.
Cochrane Database Syst Rev. 2022 May 20;5(5):CD013665. doi: 10.1002/14651858.CD013665.pub3.
10
Drugs for preventing postoperative nausea and vomiting in adults after general anaesthesia: a network meta-analysis.
Cochrane Database Syst Rev. 2020 Oct 19;10(10):CD012859. doi: 10.1002/14651858.CD012859.pub2.

本文引用的文献

1
Demographic Representation of Generative Artificial Intelligence Images of Physicians.
JAMA Netw Open. 2024 Aug 1;7(8):e2425993. doi: 10.1001/jamanetworkopen.2024.25993.
3
AI image generators often give racist and sexist results: can they be fixed?
Nature. 2024 Mar;627(8005):722-725. doi: 10.1038/d41586-024-00674-9.
4
Propagation of societal gender inequality by internet search algorithms.
Proc Natl Acad Sci U S A. 2022 Jul 19;119(29):e2204529119. doi: 10.1073/pnas.2204529119. Epub 2022 Jul 12.
5
Representation of Women and Underrepresented Groups in US Academic Medicine by Specialty.
JAMA Netw Open. 2021 Aug 2;4(8):e2123512. doi: 10.1001/jamanetworkopen.2021.23512.
6
Estimation and Comparison of Current and Future Racial/Ethnic Representation in the US Health Care Workforce.
JAMA Netw Open. 2021 Mar 1;4(3):e213789. doi: 10.1001/jamanetworkopen.2021.3789.
7
When people hear "doctor," most still picture a man.
CMAJ. 2019 Mar 11;191(10):E295-E296. doi: 10.1503/cmaj.109-5723.

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验