Fu Sunyang, Calley Darren Q, Rasmussen Veronica A, Hamilton Marissa D, Lee Christopher K, Kalla Austin, Liu Hongfang
Department of AI and Informatics, Mayo Clinic, Rochester, MN.
Department of Physical Medicine and Rehabilitation, Mayo Clinic, Rochester, MN.
AMIA Jt Summits Transl Sci Proc. 2023 Jun 16;2023:196-205. eCollection 2023.
Gender stereotyping is the practice of assigning or ascribing specific characteristics, differences, or identities to a person solely based on their gender. Biased conceptions of gender can create barriers to equality and need to be proactively identified and addressed. In biomedical education, letters of recommendation (LOR) are considered an important source for evaluating candidates' past performance. Because LOR is subjective and has no standard formatting requirements for the writer, potential language bias can be introduced. Natural language processing (NLP) offers a promising solution to detect language bias in LOR through automatic extraction of sensitive language and identification of letters with strong biases. In our study, we developed, evaluated, and deployed four NLP different methods (sublanguage analysis, dictionary-based approach, rule-based approach, and deep learning approach) for the extraction of psycholinguistics and thematic characteristics in LORs from three different physical therapy residency programs (Neurologic, Orthopaedic, and Sport) at Mayo Clinic. The evaluation statistics suggest that both MedTaggerIE model and Bidirectional Encoder Representations from Transformers model achieved moderate-high performance across eight different thematic categories. Through the pilot demonstration study, we learned that male writers were more likely to use the words 'intelligence', 'exceptional', and 'pursue' and male applicants were more likely to have the words 'strength', 'interpersonal skills', 'conversations', and 'pursue' in their letters of recommendation. Thematic analysis suggested that male and female writers have significant differences in expressing doubt, motivation, and recommendation. Findings derived from the study needed to be carefully interpreted based on the context of the study setting, residency programs, and data. A follow-up demonstration study is needed to further evaluate and interpret the findings.
性别刻板印象是指仅根据一个人的性别就赋予或归因于其特定特征、差异或身份的做法。有偏见的性别观念会给平等带来障碍,需要积极地加以识别和解决。在生物医学教育中,推荐信被认为是评估候选人过去表现的重要来源。由于推荐信具有主观性,且对撰写者没有标准的格式要求,因此可能会引入潜在的语言偏见。自然语言处理(NLP)提供了一个有前景的解决方案,通过自动提取敏感语言和识别有强烈偏见的信件来检测推荐信中的语言偏见。在我们的研究中,我们开发、评估并部署了四种不同的NLP方法(子语言分析、基于词典的方法、基于规则的方法和深度学习方法),用于从梅奥诊所三个不同的物理治疗住院医师项目(神经科、骨科和运动科)的推荐信中提取心理语言学和主题特征。评估统计表明,MedTaggerIE模型和来自Transformer的双向编码器表征模型在八个不同的主题类别中均取得了中高水平的表现。通过试点示范研究,我们了解到男性撰写者更有可能使用“智力”“杰出”和“追求”等词,而男性申请者的推荐信中更有可能出现“力量”“人际交往能力”“对话”和“追求”等词。主题分析表明,男性和女性撰写者在表达怀疑、动机和推荐方面存在显著差异。需要根据研究背景、住院医师项目和数据对该研究得出的结果进行仔细解读。需要进行后续的示范研究来进一步评估和解读这些结果。