Implicit Bias and the Association of Redaction of Identifiers With Residency Application Screening Scores.

IMPORTANCE

Diversity in the ophthalmology profession is important when providing care for an increasingly diverse patient population. However, implicit bias may inadvertently disadvantage underrepresented applicants during resident recruitment and selection.

OBJECTIVE

To evaluate the association of the redaction of applicant identifiers with the review scores on ophthalmology residency applications as an intervention to address implicit bias.

DESIGN, SETTING, AND PARTICIPANTS: In this quality improvement study, 46 faculty members reviewed randomized sets of 462 redacted and unredacted applications from a single academic institution during the 2019-2020 ophthalmology residency application cycle.

INTERVENTIONS

Applications electronically redacted for applicant identifiers, including name, sex or gender, race and ethnicity, and related terms.

MAIN OUTCOMES AND MEASURES

The main outcome was the distribution of scores on redacted and unredacted applications, stratified by applicant's sex, underrepresentation in medicine (URiM; traditionally comprising American Indian or Alaskan Native, Black, and Hispanic individuals) status, and international medical graduate (IMG) status; the application score β coefficients for redaction and the applicant and reviewer characteristics were calculated. Applications were scored on a scale of 1 to 9, where 1 was the best score and 9 was the worst score. Scores were evaluated for a significant difference based on redaction among female, URiM, and IMG applicants. Linear regression was used to evaluate the adjusted association of redaction, self-reported applicant characteristics, and reviewer characteristics with scores on ophthalmology residency applications.

RESULTS

In this study, 277 applicants (60.0%) were male and 71 (15.4%) had URiM status; 32 faculty reviewers (69.6%) were male and 2 (0.4%) had URiM status. The distribution of scores was similar for redacted vs unredacted applications, with no difference based on sex, URiM status, or IMG status. Applicant's sex, URiM status, and IMG status had no association with scores in multivariable analysis (sex, β = -0.08; 95% CI, -0.32 to 0.15; P = .26; URiM status, β = -0.03; (95% CI, -0.36 to 0.30; P = .94; and IMG status, β = 0.39; 95% CI, -0.24 to 1.02; P = .35). In adjusted regression, redaction was not associated with differences in scores (β = -0.06 points on a 1-9 scale; 95% CI, -0.22 to 0.10 points; P = .48). Factors most associated with better scores were attending a top 20 medical school (β = -1.06; 95% CI, -1.37 to -0.76; P < .001), holding an additional advanced degree (β = -0.86; 95% CI, -1.22 to -0.50; P < .001), and having a higher United States Medical Licensing Examination Step 1 score (β = -0.35 per 10-point increase; 95% CI, -0.45 to -0.26; P < .001).

CONCLUSIONS AND RELEVANCE

This quality improvement study did not detect an association between the redaction of applicant characteristics on ophthalmology residency applications and the application review scores among underrepresented candidates at this institution. Although the study may not have been powered adequately to find a difference, these findings suggest that the association of redaction with application review scores may be preempted by additional approaches to enhance diversity, including pipeline programs, implicit bias training, diversity-centered culture and priorities, and targeted applicant outreach. Programs may adapt this study design to probe their own application screening biases and track over time before-and-after bias-related interventions.