Using word clouds to Re-envision letters of recommendation for residency applicants.

OBJECTIVES/HYPOTHESIS: To develop a "word cloud"-based visual letter of recommendation (VLOR) and to evaluate its efficiency in discerning applicant quality compared to narrative letters of recommendation (NLORs).

STUDY DESIGN

Cross-sectional cohort study.

METHODS

NLORs for 48 otolaryngology residency applicants interviewed from the 2016 application cycle were identified and mined for descriptive terms to generate a word cloud, referred to as a VLOR. Eight individuals reviewed and rated a total of 187 blinded NLORs and 48 VLORs on a four-point scale (negative to exceptional). Median VLOR and NLOR scores and the time to review for each candidate were compared using the Wilcoxon signed rank test.

RESULTS

It took significantly more time to review the NLORs in comparison to the VLORs (67 seconds, interquartile range [IQR]: 41-98 seconds vs. 17 seconds, IQR: 11-26 seconds, P < .001). There was no significant difference between median scores for VLORs and NLORs (P = .136). Review time and score correlated positively for VLORs and was statistically significant (ρ = 0.459, P = .001), indicating that more time spent reviewing equates to higher scores. The same relationship appeared with NLORs, but was not statistically significant (ρ = 0.276, P = .058).

CONCLUSIONS

VLORs are a novel and efficient additive tool for screening candidates for otolaryngology residency interview slots. Their scores do not significantly vary from NLOR scores and are significantly faster to evaluate.

LEVEL OF EVIDENCE

2b Laryngoscope, 129:2026-2030, 2019.