Comparing assessments of medical students' potentials as residents made by the residency directors and deans at two schools.

The Association of American Medical Colleges' Committee on Dean's Letters advised in 1988 that the dean's letter should be a letter of evaluation rather than a letter of recommendation. The committee also recommended that the letter contain some form of comparative information to let the residency director know how individual students fared in comparison with their peers. This article reports the results of a 1989 study of the types of agreements between the letter writers and the residency directors of two schools. Three standard methods of providing comparative information were used in their ranking of 20 graduates from the class of 1987 at each school. Ordinal ranking from best to worst students revealed a surprisingly high degree of rank-order agreement, but only for 15 of the 16 participating residency directors. Clustering into fixed groups ("top third," etc.) gave high agreement for top students but weaker agreement for the middle and lower groupings. The advantages and disadvantages of these evaluation methods are discussed.