Faculty evaluations of resident medical knowledge: can they be used to predict American Board of Surgery In-Training Examination performance?

BACKGROUND

The American Board of Surgery In-Training Examination (ABSITE) offers annual assessment of resident medical knowledge. We sought to determine if ongoing end-of-rotation evaluations by faculty of residents' medical knowledge correlate with ABSITE performance.

METHODS

Retrospective cross-sectional study was conducted over 3 years at 2 institutions. Faculty rated residents' clinical knowledge as part of a global summative evaluation. The intraclass correlation coefficient and convergent validity between faculty evaluations and ABSITE performance were assessed.

RESULTS

A total of 1,562 faculty evaluations were completed for about 147 residents. There was poor agreement among faculty for each resident, with intraclass correlation coefficients of less than 0.2. Spearman's correlation coefficient was calculated for evaluations and ABSITE scores and were found to be weakly correlative at one institution and not correlated at all at the other. Finally, evaluations across quartiles of resident ABSITE scores were examined and show no correlation.

CONCLUSION

Faculty evaluations of resident medical knowledge correlate poorly with resident ABSITE performance, and should not be used as an ongoing predictive tool.