Vital signs: how early can resident evaluation predict acquisition of competency in surgical pathology?

OBJECTIVES

To determine whether a longitudinal, case-based evaluation system can predict acquisition of competency in surgical pathology and how trainees at risk can be identified early.

DESIGN

Data were collected for trainee performance on surgical pathology cases (how well their diagnosis agreed with the faculty diagnosis) and compared with training outcomes. Negative training outcomes included failure to complete the residency, failure to pass the anatomic pathology component of the American Board of Pathology examination, and/or failure to obtain or hold a position immediately following training.

FINDINGS

Thirty-three trainees recorded diagnoses for 54 326 surgical pathology cases, with outcome data available for 15 residents. Mean case-based performance was significantly higher for those with positive outcomes, and outcome status could be predicted as early as postgraduate year-1 (P  =  .0001). Performance on the first postgraduate year-1 rotation was significantly associated with the outcome (P  =  .02). Although trainees with unsuccessful outcomes improved their performance more rapidly, they started below residents with successful outcomes and did not make up the difference during training. There was no significant difference in Step 1 or 2 United States Medical Licensing Examination (USMLE) scores when compared with performance or final outcomes (P  =  .43 and P  =  .68, respectively) and the resident in-service examination (RISE) had limited predictive ability.

DISCUSSION

Differences between successful- and unsuccessful-outcome residents were most evident in early residency, ideal for designing interventions or counseling residents to consider another specialty.

CONCLUSION

Our longitudinal case-based system successfully identified trainees at risk for failure to acquire critical competencies for surgical pathology early in the program.