The Use of an Objective Structured Clinical Examination With a Standardized Patient Actor to Assess Professionalism and Communication for the Oral Certifying Examination: The Initial Experience of the American Board of Urology.

INTRODUCTION

Promotion of professionalism/communication (P/C) is a strategic initiative within the American Board of Medical Specialties; however, reliable assessment of this competency in the certification process is lacking. In this article, we present the findings of the American Board of Urology's P/C objective structured clinical examination (OSCE), which was implemented on the 2023 and 2024 certifying examinations.

METHODS

The certifying examination was administered to 694 candidates (335 in 2023 and 359 in 2024). Each candidate was administered two 5-item OSCEs and four 10-item standard oral examinations (SOEs). One OSCE focused on P/C with simulated patient (SP) actors and one on diagnosis/imaging. SP actors interacted with examinees on the 10-minute P/C OSCE. A criterion-referenced standard was used for pass/fail decisions, and the Rasch model was used for scoring.

RESULTS

The candidate's mean score and protocol difficulty did not differ significantly from 2023 to 2024. Reliability with the P/C and diagnostic OSCEs was similar both years. In 2023, the diagnostic OSCE had a higher average score than the P/C OSCE, with both OSCEs scoring higher than the SOEs. In 2024, the average diagnostic OSCE score was higher than that of the P/C OSCE and SOEs, which had similar scores. There was low correlation between the OSCEs and SOEs both years.

CONCLUSIONS

The initial experience of P/C OSCEs on the American Board of Urology certification examination showed acceptable scoring, reliability, and low correlation with standard protocols. Initial data suggest that the P/C OSCE is a distinct construct from SOEs that specifically assesses P/C skills in the oral certification process.