Resident Preparation for the American Board of Anesthesiology Objective Standardized Clinical Examination: A Comparison of Virtual Telesimulation With In-person Simulation.

BACKGROUND

The Objective Structured Clinical Examination (OSCE) is part of the American Board of Anesthesiology (ABA) certification process. A simulated OSCE can aid examination preparation, but the COVID-19 pandemic prevented in-person simulation training. Therefore, we adapted our in-person simulated OSCE (SOSCE) as a Zoom-based telesimulation OSCE (ZOSCE), permitting examinees to participate remotely. Comparing this process with historical in-person SOSCE cohorts, we hypothesized that this telesimulation-based format would still be well received by the trainees as a substitute when it was not possible to provide in-person practice and formative assessment. Subsequently, the ABA proposed a virtual-format OSCE.

METHODS

We conducted our 7-station ZOSCE according to the ABA content outline for all graduating third-year clinical anesthesia residents (CA-3) in 2020. From a main meeting room, the facilitator paired each CA-3 with a faculty proctor, assigned them to their own breakout room for each station, and rotated standardized patients in. The faculty proctor observed the CA-3's performance in real time using an assessment tool with objectives graded on a 0-2 scale. At the conclusion of the ZOSCE, proctors reviewed the assessment tool with the CA-3 and provided personalized global feedback. Assessment tool scores were used to calculate performance data for the study group that were compared with a SOSCE historical cohort from 2017 and 2018. All parties completed a Likert-style evaluation specific to the ZOSCE.

RESULTS

A total of 22 CA-3 residents participated. Mean performance scores ranged from 82.2%-94.9% (minimum = 38%, maximum = 100%). Compared with the historical SOSCE cohort, ZOSCE scores for 5 of 7 stations were not different, but scores in communication with professionals ( = .007) and ultrasound ( < .001) stations were lower. Overall, CA-3 participants rated the learning experience positively and felt it was a reasonable substitution for in-person simulation, with responses similar to those of a historical in-person SOSCE cohort.

CONCLUSIONS

A telesimulation-based practice ZOSCE for formative examination preparation for the ABA OSCE resulted in similar institutional scoring for most stations compared with in-person SOSCE, but some stations may be better practiced in person or require modifications. The virtual format may permit flexible scheduling during nonclinical times or for learners in remote locations. These findings have implications for future formative exercises and the formal summative examination process.