Development of simulation scenarios for surgeons' non-technical skills evaluation.

PURPOSE

The purpose of this study was to identify specific simulated scenarios and events for surgeons that discretely and effectively measure non-technical skills (NTS) constructs and appropriately measure NTS using specific behavioral anchors.

METHODS

Over the course of two rounds, experts in NTS accumulated a comprehensive list of simulated scenarios used at our institution to evaluate NTS. Utilizing a survey, experts were able to review scenarios, events, and behavioral anchors to evaluate their agreement with the appropriateness of the events and effectiveness of the anchors. Coefficient of variation (CV) was used to evaluate agreement between raters. The highest rated events were identified.

RESULTS

The CV between raters was moderate, and further inspection revealed that 86% of event ratings had discordance within ± 1 point. The highest rated events in terms of appropriateness and behavioral anchor effectiveness evaluated surgeons' leadership and communication.

CONCLUSIONS

In this study, experts identified three simulated events that isolate and discretely evaluate specific NTS among surgeons. In the future, these scenarios can be used to effectively evaluate surgeons' NTS.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s44186-025-00390-6.