Validity Argument for a Simulation-Based Objective Structured Clinical Examination Scenario for Evaluation of Surgical Skills in Trauma.

BACKGROUND

Trauma is one of the main causes of death globally, and appropriate surgical care is crucial to impact mortality. However, resident-performed trauma cases have diminished in the last 10 years. Simulation-based tools have proven to be effective to evaluate practical skills in a variety of settings. However, there is a lack of evidence regarding proper validation of trauma surgery models.

OBJECTIVE

The aim of this study was to evaluate under a contemporary validity framework, an objective structured clinical evaluation (OSCE) scenario for the assessment of basic and advanced surgical skills in trauma and emergency surgery.

METHODS

An OSCE-type simulation assessment program was developed incorporating six stations representing basic and advanced surgical skills that are essential in trauma surgery. Each station was designed using ex-vivo animal tissue. The stations included basic knots and sutures, bowel resection and anastomosis, vascular end-to-end anastomosis, lung injury repair, cardiac injury repair, and laparoscopic suturing. Eight postgraduate year 2 (PY-2), eight recently graduated surgeons (RGS), and 3 experts were recruited, and their performance was blindly assessed by experts using the validated general rating scale OSATS (Objective Structured Assessment of Technical Skills) as well as the time taken to complete the procedure.

RESULTS

Significant differences were identified among groups. The average OSATS score was 82 for the PY2 group, 113 for the RGS group, and 147 for the experts (P < 0.01). The average procedural time to complete all the stations was 98 minutes for the PY2 group, 68 minutes for the RGS group, and 35 minutes for the expert surgeons (P < 0.01).

CONCLUSION

An OSCE scenario designed using ex-vivo tissue met 4 out of 5 criteria of the Messick validity framework: content, relation to other variables, response process and consequences of the test. The results show it is a valid strategy for the evaluation of practical skills in trauma surgery.