The American Society of Colon and Rectal Surgeons Assessment Tool for Performance of Laparoscopic Colectomy.

BACKGROUND

The lack of consensus for performance assessment of laparoscopic colorectal resection is a major impediment to quality improvement.

OBJECTIVE

The purpose of this study was to develop and assess the validity of an evaluation tool for laparoscopic colectomy that is feasible for wide implementation.

DESIGN

During the pilot phase, a small group of experts modified previous assessment tools by watching videos for laparoscopic right colectomy with the following categories of experience: novice (less than 20 cases), intermediate (50-100 cases), and expert (more than 500 cases). After achieving sufficient reliability (κ > 0.8), a user-friendly tool was validated among a large group of blinded, trained experts.

SETTING

The study was conducted through the American Society of Colon and Rectal Surgeons Operative Competency Evaluation Committee.

PATIENTS

Raters were from the Operative Competency Evaluation Committee of the American Society of Colon and Rectal Surgeons.

MAIN OUTCOME MEASURES

Assessment tool reliability and internal consistency were measured.

RESULTS

From October 2014 through February 2015, 4 groups of 5 raters blinded to surgeon skill level evaluated 6 different laparoscopic right colectomy videos (novice = 2, intermediate = 2, expert = 2). The overall Cronbach α was 0.98 (>0.9 = excellent internal consistency). The intraclass correlation for the overall assessment was 0.93 (range, 0.77-0.93) and was >0.74 (excellent) for each step. The average scores (scale, 1-5) for experts were significantly better than those in the intermediate category, with a mean (SD) of 4.51 (0.56) versus 2.94 (0.56; p = 0.003). Videos in the intermediate group scored more favorably than beginner videos for each individual step and overall performance (mean (SD) = 3.00 (0.32) vs 1.78 (0.42); p = 0.006).

LIMITATIONS

The study was limited by rater bias to technique and style.

CONCLUSIONS

The unique and robust methodology in this trial produced an assessment tool that was feasible for raters to use when assessing videotaped laparoscopic right hemicolectomies. The potential applications for this new tool are widespread, including both training and evaluation of competence at the attending level. See Video Abstract at http://links.lww.com/DCR/A369, http://links.lww.com/DCR/A370, http://links.lww.com/DCR/A371.