British surgeons' experiences of a mandatory online workplace based assessment portfolio resurveyed three years on.

INTRODUCTION

An online portfolio, the Intercollegiate Surgical Curriculum Programme (ISCP; http://www.iscp.ac.uk), became mandatory for British surgical trainees 5 years ago, alongside a compulsory £125 (US$200) annual fee. We previously demonstrated widespread dissatisfaction with its 2008 ver. 5. Here we evaluate and contrast user satisfaction with ver. 8.

DESIGN, SETTING, AND PARTICIPANTS: A total of 359 users across all surgical specialties and UK regions were randomly sampled and surveyed in 2011 by online questionnaire regarding ISCP, elogbook (http://www.elogbook.org.uk), and results compared with 539 users surveyed in 2008. Likert 5-point rating scales were largely used and data analyzed using χ(2) tests.

RESULTS

Seventy-nine percent used ISCP and 38% elogbook (http://www.elogbook.org); 201 responders (56%) evaluated ISCP ver. 8; 31% had registered for at least 1 year, and 59% for 3 years or more. Modal ratings were "average" throughout, with the following percentages of responders rating "poor" or worse vs "good" or better the following domains: registration 12% vs 35%; induction procedure 41% vs. 15%; workplace based assessments 36% vs 22%; peer assessment tool 34% vs 25%; recording meetings 34% vs 19%; Helpdesk 11% vs 40%; user friendliness 29% vs 24%. Trainees were neutral about ISCP's impact upon training and 44% thought that ISCP was needed. Statistically significant (p < 0.05) improvements were seen in user satisfaction with ISCP throughout all domains comparing ver. 8 (2011) to ver. 5 (2008).

CONCLUSIONS

The performance of ISCP has improved in the 4 years since its inception with proportionately less negative feedback. British surgeons remain dissatisfied with several of its tools, in particular its workplace-based assessments. Half a decade on, these assessments remain without appropriate evidence of validity despite increasing demands upon trainees to complete quotas of them. With reduced permitted training hours, the growing online bureaucratic burden continues to demoralize busy surgical trainers and trainees.