Centre for Medical and Health Sciences Education, Faculty of Medical and Health Sciences, University of Auckland, 2 Park Rd, Grafton, Auckland 1010, New Zealand Department of Anaesthesia, Auckland City Hospital, 2 Park Rd, Grafton, Auckland 1010, New Zealand
Department of Anaesthesia, Auckland City Hospital, 2 Park Rd, Grafton, Auckland 1010, New Zealand.
Br J Anaesth. 2014 Jun;112(6):1083-91. doi: 10.1093/bja/aeu052. Epub 2014 Mar 17.
The value of workplace-based assessments such as the mini-clinical evaluation exercise (mini-CEX), and clinicians' confidence and engagement in the process, has been constrained by low reliability and limited capacity to identify underperforming trainees. We proposed that changing the way supervisors make judgements about trainees would improve score reliability and identification of underperformers. Anaesthetists regularly make decisions about the level of trainee independence with a case, based on how closely they need to supervise them. We therefore used this as the basis for a new scoring system.
We analysed 338 mini-CEXs where supervisors scored trainees using the conventional system, and also scored trainee independence, based on the need for direct, or more distant, supervision. As supervisory requirements depend on case difficulty, we then compared the actual trainee independence score and the expected trainee independence score obtained externally.
Compared with the conventional scoring system used in previous studies, reliability was very substantially improved using a system based on a trainee's level of independence with a case. Reliability improved further when this score was corrected for case difficulty. Furthermore, the new scoring system overcame the previously identified problem of assessor leniency and identified a number of trainees performing below expectations.
Supervisors' judgements on trainee independence with a case, based on the need for direct or more distant supervision, can generate reliable scores of trainee ability without the need for an onerous number of assessments, identify trainees performing below expectations, and track trainee progress towards independent specialist practice.
基于工作场所的评估（如迷你临床评估练习（mini-CEX））的价值，以及临床医生对该过程的信心和参与度，由于可靠性低且识别表现不佳的受训者的能力有限，一直受到限制。我们提出改变主管对受训者进行判断的方式将提高分数的可靠性和识别表现不佳者的能力。麻醉师通常根据需要密切监督他们的程度，对每个病例中受训者的独立性水平做出决定。因此，我们以此为基础制定了一个新的评分系统。
我们分析了 338 个迷你临床评估练习，其中主管使用传统系统对受训者进行评分，还根据直接或更远距离监督的需要，对受训者的独立性进行评分。由于监督要求取决于病例的难度，因此我们比较了实际的受训者独立性评分和从外部获得的预期的受训者独立性评分。
与之前研究中使用的传统评分系统相比，使用基于病例中受训者独立性水平的系统，可靠性得到了非常显著的提高。当对病例难度进行评分校正时，可靠性进一步提高。此外，新的评分系统克服了之前评估员宽容性的问题，并确定了一些表现低于预期的受训者。
基于直接或更远距离监督的需要，主管对病例中受训者独立性的判断，可以在无需大量评估的情况下产生可靠的受训者能力分数，识别表现低于预期的受训者，并跟踪受训者向独立专业实践的进展。
