Department of Medicine for the Elderly, Imperial College Healthcare Trust, London, UK.
Med Educ. 2011 Dec;45(12):1190-8. doi: 10.1111/j.1365-2923.2011.04056.x. Epub 2011 Oct 13.
Workplace-based assessment (WPBA) is an increasingly important part of postgraduate medical training and its results may be used as evidence of professional competence. This study evaluates the ability of WPBA to distinguish UK Foundation Programme (FP) doctors with training difficulties and its effectiveness as a surrogate marker for deficiencies in professional competence.
We conducted a retrospective observational study using anonymised records for 1646 trainees in a single UK postgraduate deanery. Data for WPBAs conducted from August 2005 to April 2009 were extracted from the e-portfolio database. These data included all scores submitted by trainees in FP years 1 and 2 on mini-clinical evaluation exercise (mini-CEX), case-based discussion (CbD), direct observation of procedural skills (DOPS) and mini-peer assessment tool (mini-PAT) assessments. Records of trainees in difficulty, as identified by their educational supervisors, were tagged as index cases. Main outcome measures were odds ratios (ORs) for associations between mean WPBA scores and training difficulties. Further analyses by the reported aetiology of the training difficulty (health-, conduct- or performance-related) were performed.
Of the 1646 trainees, 92 had been identified as being in difficulty. Mean CbD and mini-CEX scores were lower for trainees in difficulty and an association was found between identified training difficulties and average scores on the mini-CEX (OR = 0.54; p = 0.034) and CbD (OR = 0.39; p = 0.002). A receiver operator characteristic curve analysis of mean WPBA scores for diagnosing 'in difficulty' status yielded an area under the curve of 0.64, indicating weak predictive value. There was no statistical evidence that mean scores on DOPS and mini-PAT assessments differed between the two groups.
Analysis of a large dataset of WPBA scores revealed significant associations between training difficulties and lower mean scores on both the mini-CEX and CbD. Models show that using WPBA scores is, however, not a valid way of screening for trainees in difficulty. Workplace-based assessments have value as formative assessments that prompt supervision, feedback and reflection. They should not be relied upon to certify competence and their use for such ends may reduce their effectiveness in training. Their results should be interpreted in the context of multiple other methods of assessment, with the aim of achieving a genuinely holistic and representative assessment of professional competence.
基于工作场所的评估（WPBA）是研究生医学培训中越来越重要的一部分，其结果可作为专业能力的证据。本研究评估 WPBA 区分有培训困难的英国基础计划（FP）医生的能力及其作为专业能力缺陷替代指标的有效性。
我们使用单个英国研究生教务长的 1646 名受训者的匿名记录进行了回顾性观察研究。从 2005 年 8 月至 2009 年 4 月，从电子档案数据库中提取了 WPBA 的数据。这些数据包括 FP 第 1 年和第 2 年的学员在迷你临床评估练习（mini-CEX）、基于案例的讨论（CbD）、程序性技能直接观察（DOPS）和迷你同行评估工具（mini-PAT）评估中提交的所有分数。记录由教育主管标记为索引案例的有困难的学员。主要观察指标是 WPBA 分数与培训困难之间的关联的平均值的优势比（OR）。进一步根据培训困难的报告病因（健康、行为或表现相关）进行了分析。
在 1646 名学员中，有 92 名被确定为有困难。有困难的学员的 CbD 和 mini-CEX 平均分较低，并且发现培训困难与 mini-CEX（OR=0.54;p=0.034）和 CbD（OR=0.39;p=0.002）的平均得分之间存在关联。基于 WPBA 分数诊断“困难”状态的受试者工作特征曲线分析得出曲线下面积为 0.64，表明预测值较弱。没有统计学证据表明 DOPS 和 mini-PAT 评估的平均值在两组之间存在差异。
对 WPBA 分数的大型数据集进行分析表明，培训困难与 mini-CEX 和 CbD 的平均分较低之间存在显著关联。模型表明，使用 WPBA 分数不是筛选困难学员的有效方法。基于工作场所的评估具有作为形成性评估的价值，可以提示监督、反馈和反思。不应依赖它们来证明能力，并且将其用于此类目的可能会降低它们在培训中的有效性。应根据其他多种评估方法来解释其结果，目的是实现对专业能力的真正全面和有代表性的评估。
