NIHR ACL, Peninsula College of Medicine & Dentistry, University of Plymouth, C408 Portland Square, Drake Circus, Plymouth, UK.
Med Educ. 2011 Sep;45(9):886-93. doi: 10.1111/j.1365-2923.2011.04023.x.
Multi-source feedback (MSF) and patient feedback (PF) are used increasingly around the world to assess and quality-assure clinical practice. However, concerns about the evidence for their utility pertain to their ability to identify poor performance, the impact of allowing assessees to select their own assessors and the many confounders that may undermine validity.
This study was conducted in conjunction with the National Clinical Assessment Service (NCAS) in the UK and used established MSF and PF instruments to assess doctors in potential difficulty. Multi-source feedback assessors were nominated by both the practitioner (Pnom) and the referring body (RBnom). Demographics were collected to elucidate any differences found. Ratings generated by MSF and PF were compared with one another and with findings of a previous study that provided a normative cohort.
Using MSF, NCAS-assessed doctors scored significantly lower than the reference cohort. Nineteen (28%) NCAS-assessed doctors achieved scores that were less than satisfactory. This rose to 50% when only RBnom assessors were used. Overall, ratings awarded by RBnom assessors were significantly lower than those awarded by Pnom assessors. Collected demographics did not help to explain the difference. Only one NCAS-assessed doctor scored below average according to PF. Doctors in the NCAS-assessed group did not score significantly lower than the reference cohort in PF. Doctor assessment scores awarded by patients were significantly higher than those awarded by colleagues.
Although colleagues appear to report poor performance using MSF, patients fail to report concurrent findings. This challenges the validity of PF as it is currently constructed. Scores in MSF differ significantly depending on whether they are practitioner- or third party-nominated. Previously recognised confounding factors do not help to explain this difference.
多源反馈（MSF）和患者反馈（PF）在全球范围内越来越多地用于评估和保证临床实践质量。然而，关于其效用的证据存在一些问题，包括其识别表现不佳的能力、允许评估者选择自己的评估者的影响，以及可能破坏有效性的许多混杂因素。
这项研究是在英国国家临床评估服务（NCAS）的配合下进行的，使用了经过验证的 MSF 和 PF 工具来评估可能有困难的医生。多源反馈评估者由医生（Pnom）和转介机构（RBnom）共同提名。收集人口统计学数据以阐明发现的任何差异。将 MSF 和 PF 生成的评分进行比较，并与之前提供规范队列的研究结果进行比较。
使用 MSF，NCAS 评估的医生得分明显低于参考队列。19 名（28%）NCAS 评估的医生得分低于满意水平。当仅使用 RBnom 评估者时，这一比例上升到 50%。总体而言，RBnom 评估者授予的评分明显低于 Pnom 评估者授予的评分。收集的人口统计学数据并不能帮助解释差异。只有一名 NCAS 评估的医生根据 PF 得分低于平均水平。NCAS 评估组的医生在 PF 中的得分并不明显低于参考队列。患者评估分数明显高于同事评估分数。
尽管同事们似乎通过 MSF 报告了较差的表现，但患者没有报告同时发现的问题。这挑战了 PF 的有效性，因为它目前的构建方式。根据是否由医生或第三方提名，MSF 的评分差异很大。以前认识到的混杂因素并不能帮助解释这种差异。
