IQ healthcare, Radboud University Nijmegen Medical Centre, Nijmegen, The Netherlands.
BMC Health Serv Res. 2012 Mar 26;12:80. doi: 10.1186/1472-6963-12-80.
There is a global need to assess physicians' professional performance in actual clinical practice. Valid and reliable instruments are necessary to support these efforts. This study focuses on the reliability and validity, the influences of some sociodemographic biasing factors, associations between self and other evaluations, and the number of evaluations needed for reliable assessment of a physician based on the three instruments used for the multisource assessment of physicians' professional performance in the Netherlands.
This observational validation study of three instruments underlying multisource feedback (MSF) was set in 26 non-academic hospitals in the Netherlands. In total, 146 hospital-based physicians took part in the study. Each physician's professional performance was assessed by peers (physician colleagues), co-workers (including nurses, secretary assistants and other healthcare professionals) and patients. Physicians also completed a self-evaluation. Ratings of 864 peers, 894 co-workers and 1960 patients on MSF were available. We used principal components analysis and methods of classical test theory to evaluate the factor structure, reliability and validity of instruments. We used Pearson's correlation coefficient and linear mixed models to address other objectives.
The peer, co-worker and patient instruments respectively had six factors, three factors and one factor with high internal consistencies (Cronbach's alpha 0.95 - 0.96). It appeared that only 2 percent of variance in the mean ratings could be attributed to biasing factors. Self-ratings were not correlated with peer, co-worker or patient ratings. However, ratings of peers, co-workers and patients were correlated. Five peer evaluations, five co-worker evaluations and 11 patient evaluations are required to achieve reliable results (reliability coefficient ≥ 0.70).
The study demonstrated that the three MSF instruments produced reliable and valid data for evaluating physicians' professional performance in the Netherlands. Scores from peers, co-workers and patients were not correlated with self-evaluations. Future research should examine improvement of performance when using MSF.
全球都需要评估医生在实际临床实践中的专业表现。需要有效的和可靠的工具来支持这些努力。本研究重点关注的是可靠性和有效性，一些社会人口统计学偏见因素的影响，自我评估和其他评估之间的关联，以及基于荷兰医生专业表现多源评估中使用的三种工具，为可靠评估医生所需的评估数量。
这是一项在荷兰 26 家非学术医院进行的三种多源反馈（MSF）工具的观察性验证研究。共有 146 名医院医生参与了这项研究。每位医生的专业表现都由同行（医生同事）、同事（包括护士、秘书助理和其他医疗保健专业人员）和患者进行评估。医生还完成了自我评估。共有 864 名同行、894 名同事和 1960 名患者对 MSF 进行了评分。我们使用主成分分析和经典测试理论的方法来评估工具的因子结构、可靠性和有效性。我们使用 Pearson 相关系数和线性混合模型来解决其他目标。
同行、同事和患者工具分别有六个因素、三个因素和一个因素具有较高的内部一致性（Cronbach's alpha 0.95-0.96）。只有 2%的平均评分差异可以归因于偏见因素。自我评分与同行、同事或患者评分不相关。然而，同行、同事和患者的评分是相关的。需要 5 次同行评价、5 次同事评价和 11 次患者评价才能获得可靠的结果（可靠性系数≥0.70）。
该研究表明，三种 MSF 工具在评估荷兰医生的专业表现方面产生了可靠和有效的数据。同行、同事和患者的评分与自我评估不相关。未来的研究应该检验使用 MSF 时的绩效提高情况。
