探讨基于问卷的医师专业表现评估工具的有效性证据：系统评价。

Exploring Validity Evidence Associated With Questionnaire-Based Tools for Assessing the Professional Performance of Physicians: A Systematic Review.

机构信息

M.W. van der Meulen is PhD candidate, Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, School of Health Professions Education, Maastricht University, Maastricht, The Netherlands, and member, Professional Performance Research Group, Medical Psychology, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands; ORCID: https://orcid.org/0000-0003-3636-5469. A. Smirnova is PhD graduate and researcher, Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, School of Health Professions Education, Maastricht University, Maastricht, The Netherlands, and member, Professional Performance Research Group, Medical Psychology, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands; ORCID: https://orcid.org/0000-0003-4491-3007. S. Heeneman is professor, Department of Pathology, Faculty of Health, Medicine and Life Sciences, School of Health Professions Education, Maastricht University, Maastricht, The Netherlands; ORCID: https://orcid.org/0000-0002-6103-8075. M.G.A. oude Egbrink is professor, Department of Physiology, Faculty of Health, Medicine and Life Sciences, School of Health Professions Education, Maastricht University, Maastricht, The Netherlands; ORCID: https://orcid.org/0000-0002-5530-6598. C.P.M. van der Vleuten is professor, Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, School of Health Professions Education, Maastricht University, Maastricht, The Netherlands; ORCID: https://orcid.org/0000-0001-6802-3119. K.M.J.M.H. Lombarts is professor, Professional Performance Research Group, Medical Psychology, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands; ORCID: https://orcid.org/0000-0001-6167-0620.

出版信息

Acad Med. 2019 Sep;94(9):1384-1397. doi: 10.1097/ACM.0000000000002767.

DOI:10.1097/ACM.0000000000002767

PMID:31460937

Abstract

PURPOSE

To collect and examine-using an argument-based validity approach-validity evidence of questionnaire-based tools used to assess physicians' clinical, teaching, and research performance.

METHOD

In October 2016, the authors conducted a systematic search of the literature seeking articles about questionnaire-based tools for assessing physicians' professional performance published from inception to October 2016. They included studies reporting on the validity evidence of tools used to assess physicians' clinical, teaching, and research performance. Using Kane's validity framework, they conducted data extraction based on four inferences in the validity argument: scoring, generalization, extrapolation, and implications.

RESULTS

They included 46 articles on 15 tools assessing clinical performance and 72 articles on 38 tools assessing teaching performance. They found no studies on research performance tools. Only 12 of the tools (23%) gathered evidence on all four components of Kane's validity argument. Validity evidence focused mostly on generalization and extrapolation inferences. Scoring evidence showed mixed results. Evidence on implications was generally missing.

CONCLUSIONS

Based on the argument-based approach to validity, not all questionnaire-based tools seem to support their intended use. Evidence concerning implications of questionnaire-based tools is mostly lacking, thus weakening the argument to use these tools for formative and, especially, for summative assessments of physicians' clinical and teaching performance. More research on implications is needed to strengthen the argument and to provide support for decisions based on these tools, particularly for high-stakes, summative decisions. To meaningfully assess academic physicians in their tripartite role as doctor, teacher, and researcher, additional assessment tools are needed.

摘要

目的

采用基于循证的有效性方法收集并检验用于评估医师临床、教学和科研绩效的基于问卷的工具的有效性证据。

方法

2016 年 10 月，作者对文献进行了系统检索，寻找自 2016 年 10 月之前发表的评估医师专业绩效的基于问卷的工具的相关文章。纳入的研究报告了用于评估医师临床、教学和科研绩效的工具的有效性证据。作者使用 Kane 的有效性框架，根据有效性论证中的四个推断（评分、推广、推断和影响）进行数据提取。

结果

纳入了 15 种评估临床绩效的工具的 46 篇文章和 38 种评估教学绩效的工具的 72 篇文章。他们没有发现研究科研绩效工具的文章。只有 12 种工具（23%）收集了 Kane 有效性论证的所有四个组成部分的证据。有效性证据主要集中在推广和推断推断上。评分证据的结果好坏参半。关于影响的证据通常缺失。

结论

根据基于有效性论证的方法，并非所有基于问卷的工具似乎都支持其预期用途。关于基于问卷的工具的影响的证据大多缺失，从而削弱了将这些工具用于医师临床和教学绩效的形成性评估，尤其是总结性评估的论证。需要更多关于影响的研究来加强论证，并为基于这些工具的决策提供支持，尤其是在高风险、总结性决策方面。需要额外的评估工具才能有意义地评估作为医生、教师和研究人员的三重角色的学术医师。