Möltner Andreas, Wagener Stefan, Burkert Mirka
Medical Faculty of Heidelberg, Baden-Württemberg Center of Excellence for Assessment in Medicine, Heidelberg, Germany.
GMS J Med Educ. 2020 Feb 17;37(1):Doc6. doi: 10.3205/zma001299. eCollection 2020.
Since 2013 a competency-oriented student progress test (SKPT) has been administered at a number of German medical schools. The questions are generated on the basis of a two-dimensional blueprint, on which one axis contains the five competency domains - communicative competence (CO), practical clinical competence (CP), theoretical clinical competence (CT), scientific competence (SC), and professional decision-making competence (PR) - that form part of the competency model of the National Competency-based Catalogue of Learning Objectives for Undergraduate Medicine (NKLM). The feedback for students is structured in part according to these domains. The aim of our study is to examine if the results differentiated by competency domain show adequate measurement accuracy and if the results for the different domains also contain different information. The SKPTs for the years 2013 to 2017, taken by a total of 3,027 students, were examined. The measurement accuracy was determined using the coefficient glb (greatest bound to reliability) and the standard error of measurement; discriminant analysis of the principal components was carried out to demonstrate differentiation between the competency domains. The reliability of the competency domains was above 0.8 for all SKPTs; exceptions to this were seen in two of the tests for CO and PR that had a reliability of 0.7-0.8. The results for all of the individual competency domains differed in their informational content compared to the overall of the other domains; the same applies for all pairwise comparisons, with the exceptions of CP and CT. The SKPT feedback for students that is differentiated by competency domains basically fulfills the requirements for measurement reliability and distinctness. An improvement of the measurement quality for CO and PR and a better differentiation between CP and CT is desirable.
自2013年以来，德国多所医学院校开展了以能力为导向的学生进步测试（SKPT）。测试题目基于二维蓝图生成，其中一个轴包含五个能力领域——沟通能力（CO）、临床实践能力（CP）、临床理论能力（CT）、科研能力（SC）和专业决策能力（PR），这些构成了本科医学国家能力本位学习目标目录（NKLM）能力模型的一部分。学生反馈部分是按照这些领域构建的。我们研究的目的是检验按能力领域区分的结果是否显示出足够的测量准确性，以及不同领域的结果是否也包含不同信息。我们对2013年至2017年共3027名学生参加的SKPT进行了研究。使用系数glb（可靠性的最大界限）和测量标准误差来确定测量准确性；进行主成分判别分析以证明能力领域之间的差异。所有SKPT中各能力领域的可靠性均高于0.8；CO和PR的两项测试除外，其可靠性为0.7 - 0.8。与其他领域的总体情况相比，所有单个能力领域的结果在信息内容上都有所不同；所有两两比较均如此，但CP和CT除外。按能力领域区分的学生SKPT反馈基本满足测量可靠性和区分度的要求。CO和PR的测量质量有待提高，CP和CT之间的区分度也有待改善。
