Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt Universität zu Berlin, AG Progress Test Medizin, Charitéplatz 1, 10117, Berlin, Germany.
Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt Universität zu Berlin, Institute of Biometry and Clinical Epidemiology, Charitéplatz 1, 10117, Berlin, Germany.
BMC Med Educ. 2023 Mar 29;23(1):193. doi: 10.1186/s12909-023-04172-w.
The Progress Test Medizin (PTM) is a 200-question formative test that is administered to approximately 11,000 students at medical universities (Germany, Austria, Switzerland) each term. Students receive feedback on their knowledge (development) mostly in comparison to their own cohort. In this study, we use the data of the PTM to find groups with similar response patterns.
We performed k-means clustering with a dataset of 5,444 students, selected cluster number k = 5, and answers as features. Subsequently, the data was passed to XGBoost with the cluster assignment as target enabling the identification of cluster-relevant questions for each cluster with SHAP. Clusters were examined by total scores, response patterns, and confidence level. Relevant questions were evaluated for difficulty index, discriminatory index, and competence levels.
Three of the five clusters can be seen as "performance" clusters: cluster 0 (n = 761) consisted predominantly of students close to graduation. Relevant questions tend to be difficult, but students answered confidently and correctly. Students in cluster 1 (n = 1,357) were advanced, cluster 3 (n = 1,453) consisted mainly of beginners. Relevant questions for these clusters were rather easy. The number of guessed answers increased. There were two "drop-out" clusters: students in cluster 2 (n = 384) dropped out of the test about halfway through after initially performing well; cluster 4 (n = 1,489) included students from the first semesters as well as "non-serious" students both with mostly incorrect guesses or no answers.
Clusters placed performance in the context of participating universities. Relevant questions served as good cluster separators and further supported our "performance" cluster groupings.
Progress Test Medizin (PTM) 是一项 200 题的形成性测试，每学期约有 11000 名医科大学生（德国、奥地利、瑞士）参加。学生主要通过与自己的同学进行比较来获得知识（发展）的反馈。在这项研究中，我们使用 PTM 的数据来找到具有相似反应模式的群体。
我们使用了一个包含 5444 名学生的数据进行 k-均值聚类，选择聚类数量 k=5，并将答案作为特征。随后，我们将数据传递给 XGBoost，以簇分配作为目标，使用 SHAP 识别每个簇的相关问题。我们通过总分、反应模式和置信水平来检查簇。对相关问题进行了难度指数、区分指数和能力水平的评估。
五个簇中有三个可以看作是“表现”簇：第 0 簇（n=761）主要由接近毕业的学生组成。相关问题往往比较难，但学生回答得自信且正确。第 1 簇（n=1357）的学生较为先进，第 3 簇（n=1453）主要由初学者组成。这些簇的相关问题相对简单，猜测的答案数量增加。有两个“辍学”簇：第 2 簇（n=384）的学生在最初表现良好后中途退出测试；第 4 簇（n=1489）包括来自第一学期的学生和“不认真”的学生，他们的答案大多是错误的或没有答案。
簇将表现置于参与大学的背景下。相关问题是很好的簇分隔符，并进一步支持了我们的“表现”簇分组。
