Research Methods, Assessment and iScience, Department of Psychology, University of Konstanz, Konstanz, Germany.
PLoS One. 2024 Jul 25;19(7):e0307594. doi: 10.1371/journal.pone.0307594. eCollection 2024.
Teachers' judgment accuracy is a core competency in their daily business. Due to its importance, several meta-analyses have estimated how accurately teachers judge students' academic achievements by measuring teachers' judgment accuracy (i.e., the correlation between teachers' judgments of students' academic abilities and students' scores on achievement tests). In our study, we considered previous meta-analyses and updated these databases and the analytic combination of data using a psychometric meta-analysis to explain variations in results across studies. Our results demonstrate the importance of considering aggregation and publication bias as well as correcting for the most important artifacts (e.g., sampling and measurement error), but also that most studies fail to report the data needed for conducting a meta-analysis according to current best practices. We find that previous reviews have underestimated teachers' judgment accuracy and overestimated the variance in estimates of teachers' judgment accuracy across studies because at least 10% of this variance may be associated with common artifacts. We conclude that ignoring artifacts, as in classical meta-analysis, may lead one to erroneously conclude that moderator variables, instead of artifacts, explain any variation. We describe how online data repositories could improve the scientific process and the potential for using psychometric meta-analysis to synthesize results and assess replicability.
教师的判断准确性是其日常工作的核心能力。由于其重要性，已经有几项荟萃分析通过衡量教师对学生学业成绩的判断准确性（即教师对学生学术能力的判断与学生在成就测试中的得分之间的相关性）来估计教师对学生学业成绩的判断有多准确。在我们的研究中，我们考虑了以前的荟萃分析，并使用心理测量荟萃分析更新了这些数据库和数据分析的组合，以解释研究之间结果的差异。我们的结果表明，考虑聚合和出版偏差以及纠正最重要的人为因素（例如抽样和测量误差）非常重要，但也表明大多数研究未能按照当前最佳实践报告进行荟萃分析所需的数据。我们发现，以前的综述低估了教师的判断准确性，并高估了研究之间教师判断准确性估计的差异，因为至少 10%的这种差异可能与常见的人为因素有关。我们得出结论，像经典荟萃分析那样忽略人为因素可能会导致错误地认为是调节变量而不是人为因素解释了任何变化。我们描述了在线数据存储库如何改善科学过程，以及使用心理测量荟萃分析综合结果和评估可重复性的潜力。
