Hamamoto Filho Pedro Tadao, Hafner Maria de Lourdes Marmorato Botta, Ribeiro Zilda Maria Tosta, Lima Alba Regina de Abreu, Diehl Leandro Arthur, Costa Neide Tomimura, de Andrade Maria Cristina, Yarak Samira, Rehder Patrícia Moretti, Moriguti Júlio César, Bicudo Angélica Maria
UNESP, Universidade Estadual Paulista, Botucatu, São Paul, Brazil.
FAMEMA, Faculdade de Medicina de Marília, Marília, São Paul, Brazil.
PLoS One. 2025 Jun 9;20(6):e0325734. doi: 10.1371/journal.pone.0325734. eCollection 2025.
It has been proposed that the school origin of items for cross-institutional Progress Tests (PTs) may introduce a bias in favour of students from the same school, posing a potential threat to the validity and reliability of PT results and cross-institutional comparisons. The aim of this study was to examine whether origin bias is present in a Brazilian cross-institutional PT examination.
This study conducted a cross-sectional analysis of seven schools affiliated with the oldest PT consortium in Brazil, utilising a pooled analysis of differences in students' performance concerning self and non-self items. A proportional meta-analysis of the items' rate differences and confidence intervals with random effects was performed, providing an odds ratio (OR) for self and non-self items. Differences between the two groups of items were assessed by scrutinising whether the OR and 95% confidence intervals overlapped.
The findings indicated no discernible differences in psychometric indices based on the school responsible for item creation. Three schools consistently demonstrate superior performance on items authored by their faculty, however, these they also excelled on non-self items. Furthermore, an overlap in the 95% confidence intervals for both self and non-self items was observed across all seven schools.
In contrast to prior reports, this study revealed the absence of origin bias, suggesting that adoption of best practices in blueprinting, item writing, and editing may have played a role in mitigating such bias.
有人提出，跨机构进展测试（PTs）项目的学校来源可能会导致有利于来自同一学校学生的偏差，这对PT结果的有效性和可靠性以及跨机构比较构成潜在威胁。本研究的目的是检验巴西跨机构PT考试中是否存在来源偏差。
本研究对巴西最古老的PT联盟所属的七所学校进行了横断面分析，利用对学生在自我和非自我项目上表现差异的汇总分析。对项目的率差和随机效应的置信区间进行了比例荟萃分析，得出了自我和非自我项目的优势比（OR）。通过检查OR和95%置信区间是否重叠来评估两组项目之间的差异。
研究结果表明，基于负责项目创建的学校，心理测量指标没有明显差异。然而，有三所学校在其教师编写的项目上始终表现出色，而且他们在非自我项目上也表现出色。此外，在所有七所学校中，自我和非自我项目的95%置信区间都有重叠。
与之前的报告相反，本研究表明不存在来源偏差，这表明在蓝图设计、项目编写和编辑中采用最佳实践可能在减轻此类偏差方面发挥了作用。
