Paci Matteo, Bianchini Claudio, Baccini Marco
Unit of Functional Recovery, Azienda USL Toscana Centro, Presidio Piero Palagi, Viale Michelangiolo, 41, 50134, Florence, Italy.
Private Practice, Florence, Italy.
Arch Physiother. 2022 Mar 31;12(1):10. doi: 10.1186/s40945-022-00133-6.
Lack of effective peer-review process of predatory journals, resulting in more ambiguity in reporting, language and incomplete descriptions of processes might have an impact on the reliability of PEDro scale. The aim of this investigation was to compare the reliability of the PEDro scale when evaluating the methodological quality of RCTs published in predatory (PJs) and non-predatory (NPJs) journals, to more confidently select interventions appropriate for application to practice.
A selected sample of RCTs was independently rated by two raters randomly selected among 11 physical therapists. Reliability of each item of the PEDro scale and the total PEDro score were assessed by Cohen's kappa statistic and percent of agreement and by Intraclass Correlation Coefficients (ICC) and the Standard Error of Measurement (SEM), respectively. The Chi-square test was used to compare the rate of agreement between PJs and NPJs.
A total number of 298 RCTs were assessed (119 published in NPJs). Cronbach's alphas were .704 and .845 for trials published in PJs and NPJs, respectively. Kappa values for individual scale items ranged from .14 to .73 for PJs and from .09 to .70 for NPJs. The ICC was .537 (95% CI .425-.634) and .729 (95% CI .632-.803), and SEM was 1.055 and 0.957 for PJs and NPJs, respectively. Inter-rater reliability in discriminating between studies of moderate to high and low quality was higher for NPJs (k = .57) than for PJs (k = .28).
Interrater reliability of PEDro score of RCTs published in PJs is lower than that of trials published in NPJs, likely also due to ambiguous language and incomplete reporting. This might make the detection of risk of bias more difficult when selecting interventions appropriate for application to practice or producing secondary literature.
掠夺性期刊缺乏有效的同行评审流程，导致报告、语言和过程描述方面存在更多模糊性，这可能会影响PEDro量表的可靠性。本研究的目的是比较PEDro量表在评估发表于掠夺性（PJs）和非掠夺性（NPJs）期刊上的随机对照试验（RCT）方法学质量时的可靠性，以便更有信心地选择适用于实践的干预措施。
从11名物理治疗师中随机选取两名评分者，对选定的RCT样本进行独立评分。PEDro量表各项目的可靠性和总PEDro得分分别通过Cohen's kappa统计量和一致百分比以及组内相关系数（ICC）和测量标准误（SEM）进行评估。采用卡方检验比较PJs和NPJs之间的一致率。
共评估了298项RCT（119项发表于NPJs）。发表于PJs和NPJs的试验的Cronbach's alphas分别为0.704和0.845。PJs的单个量表项目的Kappa值范围为0.14至0.73，NPJs的范围为0.09至0.70。PJs和NPJs的ICC分别为0.537（95%CI 0.425 - 0.634）和0.729（95%CI 0.632 - 0.803），SEM分别为1.055和0.957。在区分中高质量和低质量研究方面，NPJs的评分者间信度（k = 0.57）高于PJs（k = 0.28）。
发表于PJs的RCT的PEDro评分的评分者间信度低于发表于NPJs的试验，这可能也是由于语言模糊和报告不完整所致。在选择适用于实践的干预措施或生成二次文献时，这可能会使偏倚风险的检测更加困难。
