Hasbahçeci Mustafa, Başak Fatih, Acar Aylin, Şişik Abdullah
Department of General Surgery, Bezmialem Vakıf University School of Medicine, İstanbul, Turkey.
Clinic of General Surgery, Ümraniye Training and Research Hospital, İstanbul, Turkey.
Ulus Cerrahi Derg. 2016 Dec 1;32(4):267-274. doi: 10.5152/UCD.2016.3195. eCollection 2016.
To compare the quality of oral presentations presented at the 19 National Surgical Congress with a national evaluation system with respect to the applicability of systems, and consistency between systems and reviewers.
Fifty randomly selected observational studies, which were blinded for author and institute information, were evaluated by using the Strengthening the Reporting of Observational Studies (STROBE), Timmer Score, and National Evaluation System by two reviewers. Abstract scores, evaluation periods, and compatibility between reviewers were compared for each evaluation system. Abstract scores by three different evaluation systems were regarded as the main outcome. Wilcoxon matched-pairs signed rank and Friedman tests for comparison of scores and times, kappa analysis for compatibility between reviewers, and Spearman correlation for analysis of reviewers based on pairs of evaluation systems were used.
There was no significant difference between abstract scores for each system (p>0.05). A significant difference for evaluation period of reviewers was detected for each system (p<0.05). Compatibility between reviewers was the highest for the Timmer Score (medium, κ=0.523), and the compatibility for STROBE and National Evaluation System was regarded as acceptable (κ=0.394 and κ=0.354, respectively). Assessment of reviewers for pairs of evaluation systems revealed that scores increased in the same direction with each other significantly (p<0.05).
The National Evaluation System is an appropriate method for evaluation of conference abstracts due to the consistent results between the referees similarly with the current international evaluation systems and ease of applicability with regard to evaluation period.
运用国家评估系统，就系统适用性以及系统与评审员之间的一致性，比较在第19届全国外科学术大会上发表的口头报告的质量。
随机选取50项观察性研究，对作者和机构信息进行盲法处理，由两名评审员使用加强观察性研究报告标准（STROBE）、蒂默评分和国家评估系统进行评估。比较各评估系统的摘要分数、评估周期以及评审员之间的兼容性。将三种不同评估系统的摘要分数视为主要结果。使用威尔科克森配对符号秩检验和弗里德曼检验比较分数和时间，使用kappa分析评估评审员之间的兼容性，使用斯皮尔曼相关性分析基于成对评估系统的评审员。
各系统的摘要分数之间无显著差异（p>0.05）。各系统均检测到评审员评估周期存在显著差异（p<0.05）。蒂默评分的评审员之间兼容性最高（中等，κ=0.523），STROBE和国家评估系统的兼容性被认为是可接受的（分别为κ=0.394和κ=0.354）。对成对评估系统的评审员评估显示，分数彼此显著同向增加（p<0.05）。
国家评估系统是评估会议摘要的一种合适方法，因为评审员之间的结果与当前国际评估系统一致，且在评估周期方面易于应用。
