Shea Beverley J, Hamel Candyce, Wells George A, Bouter Lex M, Kristjansson Elizabeth, Grimshaw Jeremy, Henry David A, Boers Maarten
Community Information and Epidemiological Technologies (CIET), Institute of Population Health, Ottawa, Ontario, Canada.
J Clin Epidemiol. 2009 Oct;62(10):1013-20. doi: 10.1016/j.jclinepi.2008.10.009. Epub 2009 Feb 20.
Our purpose was to measure the agreement, reliability, construct validity, and feasibility of a measurement tool to assess systematic reviews (AMSTAR).
We randomly selected 30 systematic reviews from a database. Each was assessed by two reviewers using: (1) the enhanced quality assessment questionnaire (Overview of Quality Assessment Questionnaire [OQAQ]); (2) Sacks' instrument; and (3) our newly developed measurement tool (AMSTAR). We report on reliability (interobserver kappas of the 11 AMSTAR items), intraclass correlation coefficients (ICCs) of the sum scores, construct validity (ICCs of the sum scores of AMSTAR compared with those of other instruments), and completion times.
The interrater agreement of the individual items of AMSTAR was substantial with a mean kappa of 0.70 (95% confidence interval [CI]: 0.57, 0.83) (range: 0.38-1.0). Kappas recorded for the other instruments were 0.63 (95% CI: 0.38, 0.78) for enhanced OQAQ and 0.40 (95% CI: 0.29, 0.50) for the Sacks' instrument. The ICC of the total score for AMSTAR was 0.84 (95% CI: 0.65, 0.92) compared with 0.91 (95% CI: 0.82, 0.96) for OQAQ and 0.86 (95% CI: 0.71, 0.94) for the Sacks' instrument. AMSTAR proved easy to apply, each review taking about 15 minutes to complete.
AMSTAR has good agreement, reliability, construct validity, and feasibility. These findings need confirmation by a broader range of assessors and a more diverse range of reviews.
我们的目的是评估一种用于评价系统评价的测量工具（AMSTAR）的一致性、可靠性、结构效度和可行性。
我们从一个数据库中随机选取了30篇系统评价。由两名评价者使用以下方法对每一篇进行评估：（1）增强质量评估问卷（质量评估问卷概述[OQAQ]）；（2）萨克斯工具；以及（3）我们新开发的测量工具（AMSTAR）。我们报告可靠性（AMSTAR 11个条目的观察者间kappa值）、总分的组内相关系数（ICC）、结构效度（将AMSTAR总分的ICC与其他工具的总分ICC进行比较）以及完成时间。
AMSTAR各条目之间的评价者间一致性较高，平均kappa值为0.70（95%置信区间[CI]：0.57，0.83）（范围：0.38 - 1.0）。其他工具的kappa值分别为：增强OQAQ为0.63（95%CI：0.38，0.78），萨克斯工具为0.40（95%CI：0.29，0.50）。AMSTAR总分的ICC为0.84（95%CI：0.65，0.92），而OQAQ为0.91（95%CI：0.82，0.96），萨克斯工具为0.86（95%CI：0.71，0.94）。AMSTAR证明易于应用，每篇评价大约需要15分钟完成。
AMSTAR具有良好的一致性、可靠性、结构效度和可行性。这些发现需要更多评价者和更多样化的评价进行验证。
