Ninci Jennifer, Vannest Kimberly J, Willson Victor, Zhang Nan
Texas A&M University, College Station, USA
Texas A&M University, College Station, USA.
Behav Modif. 2015 Jul;39(4):510-41. doi: 10.1177/0145445515581327. Epub 2015 Apr 14.
Visual analysis is the most widely applied method of data interpretation for single-case research as it encompasses multifaceted considerations relevant to evaluating behavior change. However, a previous research synthesis found low levels of interrater agreement between visually analyzed ratings of graphed data across all variables under analysis. The purpose of this meta-analysis was to evaluate the peer-reviewed literature to date for potential moderators affecting the proportion of interrater agreement between visual analysts. Nineteen articles with 32 effects were assembled. Potential moderators evaluated included (a) design families, (b) rater expertise, (c) the provision of contextual information for graphs, (d) the use of visual aids, (e) the provision of an operational definition of the construct being rated, and (f) rating scale ranges. Results yielded an overall weighted interrater agreement proportion of .76. Moderator variables identified produced low to adequate levels of interrater agreement. Practical recommendations for future research are discussed.
视觉分析是单案例研究中应用最广泛的数据解释方法，因为它包含了与评估行为变化相关的多方面考量。然而，先前的一项研究综述发现，在对所有分析变量的图表数据进行视觉分析评分时，评分者间的一致性水平较低。本荟萃分析的目的是评估迄今为止的同行评审文献，以寻找影响视觉分析者之间评分者间一致性比例的潜在调节因素。我们收集了19篇包含32个效应的文章。评估的潜在调节因素包括：（a）设计类型，（b）评分者专业知识，（c）为图表提供背景信息，（d）使用视觉辅助工具，（e）为被评分的构念提供操作定义，以及（f）评分量表范围。结果得出评分者间总体加权一致性比例为0.76。所确定的调节变量产生了低到适度水平的评分者间一致性。本文还讨论了对未来研究的实际建议。
