Wirtz M, Kutschmann M
Institut für Psychologie, Pädagogische Hochschule Freiburg, 79117 Freiburg.
Rehabilitation (Stuttg). 2007 Dec;46(6):370-7. doi: 10.1055/s-2007-976535.
Within rehabilitation research ratings are one of the most frequently used assessment procedures. For example, therapists frequently make categorical judgements aiming to get information whether specific patient characteristics prevail or not (dichotomous rating format) or which of several alternatives holds for a patient (polytomous rating format). Interrater agreement is an important prerequisite to ensure that reliable and meaningful information concerning patients' state can be inferred from the data obtained. Cohen's kappa (Cohen's kappa) is the most frequently used measure to quantify interrater agreement. The properties of Cohen's kappa are characterized and conditions for the appropriate application of kappa are clarified. Because sometimes specific properties of kappa are not appropriately considered, misleading interpretations of this measure may easily arise. This is the case because the value of Cohen's kappa is affected by information aspects that are independent of the quality of the rating process. In order to avoid such misconceptions, alternative evaluation strategies are described for dichotomous rating formats which enhance agreement analysis and thus ensure a more valid interpretation. In addition, it is shown how weighted Cohen's kappa (omega) may be used to analyze polytomous rating formats.
在康复研究中，评级是最常用的评估程序之一。例如，治疗师经常进行分类判断，旨在获取特定患者特征是否占主导地位的信息（二分评级格式），或者几种选择中的哪一种适用于某个患者（多分评级格式）。评分者间信度是确保能够从所获得的数据中推断出有关患者状况的可靠且有意义信息的重要前提。科恩卡方（Cohen's kappa）是最常用于量化评分者间信度的指标。本文阐述了科恩卡方的特性，并阐明了卡方恰当应用的条件。由于有时卡方的特定属性未得到恰当考虑，可能很容易对该指标产生误导性解释。出现这种情况是因为科恩卡方的值受到与评级过程质量无关的信息因素影响。为避免此类误解，针对二分评级格式描述了替代评估策略，这些策略可加强一致性分析，从而确保更有效的解释。此外，还展示了如何使用加权科恩卡方（omega）来分析多分评级格式。
