Agreement or association: choosing a measure of reliability for nominal data in the 2 x 2 case--a comparison of phi, kappa, and G.

This research note compares the phi, kappa, and G measures of association and agreement used to estimate the reliability of drug and alcohol self-report data in the 2 x 2 case. Attention is focused upon the problems encountered in using these measures when skewed marginal distributions are found in a test-retest situation. The G index of agreement was found to be a stable estimator, unaffected by skewed marginals, and equivalent to kappa and phi when marginal distributions are equal.