A prospective analysis of inter-rater agreement between a physician and a physician's assistant in selecting QMR vocabulary terms.

We prospectively identified a cohort of 19 hospital inpatients to serve as a test set to determine the agreement between a physician and a physician's assistant in selecting and entering Quick Medical Reference (QMR) vocabulary terms for use in QMR case analysis mode. For positive findings, there was good agreement between data entered by the physician's assistant and data entered by the physician. Overall matches for positive findings were 0.66 concordance for the initial state, and 0.67 for the final state. Concordance for negative findings was substantially lower. The QMR generated differential diagnoses lists of the physician and physician's assistant were judged as similar in 14 (74%) of the 19 cases when QMR critique and assert functions were used.