Reliability of assessment of adherence to an antimicrobial treatment guideline.

Assessment procedures for adherence to a guideline must be reliable and credible. The aim of this study was to explore the reliability of assessment of adherence, taking account of the professional backgrounds of the observers. A secondary analysis explored the impact of case characteristics on assessment. Six observers (two hospital pharmacists, two internists and two clinical microbiologists) assessed a random sample of 22 prescriptions made to infectious disease cases admitted to a department of internal medicine between February and August 2001. Agreement between observers with regard to adherence of these prescriptions to guideline recommendations concerning drug choice, duration of treatment, dosage and route of administration was measured using Cohen's kappa. Case characteristics were compared between cases where observers agreed and disagreed with two-sided Fisher's exact test. Agreement between all professionals was moderate for drug choice (0.59), fair for duration of therapy (0.36), moderate for dosage (0.48), and fair for route of administration (0.37). Agreement on drug choice was good within (0.75 and 0.83) and between (0.74) the internists and the hospital pharmacists, but was less within (0.31) the clinical microbiologists and between the clinical microbiologists and the internists (0.44) and the hospital pharmacists (0.42). Within the clinical microbiologists, agreement was good for dosage (0.79) and route of administration (0.66). There was frequent disagreement between observers regarding cases with combination therapy and non-immunocompromised patients. Despite the small number of cases, our results suggest that internists and hospital pharmacists can reliably be used to assess adherence for drug choice. The level of agreement seems to be affected by combination therapy and the immune status of the patient.