How robust are health plan quality indicators to data loss? A Monte Carlo simulation study of pediatric asthma treatment.

OBJECTIVES

(1) To test the robustness of a health plan quality indicator (QI) for persistent asthma to various forms of data loss and (2) to assess the implications of the findings for other health plan quality measures.

DATA SOURCES/STUDY SETTINGS: Maryland Medicaid fee-for-service (FFS) claims. Children with asthma (n = 5,804) were selected from Medicaid enrollment records and medical and pharmacy FFS claims filed between June 1996 and December 1997.

STUDY DESIGN

A variant of a HEDIS measure for treatment of persistent asthma (the percent of asthma patients filling two or more rescue medications who also filled a controller medication) was selected to test the robustness of proportion-based QIs to loss of data. Data loss was simulated through a series of Monte Carlo experiments.

DATA COLLECTION/EXTRACTION METHODS: Merged FFS medical and prescription claims.

PRINCIPAL FINDINGS

The asthma QI measure was highly robust to systematic and random data loss. The measure declined by less than 2 percent in the presence of up to a 35 percent data loss. Redundancy in the numerator of the QI significantly increased the robustness of the measure to data loss.

CONCLUSIONS

A HEDIS-related QI measure for persistent asthma is robust to data loss. The findings suggest that other proportion-based quality indicators, particularly those in which plan members have multiple opportunities to meet the numerator criterion, are likely to reflect true levels of health plan quality in the face of incomplete data capture.