Why is evaluation of the cost effectiveness of audit so difficult? The example of thrombolysis for suspected acute myocardial infarction.

BACKGROUND

Cost effectiveness analysis is an established technique for evaluation of delivery of health care, but its use to evaluate clinical audit is rarely reported. Thrombolysis for suspected acute myocardial infarction is a commonly used therapy of established effectiveness and an appropriate subject for audit in many healthcare settings.

OBJECTIVE

To measure the cost effectiveness of audit of thrombolysis in some district general hospitals.

MAIN OUTCOME MEASURE

Cost of audit per extra patient treated with thrombolysis (incremental cost effectiveness ratio).

DESIGN

Prospective agreement with physicians to undertake repeated audits of a specific aspect of the management of patients with acute myocardial infarction. Baseline measurement of the proportion of these patients given thrombolysis in each hospital were made, as were three subsequent retrospective audits, giving time series of measurements. Costs were estimated from records of staff time and other resources used in each hospital; effectiveness was estimated by fitting the results to a model which assumed a uniform rate of increase over time in the proportion of eligible patients given thrombolysis which might be accelerated by regular audit. Upper and lower limits for main outcome measure were derived from sensitivity analysis of costs and logistic regression of time series data.

SETTING

Five district general hospitals in North West Thames Regional Health Authority including one control hospital were used, starting in April 1991 when widespread medical audit was first introduced.

RESULTS

Between the first and last audits, the proportion of patients with suspected acute myocardial infarctions receiving thrombolysis rose in three of the hospitals undertaking audit by 20% to 37% and fell by 6% in the fourth (although this hospital started with a rate in excess of 90%). The corresponding change in the control hospital was an increase of 15%. The differences between each of the auditing hospitals and control hospital were not significant, except in one case, where 51 extra treatments per year were attributable to audit (95% confidence intervals (95% CIs) 0.5 to 61 cases per year). Estimated overall costs in each hospital ranged from 3700 Pounds to 5200 Pounds for data collection, a series of four audit meetings, and subsequent actions. The central estimate of cost effectiveness in the three responsive hospitals ranged from 101 Pounds to 392 Pounds per extra case given thrombolysis, with very wide 95% CIs. In the fourth hospital audit had zero effectiveness as defined in this study.

CONCLUSIONS

Methodological difficulties were encountered which need to be considered in future economic evaluations of clinical audit and related activities. These were: (a) adequate control for other factors influencing clinical behaviour; (b) uncertainties about the sustainability of changes in behaviour associated with audit; and (c) the relative infrequency in a single hospital of specific clinical events leading to small numbers for analysis. These difficulties constitute major challenges for the economic evaluation of clinical audit. They are most likely to be overcome in a large study which compares clinical audit with other interventions aiming for the same quality improvement, such as patient specific reminders or educational programmes.