Project IMPACT: results from a pilot validity study of a new observational database.

OBJECTIVE

The objective of this study was to evaluate the accuracy of the information contained in the Project IMPACT database. Project IMPACT is a comprehensive database system developed to measure and describe the care of intensive care patients. This database is being used by a large group of hospitals to help clinicians improve the care of these patients. Data on patient demographics, diagnoses, treatment, and outcomes are entered into the Project IMPACT database by staff at participating hospitals. This pilot study was a first step in assessing the accuracy of these data to determine the usefulness of the Project IMPACT database for measuring intensive care unit (ICU) performance and patient outcomes.

DESIGN

The design of the pilot study was the independent abstraction of selected data items from a random sample of ICU patient records from two hospitals participating in Project IMPACT. The abstracted data were compared with the data existing in the Project IMPACT database for agreement.

SETTING

Abstraction was performed onsite at the two pilot hospitals by a trained abstractor who was not affiliated with either hospital.

PATIENTS

Patients whose records were abstracted included 45 randomly selected ICU patients at each of the two pilot hospitals.

MEASUREMENTS AND MAIN RESULTS

Comparison of the Project IMPACT data with the independently abstracted data indicated good agreement (80% or above) on discrete items, such as type of ICU patient. Poorer agreement (under 80%) was seen for continuous items (e.g., 24-hr urine output) and coded items requiring judgment (e.g., reason for ICU admission).

CONCLUSIONS

The pilot study showed good internal validity for most of the abstracted variables. High agreement rates were observed, regardless of method of original data capture (electronic download or manual entry), although agreement was higher for some data items that had been electronically downloaded into the Project IMPACT database. The results suggest that Project IMPACT is a valuable resource for ICUs to collect and evaluate information about treatment and patient outcomes.