Estimation of Risk-Adjusted Outcomes for Non-Infectious Postoperative Complications using Interpretable Machine Learning and Electronic Health Record Data.

OBJECTIVE

To compare statistical models applied to electronic health record (EHR) data to predict and identify non-infectious postoperative complications. The models have been published and are part of the Automated Surveillance of Postoperative Infections (ASPIN) project, which has expanded to include non-infectious complications.

SUMMARY OF BACKGROUND DATA

Postoperative complications occur in 15% of nonemergent inpatient surgeries. Most reporting of postoperative complications relies on manual chart abstraction.

METHODS

Preoperative and postoperative probabilities of non-infectious complications for patients from 5 large hospitals in Colorado were estimated using ASPIN models that were developed using the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) gold standard outcomes. Observed:expected (O:E) ratios were estimated by dividing the sum of the postoperative probabilities by the sum of the preoperative probabilities. O:E ratios were compared between local ACS-NSQIP patients using ACS-NSQIP data, local ACS-NSQIP patients using EHR data, and all patients undergoing operations in the study period using EHR data.

RESULTS

O:E ratios for 9 non-infectious postoperative complications were estimated. Comparison of the O:E ratios of ACS-NSQIP patients using ACS-NSQIP data vs. EHR data showed overlapping confidence intervals in 44 (98%) of 45 comparisons (5 hospitals x 9 outcomes) and agreement in outlier status for 35 (78%).

CONCLUSIONS

Risk-adjusted postoperative outcomes estimated using machine learning on EHR data were similar to those produced by manual chart review. These models could be used to augment manual chart review to guide surgical quality improvement.