BACKGROUND

The identification of surgical site infections for infection surveillance in hospitals depends on the manual abstraction of medical records and, for research purposes, depends mainly on the use of administrative or claims data. The objective of this study was to determine whether automating the abstraction process with natural language processing (NLP)-based models that analyze the free-text notes of the medical record can identify surgical site infections with predictive abilities that match the manual abstraction process and that surpass surgical site infection identification from administrative data.

METHODS

We used surgical site infection surveillance data compiled by the infection prevention team to identify surgical site infections among patients undergoing orthopaedic surgical procedures at a tertiary care academic medical center from 2011 to 2017. We compiled a list of keywords suggestive of surgical site infections, and we used NLP to identify occurrences of these keywords and their grammatical variants in the free-text notes of the medical record. The key outcome was a binary indicator of whether a surgical site infection occurred. We estimated 7 incremental multivariable logistic regression models using a combination of administrative and NLP-derived variables. We split the analytic cohort into training (80%) and testing data sets (20%), and we used a tenfold cross-validation approach. The main analytic cohort included 172 surgical site infection cases and 200 controls that were repeatedly and randomly selected from a pool of 1,407 controls.

RESULTS

For Model 1 (variables from administrative data only), the sensitivity was 68% and the positive predictive value was 70%; for Model 4 (with NLP 5-grams [distinct sequences of 5 contiguous words] from the medical record), the sensitivity was 97% and the positive predictive value was 97%; and for Model 7 (a combination of Models 1 and 4), the sensitivity was 97% and the positive predictive value was 97%. Thus, NLP-based models identified 97% of surgical site infections identified by manual abstraction with high precision and 43% more surgical site infections compared with models that used administrative data only.

CONCLUSIONS

Models that used NLP keywords achieved predictive abilities that were comparable with the manual abstraction process and were superior to models that used administrative data only. NLP has the potential to automate and aid accurate surgical site infection identification and, thus, play an important role in their prevention.

CLINICAL RELEVANCE

This study examines NLP's potential to automate the identification of surgical site infections. This automation can potentially aid the prevention and early identification of these surgical complications, thereby reducing their adverse clinical and economic impact.