Development and Validation of VTE-BERT Natural Language Processing Model for Venous Thromboembolism.

BACKGROUND

Accurate and rapid phenotyping of venous thromboembolism (VTE) in longitudinal studies is important. A natural language processing (NLP) tool externally validated in representative patients is lacking.

METHODS

We designed a novel NLP platform, NLPMed, to assist thrombosis researchers with data preprocessing, phenotype annotation, language model finetuning, and NLP application. Utilizing clinical notes, discharge summaries, and radiology reports in patients with cancer from two healthcare institutions, we finetuned Bio_ClinicalBERT to develop VTE-BERT. The new model was trained to detect acute VTE events and their anatomical locations longitudinally. We internally and externally validated the model's performance in two randomly sampled cohorts of patients with advanced cancer.

RESULTS

The training cohort consisted of 715 patients and 14,013 annotated notes with ≥1 VTE keyword from the Harris Health System (HHS). The internal validation cohort included 400 additional patients with 7,190 VTE keyword-containing notes from HHS. The external validation cohort included 400 patients with 7,371 VTE keyword-containing notes from the National Veterans Affairs Healthcare System. VTE-BERT was trained until it reached a precision of 95% and recall of 98% on the patient level. Using independent datasets, the model achieved precision and recall of 95% and 91% in internal validation and of 85% and 92% in external validation.

CONCLUSIONS

We trained and externally validated an efficient NLP model to detect incident VTE events longitudinally. We believe its adoption will accelerate thrombosis research by improving VTE detection at scale and decreasing the time and expense involved with manual chart review in big data epidemiological studies.