Multi-modal deep learning based on multi-dimensional and multi-level temporal data can enhance the prognostic prediction for multi-drug resistant pulmonary tuberculosis patients.

Despite the advent of new diagnostics, drugs and regimens, multi-drug resistant pulmonary tuberculosis (MDR-PTB) remains a global health threat. It has a long treatment cycle, low cure rate and heavy disease burden. Factors such as demographics, disease characteristics, lung imaging, biomarkers, therapeutic schedule and adherence to medications are associated with MDR-PTB prognosis. However, thus far, the majority of existing studies have focused on predicting treatment outcomes through static single-scale or low dimensional information. Hence, multi-modal deep learning based on dynamic data for multiple dimensions can provide a deeper understanding of personalized treatment plans to aid in the clinical management of patients.