Interpretable longitudinal glaucoma visual field estimation deep learning system from fundus images and clinical narratives.

Glaucoma is a globally prevalent disease that leads irreversible blindness. The visual field (VF) examination is important but time-consuming for visual function evaluation with high requirement of cooperation and reliability of patients. While color fundus photographs (CFPs) are easy to access. Here, we proposed a multi-modal longitudinal estimation deep learning (MLEDL) system, capable of predicting present and future VFs from CFPs and clinical text. This model was developed on 1598 records in cross-sectional and 3278 records in longitudinal dataset, with 446 external testing records. The pointwise mean absolute error across five models ranged from 3.098 to 4.131 dB. Heatmaps demonstrated the spatial relationship between fundus damage and vision loss. VF grading methods were employed for verifying the clinical reliability. Consequently, our MLEDL facilitates VF prediction by CFPs and clinical narratives, offering potential as function assessment tool over the long-duration course of glaucoma and thereby improving clinical practice efficiency.