In drug-resistant temporal lobe epilepsy, precise predictions of drug response, surgical outcome and cognitive dysfunction at an individual level remain challenging. A possible explanation may lie in the dominant 'one-size-fits-all' group-level analytical approaches that does not allow parsing interindividual variations along the disease spectrum. Conversely, analysing inter-patient heterogeneity is increasingly recognized as a step towards person-centred care. Here, we used unsupervised machine learning to estimate latent relations (or disease factors) from 3 T multimodal MRI features [cortical thickness, hippocampal volume, fluid-attenuated inversion recovery (FLAIR), T1/FLAIR, diffusion parameters] representing whole-brain patterns of structural pathology in 82 patients with temporal lobe epilepsy. We assessed the specificity of our approach against age- and sex-matched healthy individuals and a cohort of frontal lobe epilepsy patients with histologically verified focal cortical dysplasia. We identified four latent disease factors variably co-expressed within each patient and characterized by ipsilateral hippocampal microstructural alterations, loss of myelin and atrophy (Factor 1), bilateral paralimbic and hippocampal gliosis (Factor 2), bilateral neocortical atrophy (Factor 3) and bilateral white matter microstructural alterations (Factor 4). Bootstrap analysis and parameter variations supported high stability and robustness of these factors. Moreover, they were not expressed in healthy controls and only negligibly in disease controls, supporting specificity. Supervised classifiers trained on latent disease factors could predict patient-specific drug response in 76 ± 3% and postsurgical seizure outcome in 88 ± 2%, outperforming classifiers that did not operate on latent factor information. Latent factor models predicted inter-patient variability in cognitive dysfunction (verbal IQ: r = 0.40 ± 0.03; memory: r = 0.35 ± 0.03; sequential motor tapping: r = 0.36 ± 0.04), again outperforming baseline learners. Data-driven analysis of disease factors provides a novel appraisal of the continuum of interindividual variability, which is probably determined by multiple interacting pathological processes. Incorporating interindividual variability is likely to improve clinical prognostics.