Previous structural and functional neuroimaging studies have implicated distributed brain regions and networks in depression. However, there are no robust imaging biomarkers that are specific to depression, which may be due to clinical heterogeneity and neurobiological complexity. A dimensional approach and fusion of imaging modalities may yield a more coherent view of the neuronal correlates of depression. We used linked independent component analysis to fuse cortical macrostructure (thickness, area, gray matter density), white matter diffusion properties and resting-state functional magnetic resonance imaging default mode network amplitude in patients with a history of depression (n = 170) and controls (n = 71). We used univariate and machine learning approaches to assess the relationship between age, sex, case-control status, and symptom loads for depression and anxiety with the resulting brain components. Univariate analyses revealed strong associations between age and sex with mainly global but also regional specific brain components, with varying degrees of multimodal involvement. In contrast, there were no significant associations with case-control status, nor symptom loads for depression and anxiety with the brain components, nor any interaction effects with age and sex. Machine learning revealed low model performance for classifying patients from controls and predicting symptom loads for depression and anxiety, but high age prediction accuracy. Multimodal fusion of brain imaging data alone may not be sufficient for dissecting the clinical and neurobiological heterogeneity of depression. Precise clinical stratification and methods for brain phenotyping at the individual level based on large training samples may be needed to parse the neuroanatomy of depression.