Several brain magnetic resonance imaging (MRI) changes are observed in older individuals including white matter lesions (WML), silent brain infarcts (SBI), and cerebral atrophy. Few studies, however, have assessed the combined association of these changes on the severity of future cognitive decline. In the prospective population-based 3C-Dijon MRI study, 1701 non-demented participants aged 65 to 80 years at entry had a brain MRI. Information on WML, hippocampal volumes, SBI presence, and brain parenchymal fraction were obtained. At 4-year follow-up, participants were screened for cognitive decline and dementia. Severity of cognitive decline was defined as none, moderate, or severe calculated from neuropsychological test performance change. The relation between brain MRI markers and longitudinal change in cognition was studied using polytomous logistic regression and multiple linear regression models controlling for potential confounders. Two-by-two interactions were tested including with the apolipoprotein E genotype. At follow-up, 46 participants showed severe cognitive deterioration and 224 participants showed moderate cognitive deterioration. In multivariable analyses, risk of severe cognitive deterioration as well as the cognitive decline rate were significantly increased in participants with higher WML volume (p< 0.01) and smaller hippocampal volume (p< 0.01). The results suggested that WML and hippocampal volumes had a cumulative effect on the future level of cognitive decline. The APOE genotype was found to be an effect modifier of this association. Vascular brain changes and degenerative processes coexist in normal older individuals. The co-occurrence of degenerative and non-degenerative pathologies could strongly affect the course of dementia expression.