Olfactory impairment is a potential marker for prodromal dementia, but the underlying mechanisms are poorly understood. This population-based study included 4214 dementia-free participants (age ≥65 years). Olfaction was assessed using the 16-item Sniffin' Sticks identification test. In the subsamples, we measured plasma amyloid beta (Aβ)40, Aβ42, total tau, and neurofilament light chain (NfL; n = 1054); and quantified hippocampal, entorhinal cortex, and white matter hyperintensity (WMH) volumes, and Alzheimer's disease (AD)-signature cortical thickness (n = 917). Data were analyzed with logistic and linear regression models. In the total sample, mild cognitive impairment (MCI) was diagnosed in 1102 persons (26.2%; amnestic MCI, n = 931; non-amnestic MCI, n = 171). Olfactory impairment was significantly associated with increased likelihoods of MCI, amnestic MCI, and non-amnestic MCI. In the subsamples, anosmia was significantly associated with higher plasma total tau and NfL concentrations, smaller hippocampal and entorhinal cortex volumes, and greater WMH volume, and marginally with lower AD-signature cortical thickness. These results suggest that cerebral neurodegenerative and microvascular lesions are common neuropathologies linking anosmia with MCI in older adults.