The impact of SARS-CoV-2 and the COVID-19 pandemic on brain health is recognised, yet specific effects remain understudied. We investigate the pandemic's impact on brain ageing using longitudinal neuroimaging data from the UK Biobank. Brain age prediction models are trained from hundreds of multi-modal imaging features using a cohort of 15,334 healthy participants. These models are then applied to an independent cohort of 996 healthy participants with two magnetic resonance imaging scans: either both collected before the pandemic (Control groups), or one before and one after the pandemic onset (Pandemic group). Our findings reveal that, even with initially matched brain age gaps (predicted brain age vs. chronological age) and matched for a range of health markers, the pandemic significantly accelerates brain ageing. The Pandemic group shows on average 5.5-month higher deviation of brain age gap at the second time point compared with controls. Accelerated brain ageing is more pronounced in males and those from deprived socio-demographic backgrounds and these deviations exist regardless of SARS-CoV-2 infection. However, accelerated brain ageing correlates with reduced cognitive performance only in COVID-infected participants. Our study highlights the pandemic's significant impact on brain health, beyond direct infection effects, emphasising the need to consider broader social and health inequalities.