The Coronavirus disease 2019 (COVID-19) pandemic seriously threatened the health and well-being of the population. This study aims to investigate the relevance of the COVID-19 pandemic on the stress, mental distress, and well-being of older people in Spain. The design was quantitative repeated cross-sectional. The sample was non-probability and consisted of 1436 persons from the general population divided into two groups: (1) the study group, composed of 718 women (61.3%) and men aged 60 to 89; (2) the comparison group, composed of the same number of women and men aged 30 to 45. All were assessed in three phases of the COVID-19 pandemic: the first pandemic year, the last pandemic year, and the post-pandemic period. The results showed that during the first year of the pandemic, the prevalence of mental distress was higher in older women (50%) than in older men (37.2%), while the rates in the comparison group were 57.2% for women and 53.2% for men. In the post-pandemic period, the rates of mental distress were 30.2% for older women and 29.8% for older men while in the comparison group, the rates were 48.5% for women and 26.5% for men. No significant differences in well-being were found between the groups or between the different phases of the pandemic. The most common stressors reported by older people were illness and death of family and/or loved ones, followed by personal illness. In the post-pandemic period, more stressful events and lower stress resilience were found to predict mental distress in older women and men. Greater perceived vulnerability to infection was another important predictor for women. Low self-esteem and younger age were also predictors of mental distress for men. High self-esteem, high social support, greater stress resilience and fewer stressful events were predictors of well-being for both genders. The results of this study are relevant for the design of policies, programs, and strategies to improve the health and well-being of older people.