Hip and knee osteoarthritis (HKOA) is a chronic disease characterized by joint pain that leads to reduced physical function and health-related quality of life (HRQoL). At present, no cure is available. Clinical trials indicate that people with HKOA benefit from physical activity in several health-related outcomes. However, few studies have evaluated the long-term positive effect of regular physical activity. This study analyzed participants with HKOA from a nationwide population-based cohort (EpiDoC Cohort) to assess the impact of physical activity on patients' physical function and HRQoL over a long-term follow-up. The regular weekly frequency of intentional physical activity was self-reported as non-frequent (0 times/week), frequent (1-2 times/week), or very frequent (≥ 3 times/week). This study followed 1086 participants over a mean period of 4.7 ± 3.4 years, during which 6.3% and 14.9% of participants reported frequent and very frequent physical activity, respectively. Using linear mixed models, we found that frequent (β =  - 0.101 [- 0.187, - 0.016]; β = 0.039 [- 0.002, 0.080]) and very frequent physical activity (β =  - 0.061 [- 0.118, - 0.004]; β = 0.057 [0.029, 0.084]) were associated with improved physical function and HRQoL over time, respectively, when compared with non-frequent exercise, adjusting for years to baseline, sex, age, years of education, body mass index, multimorbidity, hospitalizations, clinical severity, and unmanageable pain levels. These findings raise awareness of the importance of maintaining exercise/physical activity long term to optimize HRQoL and physical function. Further studies must address barriers and facilitators to improve the adoption of regular physical activity among citizens with HKOA.