Depression has become a major global public health problem. Many studies have shown the positive effects of physical exercise on depression. However, few studies have examined the relationship between frequency of leisure time physical exercise and depression without considering the time and intensity of exercise among middle-aged and elderly people of urban communities in northern China. We conducted a cross-sectional survey that included 1604 participants among urban residents aged 50 years or older in China to evaluate how the frequency of physical exercise was related to depression. Our study showed that the prevalence of depression in the urban community of Jinan is 16.52%. For physical exercise, the odds ratios (ORs) and 95% confidence intervals (CIs) for 12 times per week, 34 times per week and ≥5 times per week were 1.137 (0.661, 1.953), 0.516 (0.304, 0.875) and 0.548 (0.392, 0.768) respectively, with adjustment for age, gender, marital status, BMI, hypertension, previously diagnosed type 2 diabetes, triglyceride, total cholesterol, soy food intake, milk food intake, vegetable and fruit intake and meat intake. We concluded that physically exercising three times a week is associated with a low prevalence of depression.