The aim of this study was to analyse the influence of the severity of depressive symptoms on different domains of cognitive function in the elderly. In a population-based cross-sectional study, 385 participants aged 65-83 years were interviewed with the Center for Epidemiologic Studies Depression Scale (CES-D) and performed a standardized neuropsychological test assessing attention, memory, cognitive speed and motor function. Multivariate linear regression analyses revealed a significant effect of depressive symptoms on a single test (Stroop test 1) and two summary scores (memory and motor function). After full adjustment for education and Mini Mental State Examination, the memory score was partly attenuated. Stratified analysis showed that an increase in CES-D scores led to a larger decline of cognitive test results in participants with mild to moderate depressive symptoms, compared to those with a high degree of depressive symptoms. Our results suggest that depressive mood in older adults is primarily associated with decreased processing speed and motor functioning, but not executive control functions. According to our results depressive mood is not necessarily associated with memory deficits in older adults. Changes in depressive symptoms in milder forms of depressive mood are associated with a larger decline in cognitive function than in severer forms of depressive mood.