Age effects on cognitive functioning are well-documented, but effects of sex on trajectories of cognitive aging are less clear. We examined cognitive ability across a variety of measures for 1,065 to 2,127 participants (mean baseline age 64.1 to 69.7 years) from the Baltimore Longitudinal Study of Aging who were repeatedly tested over a mean follow-up interval of 3.0 to 9.0 years with a mean of 2.3 to 4.4 assessments. Memory and other cognitive tests were administered at each visit, assessing mental status, verbal learning and memory, figural memory, language, attention, perceptuomotor speed and integration, executive function, and visuospatial ability. Importantly, participants free from cognitive impairment at all time points were used in the analyses. Results showed that for all tests, higher age at baseline was significantly associated with lower scores, and performance declined over time. In addition, advancing age was associated with accelerated longitudinal declines in performance (trend for mental status). After adjusting for age, education, and race, sex differences were observed across most tests of specific cognitive abilities examined. At baseline, males outperformed females on the 2 tasks of visuospatial ability, and females outperformed males in most other tests of cognition. Sex differences in cognitive change over time indicated steeper rates of decline for men on measures of mental status, perceptuomotor speed and integration, and visuospatial ability, but no measures on which women showed significantly steeper declines. Our results highlight greater resilience to age-related cognitive decline in older women compared with men.