Aging is typically marked by a decline in some domains of cognition. Some theories have linked this decline to a reduction in distinctiveness of processing at the neural level that in turn leads to cognitive decline. Increasing correlations with age among tasks formerly considered independent have been posited, supporting dedifferentiation, although results have been mixed. An alternative view is that tasks become more, and not less, independent of one another with increasing age, suggesting age-related differentiation, or what has also been termed disintegration. In the current study, we investigated if the aging process leads to a loss of behavioral and neural specificity within latent cognitive abilities. To this end, we tested 287 participants (20-80 years) on a battery of 12 in-scanner tests, three each tapping one of four reference abilities. We performed between-task correlations within domain (pertaining to convergent validity), and between domain (pertaining to discriminant validity) at both the behavioral and neural level and found that neural convergent validity was positively associated with behavioral convergent validity. In examining neural validity across the lifespan, we found significant reductions in both within- and between-domain task correlations, with a significant decrease in construct validity (convergent or discriminant) with age. Furthermore, the effect of age on total cognition was significantly mediated by neural construct validity. Taken together, contrary to a hypothesis of dedifferentiation, these correlation reductions suggest that tasks indeed become more independent with advancing age, favoring a differentiation/disintegration hypothesis of aging.