Age-related cognitive decline is one of the major aspects that impede successful aging in humans. Environmental factors, such as chronic stress, can accelerate or aggravate cognitive deficits during aging. While there is abundant evidence that chronic stress directly affects cognitive performance, the lasting consequences of stress exposures during vulnerable developmental time windows are largely unknown. This is especially true for the adolescent period, which is critical in terms of physical, sexual, and behavioral maturation. Here we used chronic social stress during adolescence in male mice and investigated the consequences of this treatment on cognitive performance during aging. We observed a substantial impairment of spatial memory, but not other memory domains, 12 months after the end of the stress period. This hippocampus-dependent cognitive dysfunction was supported by concomitant impairment in LTP induction in CA1 neurons in 15-month-old animals. Further, we observed a decrease of hippocampal BDNF mRNA and synaptophysin immunoreactivity, suggesting plasticity and structural alterations in formerly stressed mice. Finally, we identified expression changes of specific neurotransmitter subunits critically involved in learning and memory, specifically the NMDA receptor subunit NR2B. Taken together, our results identify possible molecular mechanisms underlying cognitive impairment during aging, demonstrating the detrimental impact of stress during adolescence on hippocampus-dependent cognitive function in aged mice.