The metabolic effects of stress are known to have significant health effects in both humans and animals. Most of these effects are mediated by the major stress hormonal axis in the body, the hypothalamic-pituitary-adrenal (HPA) axis. Within the central nervous system (CNS), the hippocampus, the amygdala and the prefrontal cortex as part of the limbic system are believed to play important roles in the regulation of the HPA axis. With the advent of structural and functional neuroimaging techniques, the role of different CNS structures in the regulation of the HPA axis can be investigated more directly. In the current paper, we summarize the findings obtained in our laboratory in the context of stress and HPA axis regulation. Our laboratory has developed and contributed to the development of manual and automated segmentation protocols from structural magnetic resonance imaging (MRI) scans for assessment of hippocampus, amygdala, medial temporal lobe and frontal lobe structures. Employing these protocols, we could show significant age-related changes in HC volumes, which were different between men and women, with pre-menopausal women showing smaller age-related volume decline compared to men. We could recently extent these findings by showing how estrogen therapy after menopause leads to higher volumes in the HC. Investigating possible neurotoxicity effects of steroids, we showed effects of long-term steroid exposure on HC volumes, and investigated variability of HC volumes in relation to HPA axis regulation in young and elderly populations. Here, we were able to follow-up from non-imaging studies showing that subjects low in self-esteem have higher cortisol stress responses, and the HC emerged as the critical link between these variables. Recently, we have made two more important discoveries with regard to HC volume: we could show that HC volume is as variable in young as it is in older adults, in subjects ranging in age from 18 to 80 years. Also, we have linked birth weight and maternal care to HC volumes in young adults, demonstrating the effects of variations in maternal care on the integrity of the CNS. Besides structural assessments, there is increasing interest in functional techniques to investigate possible links between CNS activity and HPA axis regulation. These two approaches complement each other; some aspects of HPA axis regulation might be linked to the integrity of a specific CNS structure, while other aspects might be linked to the function of a specific structure with no involvement of CNS morphology. Thus, we have developed a mental arithmetic stress task that can be employed in functional neuroimaging studies, and have used it in a number of functional neuroimaging studies. Employing positron emission tomography (PET), we were able to demonstrate that stress causes dopamine release if subjects reported low maternal care early in life. Finally, employing the task in functional magnetic resonance imaging (fMRI), we could show how exposure to stress and activation of the HPA axis are associated with decreased activity in major portions of the limbic system, a result that allows to speculate on the effects of stress on cognitive and emotional regulation in the brain. Taken together, the use of neuroimaging techniques in Psychoneuroendocrinology opens exciting new possibilities for the investigation of stress effects in the central nervous system.