Brain plasticity is an intrinsic characteristic of the nervous system that allows continuous remodeling of brain functions in pathophysiological conditions. Although normal aging is associated with morphological modifications and decline of cerebral functions, brain plasticity is at least partially preserved in elderly individuals. A growing body of evidence supports the notion that cognitive enrichment and aerobic training induce a dynamic reorganization of higher cerebral functions, thereby helping to maintain operational skills in the elderly and reducing the incidence of dementia. The stroke model clearly shows that spontaneous brain plasticity exists after a lesion, even in old patients, and that it can be modulated through external factors like rehabilitation and drugs. Whether drugs can be used with the aim of modulating the effects of physical training or cognitive stimulation in healthy aged people has not been addressed until now. The risk:benefit ratio will be the key question with regard to the ethical aspect of this challenge. We review in this article the main aspects of human brain plasticity as shown in patients with stroke, the drug modulation of brain plasticity and its consequences on recovery, and finally we address the question of the influence of aging on brain plasticity.