This review discusses ex vivo diffusion magnetic resonance imaging (dMRI) as an important research tool for neuroanatomical investigations and the validation of in vivo dMRI techniques, with a focus on the human brain. We review the challenges posed by the properties of post-mortem tissue, and discuss state-of-the-art tissue preparation methods and recent advances in pulse sequences and acquisition techniques to tackle these. We then review recent ex vivo dMRI studies of the human brain, highlighting the validation of white matter orientation estimates and the atlasing and mapping of large subcortical structures. We also give particular emphasis to the delineation of layered gray matter structure with ex vivo dMRI, as this application illustrates the strength of its mesoscale resolution over large fields of view. We end with a discussion and outlook on future and potential directions of the field.