Decades of detailed anatomical tracer studies in non-human animals point to a rich and complex organization of long-range white matter connections in the brain. State-of-the art in vivo imaging techniques are striving to achieve a similar level of detail in humans, but multiple technical factors can limit their sensitivity and fidelity. In this review, we mostly focus on magnetic resonance imaging of the brain. We highlight some of the key challenges in analyzing and interpreting in vivo connectomics data, particularly in relation to what is known from classical neuroanatomy in laboratory animals. We further illustrate that, despite the challenges, in vivo imaging methods can be very powerful and provide information on connections that is not available by any other means.