If a map-like representation in the brain originates within a distributed network of brain regions, then this network should be underpinned by white-matter pathways. To test this, we had 83 young adults learn 15 virtual environments through free exploration, followed by tests of their allocentric and non-allocentric representation of these environments. Diffusion magnetic resonance imaging (dMRI) - based tractography was used to extract the main white matter tracts in the brain, including the inferior and middle longitudinal fasciculus, the main connections between the temporal lobe and the visual cortices, as well as the fornix, the main output pathway from the hippocampus to the cortex. Higher microstructural integrity in the inferior and middle longitudinal fasciculus and fornix, as indexed by higher FA, was associated with a more accurate scale, rotation, and translation-invariant allocentric representation of the objects' positional pattern. These findings show that properties of white matter fiber pathways that link visual cortices to the temporal lobe and the temporal lobe to other parts of the brain underpin a person's ability to form successful map-like representations.