Our capacity to re-experience the past and simulate the future is thought to depend heavily on visual imagery, which allows us to construct complex sensory representations in the absence of sensory stimulation. There are large individual differences in visual imagery ability, but their impact on autobiographical memory and future prospection remains poorly understood. Research in this field assumes the normative use of visual imagery as a cognitive tool to simulate the past and future, however some individuals lack the ability to visualise altogether (a condition termed "aphantasia"). Aphantasia represents a rare and naturally occurring knock-out model for examining the role of visual imagery in episodic memory recall. Here, we assessed individuals with aphantasia on an adapted form of the Autobiographical Interview, a behavioural measure of the specificity and richness of episodic details underpinning the memory of events. Aphantasic participants generated significantly fewer episodic details than controls for both past and future events. This effect was most pronounced for novel future events, driven by selective reductions in visual detail retrieval, accompanied by comparatively reduced ratings of the phenomenological richness of simulated events, and paralleled by quantitative linguistic markers of reduced perceptual language use in aphantasic participants compared to those with visual imagery. Our findings represent the first systematic evidence (using combined objective and subjective data streams) that aphantasia is associated with a diminished ability to re-experience the past and simulate the future, indicating that visual imagery is an important cognitive tool for the dynamic retrieval and recombination of episodic details during mental simulation.