Classic models propose that forming lasting visual memories involves coordinated interactions between visually selective neocortical structures and the hippocampus during memory consolidation. However, the precise role of visually selective neocortical structures in memory consolidation remains elusive, given their potential contributions spanning from initial perceptual encoding to subsequent memory reactivation. We capitalized on a unique opportunity, involving direct recording from the posterior parahippocampus and its subsequent resection in a neurological patient, to investigate the impact of scene-selective neocortical lesions on visual memory consolidation. First, with intracranial EEG, we confirmed the functional relevance of the patient's resected tissues in representing a specific visual category, in this case, scene images. Subsequently, we identified disruption of memory for scenes relative to faces and objects during the participant's postoperative visit. This finding prompted a comprehensive analysis of visual memory across different visual categories in this participant, as well as an examination of similar functions in other neurological patients with intact parahippocampi and a cohort of online participants. Through these within- and between-participant comparisons, we identified greater time-dependent reduction in visual memory for scene images following the resection of the posterior parahippocampus. Importantly, these changes in memory retention could not be attributed to a general reduction in initial memory encoding following neocortical lesions. Our findings, therefore, suggest that reactivating scene-selective neocortical areas is essential for converting transient visual perceptual experiences into lasting long-term scene memories.