Our brains must maintain a representation of the world over a period of time much longer than the typical lifetime of the biological components producing that representation. For example, recent research suggests that dendritic spines in the adult mouse hippocampus are transient with an average lifetime of ~10 days. If this is true, and if turnover is equally likely for all spines, ~95% of excitatory synapses onto a particular neuron will turn over within 30 days; however, a neuron's receptive field can be relatively stable over this period. Here, we use computational modeling to ask how memories can persist in neural circuits such as the hippocampus and visual cortex in the face of synapse turnover. We demonstrate that Hebbian plasticity during replay of presynaptic activity patterns can integrate newly formed synapses into pre-existing memories. Furthermore, we find that Hebbian plasticity during replay is sufficient to stabilize the receptive fields of hippocampal place cells in a model of the grid-cell-to-place-cell transformation in CA1 and of orientation-selective cells in a model of the center-surround-to-simple-cell transformation in V1. Together, these data suggest that a simple plasticity rule, correlative Hebbian plasticity of synaptic strengths, is sufficient to preserve neural representations in the face of synapse turnover, even in the absence of activity-dependent structural plasticity. Recent research suggests that synapses turn over rapidly in some brain structures; however, memories seem to persist for much longer. We show that Hebbian plasticity of synaptic strengths during reactivation events can preserve memory in computational models of hippocampal and cortical networks despite turnover of all synapses. Our results suggest that memory can be stored in the correlation structure of a network undergoing rapid synaptic remodeling.