While initial encoding of contextual memories involves the strengthening of hippocampal circuits, these memories progressively mature to stabilized forms in neocortex and become less hippocampus dependent. Although it has been proposed that long-term storage of contextual memories may involve enduring synaptic changes in neocortical circuits, synaptic substrates of remote contextual memories have been elusive. Here we demonstrate that the consolidation of remote contextual fear memories in mice correlated with progressive strengthening of excitatory connections between prefrontal cortical (PFC) engram neurons active during learning and reactivated during remote memory recall, whereas the extinction of remote memories weakened those synapses. This synapse-specific plasticity was CREB-dependent and required sustained hippocampal signals, which the retrosplenial cortex could convey to PFC. Moreover, PFC engram neurons were strongly connected to other PFC neurons recruited during remote memory recall. Our study suggests that progressive and synapse-specific strengthening of PFC circuits can contribute to long-term storage of contextual memories.