Learning never stops. As we navigate life, we continuously acquire and update knowledge to optimize memory-guided action, with a gradual shift from the former to the latter as we master our environment. How are these learning dynamics expressed in the brain and in behavioral patterns? Here, we devised a spatiotemporal image learning task ("Memory Arena") in which participants learn a set of 50 items to criterion across repeated exposure blocks. Critically, brief task-free periods between successive image presentations allowed us to assess multivariate electroencephalogram (EEG) patterns representing the previous and/or upcoming image identity, as well as anticipatory eye movements toward the upcoming image location. As expected, participants eventually met the performance criterion, albeit with different learning rates. During task-free periods, we were able to readily decode representations of both previous and upcoming image identities. Importantly though, decoding strength followed opposing slopes for previous vs. upcoming images across time, with a gradual decline of evidence for the previous image and a gradual increase of evidence for the upcoming image. Moreover, the ratio of upcoming vs. previous image evidence directly followed behavioral learning rates. Finally, eye movement data revealed that participants increasingly used the task-free period to anticipate upcoming image locations, with target-precision slopes paralleling both behavioral performance measures as well as EEG decodability of the upcoming image across time. Together, these results unveil the neural and behavioral dynamics underlying the gradual transition from learning to memory-guided action.