The brain site of perceptual learning has been frequently debated. Recent psychophysical evidence for complete learning transfer to new retinal locations and orientations/directions suggests that perceptual learning may mainly occur in high-level brain areas. Contradictorily, ERP C1 changes associated with perceptual learning are cited as evidence for training-induced plasticity in the early visual cortex. However, C1 can be top-down modulated, which suggests the possibility that C1 changes may result from top-down modulation of the early visual cortex by high-level perceptual learning. To single out the potential top-down impact, we trained observers with a peripheral orientation discrimination task and measured C1 changes at an untrained diagonal quadrant location where learning transfer was previously known to be significant. Our assumption was that any C1 changes at this untrained location would indicate top-down modulation of the early visual cortex, rather than plasticity in the early visual cortex. The expected learning transfer was indeed accompanied with significant C1 changes. Moreover, C1 changes were absent in an untrained shape discrimination task with the same stimuli. We conclude that ERP C1 can be top-down modulated in a task-specific manner by high-level perceptual learning, so that C1 changes may not necessarily indicate plasticity in the early visual cortex. Moreover, learning transfer and associated C1 changes may indicate that learning-based top-down modulation can be remapped to early visual cortical neurons at untrained locations to enable learning transfer.