Visual selective attention operates through top-down mechanisms of signal enhancement and suppression, mediated by α-band oscillations. The effects of such top-down signals on local processing in primary visual cortex (V1) remain poorly understood. In this work, we characterize the interplay between large-scale interactions and local activity changes in V1 that orchestrates selective attention, using Granger-causality and phase-amplitude coupling (PAC) analysis of EEG source signals. The task required participants to either attend to or ignore oriented gratings. Results from time-varying, directed connectivity analysis revealed frequency-specific effects of attentional selection: bottom-up γ-band influences from visual areas increased rapidly in response to attended stimuli while distributed top-down α-band influences originated from parietal cortex in response to ignored stimuli. Importantly, the results revealed a critical interplay between top-down parietal signals and α-γ PAC in visual areas. Parietal α-band influences disrupted the α-γ coupling in visual cortex, which in turn reduced the amount of γ-band outflow from visual areas. Our results are a first demonstration of how directed interactions affect cross-frequency coupling in downstream areas depending on task demands. These findings suggest that parietal cortex realizes selective attention by disrupting cross-frequency coupling at target regions, which prevents them from propagating task-irrelevant information.