Lesion and functional brain imaging studies have suggested that there are two anatomically nonoverlapping attention networks. The dorsal frontoparietal network controls goal-oriented top-down deployment of attention; the ventral frontoparietal network mediates stimulus-driven bottom-up attentional reorienting. The interaction between the two networks and its functional significance has been considered in the past but no direct test has been carried out. We addressed this problem by recording fMRI data from human subjects performing a trial-by-trial cued visual spatial attention task in which the subject had to respond to target stimuli in the attended hemifield and ignore all stimuli in the unattended hemifield. Correlating Granger causal influences between regions of interest with behavioral performance, we report two main results. First, stronger Granger causal influences from the dorsal attention network (DAN) to the ventral attention network (VAN), i.e., DAN→VAN, are generally associated with enhanced performance, with right intraparietal sulcus (IPS), left IPS, and right frontal eye field being the main sources of behavior-enhancing influences. Second, stronger Granger causal influences from VAN to DAN, i.e., VAN→DAN, are generally associated with degraded performance, with right temporal-parietal junction being the main sources of behavior-degrading influences. These results support the hypothesis that signals from DAN to VAN suppress and filter out unimportant distracter information, whereas signals from VAN to DAN break the attentional set maintained by the dorsal attention network to enable attentional reorienting.