Consume of stimulants (as caffeine) is very usual in different contexts where the performers have to take quick and accurate decisions during physical effort. Decision-making processes are mediated by the attentional networks. An experiment was carried out to examine the effect of caffeine intake on attention (alerting, orienting, and executive control) as a function of consumption habit under two physical exertion conditions (rest vs. aerobic exercise). Two groups of participants with different caffeine consumption profiles (moderate consumers vs. low consumers) performed the Attention Network Test-Interactions under four different conditions regarding activity (rest vs. exercise) and intake (caffeine vs. placebo). Results showed that whereas exercise led to faster reaction times (RT) in all cases, caffeine intake accelerated RT but only at rest and in moderate caffeine consumers. More importantly, caffeine intake reduced the alertness effect in moderate consumers only at the rest condition. No interactions between Intake and Activity were observed in the other attentional networks, with exercise reducing orienting independently of caffeine intake, which suggests that physical exercise and caffeine are different modulators of attention but can interact. Caffeine intake had differential effects on reaction speed at rest and during physical exercise depending on the individual consumption habit. On the basis of these finding it seems that mainly alertness is modulated differently by internal and external "arousing" conditions.