Math anxiety, defined as a negative affective response to mathematics, is suggested as a strong antecedent for the low visibility of women in the science and engineering workforce. However, the assumption of gender differences in math anxiety is still being studied and results are inconclusive, probably due to the use of explicit measures such as direct questionnaires. Thus, our primary objective was to investigate the effects of math anxiety on numerical processing in males and females by using a novel affective priming task as an indirect measure. Specifically, university students (23 males and 30 females) completed a priming task in which an arithmetic equation was preceded by one of four types of priming words (positive, neutral, negative, or related to mathematics). Participants were required to indicate whether the equation (simple math facts based on addition, subtraction, multiplication, or division) was true or false. People are typically found to respond to target stimuli more rapidly after presentation of an affectively related prime than after an affectively unrelated one. In the current study, shorter response latencies for positive as compared to negative affective primes were found in the male group. An affective priming effect was found in the female group as well, but with a reversed pattern. That is, significantly shorter response latencies were observed in the female group for negative as compared to positive targets. That is, for females, negative affective primes act as affectively related to simple arithmetic problems. In contrast, males associated positive affect with simple arithmetic. In addition, only females with lower or insignificant negative affect toward arithmetic study at faculties of mathematics and science. We discuss the advantages of examining pure anxiety factors with implicit measures which are free of response factors. In addition it is suggested that environmental factors may enhance the association between math achievements and math anxiety in females.