The evolution of plant defence in response to herbivory will depend on the fitness effects of damage, availability of genetic variation and potential ecological and genetic constraints on defence. Here, we examine the potential for evolution of tolerance to deer herbivory in Oenothera biennis while simultaneously considering resistance to natural insect herbivores. We examined (i) the effects of deer damage on fitness, (ii) the presence of genetic variation in tolerance and resistance, (iii) selection on tolerance, (iv) genetic correlations with resistance that could constrain evolution of tolerance and (v) plant traits that might predict defence. In a field experiment, we simulated deer damage occurring early and late in the season, recorded arthropod abundances, flowering phenology and measured growth rate and lifetime reproduction. Our study showed that deer herbivory has a negative effect on fitness, with effects being more pronounced for late-season damage. Selection acted to increase tolerance to deer damage, yet there was low and nonsignificant genetic variation in this trait. In contrast, there was substantial genetic variation in resistance to insect herbivores. Resistance was genetically uncorrelated with tolerance, whereas positive genetic correlations in resistance to insect herbivores suggest there exists diffuse selection on resistance traits. In addition, growth rate and flowering time did not predict variation in tolerance, but flowering phenology was genetically correlated with resistance. Our results suggest that deer damage has the potential to exert selection because browsing reduces plant fitness, but limited standing genetic variation in tolerance is expected to constrain adaptive evolution in O. biennis.