The evolution of antimicrobial resistance (AMR) in bacteria is a major public health concern, and antibiotic restriction is often implemented to reduce the spread of resistance. These measures rely on the existence of deleterious fitness effects (i.e. costs) imposed by AMR mutations during growth in the absence of antibiotics. According to this assumption, resistant strains will be outcompeted by susceptible strains that do not pay the cost during the period of restriction. The fitness effects of AMR mutations are generally studied in laboratory reference strains grown in standard growth environments; however, the genetic and environmental context can influence the magnitude and direction of a mutation's fitness effects. In this study, we measure how three sources of variation impact the fitness effects of Escherichia coli AMR mutations: the type of resistance mutation, the genetic background of the host, and the growth environment. We demonstrate that while AMR mutations are generally costly in antibiotic-free environments, their fitness effects vary widely and depend on complex interactions between the mutation, genetic background, and environment. We test the ability of the Rough Mount Fuji fitness landscape model to reproduce the empirical data in simulation. We identify model parameters that reasonably capture the variation in fitness effects due to genetic variation. However, the model fails to accommodate the observed variation when considering multiple growth environments. Overall, this study reveals a wealth of variation in the fitness effects of resistance mutations owing to genetic background and environmental conditions, which will ultimately impact their persistence in natural populations.