It is generally accepted that metabolism is able to shape the immune response. Only recently we are gaining awareness that the metabolic crosstalk between different tumor compartments strongly contributes to the harsh tumor microenvironment (TME) and ultimately impairs immune cell fitness and effector functions. The major aims of this review are to provide an overview on the immune system in cancer; to position oxygen shortage and metabolic competition as the ground of a restrictive TME and as important players in the anti-tumor immune response; to define how immunotherapies affect hypoxia/oxygen delivery and the metabolic landscape of the tumor; and vice versa, how oxygen and metabolites within the TME impinge on the success of immunotherapies. By analyzing preclinical and clinical endeavors, we will discuss how a metabolic characterization of the TME can identify novel targets and signatures that could be exploited in combination with standard immunotherapies and can help to predict the benefit of new and traditional immunotherapeutic drugs.