Macrophages, myeloid-derived suppressor cells and tolerogenic dendritic cells are central players of a heterogeneous myeloid cell population, with the ability to suppress innate and adaptive immune responses and thus to promote tumor growth. Their influx and local proliferation are mainly induced by the cancers themselves, and their numbers in the tumor microenvironment and the peripheral blood correlate with decreased survival. Therapeutic targeting these innate immune cells, either aiming at their elimination or polarization toward tumor suppressive cells is an attractive novel approach to control tumor progression and block metastasis. We review the current understanding of cancer immunology including immune surveillance and immune editing in the context of these prominent innate suppressor cells, and their targetability by nanoparticular immunotherapy with small molecules or siRNA.