Nitric oxide (NO) is an endogenous, multipotent biological signaling molecule that participates in several physiological processes. Recently, exogenous supplementation of tumor tissues with NO has emerged as a potential anticancer therapy. In particular, it induces synergistic effects with other conventional therapies (such as chemo-, radio-, and photodynamic therapies) by regulating the activity of P-glycoprotein, acting as a vascular relaxant to relieve tumor hypoxia, and participating in the metabolism of reactive oxygen species. However, NO is highly reactive, and its half-life is relatively short after generation. Meanwhile, NO-induced anticancer activity is dose-dependent. Therefore, the targeted delivery of NO to the tumor is required for better therapeutic effects. In the past decade, NO-generating nanomedicines (NONs), which enable sustained and specific NO release in tumor tissues, have been developed for enhanced cancer therapy. This review describes the recent efforts and preclinical achievements in the development of NON-based cancer therapies. The chemical structures employed in the fabrication of NONs are summarized, and the strategies involved in NON-based cancer therapies are elaborated.