The density of nerves in cancer is emerging as a relevant clinical parameter for patient survival. Nerves in the tumor microenvironment have been associated with poor survival and recurrence, particularly if involved in perineural invasion. However, usually only a few nerves inside a tumor are affected by perineural invasion, while most nerves are not. Mechanistic studies have shown nerve-secreted factors promote tumor growth and invasion thereby making tumors more aggressive. Therefore, the overall number of nerves in the tumor microenvironment should be more representative of the nerve-tumor biological interaction than perineural invasion. This review summarizes the available clinical information about nerve density as a measure of clinical outcome in cancer and explores the mechanisms underlying nerve density in cancer, specifically, neurogenesis, axonogenesis, and neurotropism.