Although distant metastases account for most of the deaths in cancer patients, fundamental questions regarding mechanisms that promote or inhibit metastasis remain unanswered. We show the impact of mutations, genomic instability, lymphatic and blood vascularization, and the immune contexture of the tumor microenvironment on synchronous metastases in large cohorts of colorectal cancer patients. We observed large genetic heterogeneity among primary tumors, but no major differences in chromosomal instability or key cancer-associated mutations. Similar patterns of cancer-related gene expression levels were observed between patients. No cancer-associated genes or pathways were associated with M stage. Instead, mutations of FBXW7 were associated with the absence of metastasis and correlated with increased expression of T cell proliferation and antigen presentation functions. Analyzing the tumor microenvironment, we observed two hallmarks of the metastatic process: decreased presence of lymphatic vessels and reduced immune cytotoxicity. These events could be the initiating factors driving both synchronous and metachronous metastases. Our data demonstrate the protective impact of the Immunoscore, a cytotoxic immune signature, and increased marginal lymphatic vessels, against the generation of distant metastases, regardless of genomic instability.