Local intestinal invasion of tumor cells often leads to recurrent and refractory colorectal cancer (CRC). However, the driven mechanism is not fully understood. A total of 145 patients with CRC and 10 patients with intestinal perforations or benign lesions were randomly enrolled. The distribution and clinical relevance of macrophages in different tissues were determined by flow cytometry and immunohistochemistry. PCR screening and RNA-sequencing analysis were used to explore the regulatory mechanisms. The functions of macrophages were further verified using an orthotopic mouse model of mice. Here, we unveil the role of M2 macrophages in local intestinal invasion. M2 macrophages infiltrated more in peritumor tissues than in tumor and normal tissues of CRC patients, which were significantly associated with the tumor local intestinal invasion and recurrence. Macrophage elimination attenuated local intestinal invasion. Mechanistically, CXCL12 is highly expressed in peritumor tissues and recruits monocytes and polarizes them into M2 macrophages by binding to CXCR4. Furthermore, -containing exosomes from primary colorectal tumor cells promoted CXCL12 secretion by peritumoral fibroblasts through HIF-2α binding to the CXCL12 promoter, which in turn induced M2 macrophage accumulation and tumor cell invasion. Macrophage-specific deletion of CXCR4 or knockdown of in colorectal tumor cells reduced M2 accumulation in peritumor tissue and subsequent local invasion of tumor cells. These data reveal a fine-tuned collaborative action between primary cancer cells and peritumoral stromal cells in distinct tumor areas, which reshape the immunosuppressive microenvironment and inducing local intestinal invasion via the HIF2A/CXCL12/CXCR4 axis.