The genomic landscape of esophageal squamous cell cancer (ESCC), as well as its impact on the regulation of immune microenvironment, is not well understood. Thus, tumor samples from 92 patients were collected from two centers and subjected to targeted-gene sequencing. We identified frequently mutated genes, including TP53, KMT2C, KMT2D, LRP1B, and FAT1. The most frequent mutation sites were ALOX12B (c.1565C > T), SLX4 (c.2786C > T), LRIG1 (c.746A > G), and SPEN (c.6915_6917del) (6.5%). Pathway analysis revealed dysregulation of cell cycle regulation, epigenetic regulation, PI3K/AKT signaling, and NOTCH signaling. A 17-mutated gene-related risk model was constructed using random survival forest analysis and showed significant prognostic value in both our cohort and the validation cohort. Based on the Estimation of Stromal and Immune cells in Malignant Tumor tissues using Expression (ESTIMATE) algorithm, the Tumor Immune Dysfunction and Exclusion (TIDE) algorithm, and the MCPcounter algorithm, we found that the risk score calculated by the risk model was significantly correlated with stimulatory immune checkpoints (TNFSF4, ITGB2, CXCL10, CXCL9, and BTN3A1; p < 0.05). Additionally, it was significantly associated with markers that are important in predicting response to immunotherapy (CD274, IFNG, and TAMM2; p < 0.05). Furthermore, the results of immunofluorescence double staining showed that patients with high risk scores had a significantly higher level of M2 macrophage than those with low risk scores (p < 0.05). In conclusion, our study provides insights into the genomic landscape of ESCC and highlights the prognostic value of a genomic mutation signature associated with the immune microenvironment in southern Chinese patients with ESCC.