Despite the promising outcomes of immune checkpoint inhibitors (ICIs), resistance to ICI presents a new challenge. Therefore, selecting patients for specific ICI applications is crucial for maximizing therapeutic efficacy. Herein, we curated 69 human esophageal squamous cell cancer (ESCC) patients' tumor microenvironment (TME) single-cell transcriptomic datasets to subtype ESCC. Integrative analyses of the cellular network and transcriptional signatures of T cells and myeloid cells define distinct ESCC subtypes characterized by T cell exhaustion, and interleukin (IL) and interferon (IFN) signaling. Furthermore, this approach classifies ESCC patients into ICI responders and non-responders, as validated by whole tumor transcriptomes and liquid biopsy-based single-cell transcriptomes of anti-PD-1 ICI responders and non-responders. Our study stratifies ESCC patients based on TME transcriptional network, providing novel insights into tumor niche remodeling and potentially predicting ICI responses in ESCC patients.