Despite the approval of immune checkpoint blockade (ICB) therapy for various tumor types, its effectiveness is limited to only approximately 15% of patients with microsatellite instability-high (MSI-H) or mismatch repair deficiency (dMMR) colorectal cancer (CRC). Approximately 80%-85% of CRC patients have a microsatellite stability (MSS) phenotype, which features a rare T-cell infiltration. Thus, elucidating the mechanisms underlying resistance to ICB in patients with MSS CRC is imperative. In this study, we demonstrate that ubiquitin-specific peptidase 4 (USP4) is upregulated in MSS CRC tumors and negatively regulates the immune response against tumors in CRC. Additionally, USP4 represses the cellular interferon (IFN) response and antigen presentation and impairs PRR signaling-mediated cell death. Mechanistically, USP4 impedes the nuclear localization of interferon regulator Factor 3 (IRF3) by deubiquitinating the K63-polyubiquitin chain of TRAF6 and IRF3. Knockdown of USP4 enhances the infiltration of T cells in CRC tumors and overcomes ICB resistance in an MC38 syngeneic mouse model. Moreover, published datasets revealed that patients showing higher USP4 expression exhibited decreased responsiveness to anti-PD-L1 therapy. These findings highlight an essential role of USP4 in the suppression of antitumor immunity in CRC.