Cancer-associated fibroblasts play a crucial role within the tumor microenvironment. However, a comprehensive characterization of CAF in colorectal cancer (CRC) is still missing. We combined scRNA-seq and spatial proteomics to decipher fibroblast heterogeneity in healthy human colon and CRC at high resolution. Analyzing nearly 23,000 fibroblasts, we identified 11 distinct clusters and verified them by spatial proteomics. Four clusters, consisting of myofibroblastic CAF (myCAF)-like, inflammatory CAF (iCAF)-like and proliferating fibroblasts as well as a novel cluster, which we named "T cell-inhibiting CAF" (TinCAF), were primarily found in CRC. This new cluster was characterized by the expression of immune-interacting receptors and ligands, including CD40 and NECTIN2. Co-culture of CAF and T cells resulted in a reduction of the effector T cell compartment, impaired proliferation, and increased exhaustion. By blocking its receptor interaction, we demonstrated that NECTIN2 was the key driver of T cell inhibition. Analysis of clinical datasets showed that NECTIN2 expression is a poor prognostic factor in CRC and other tumors. In conclusion, we identified a new class of immuno-suppressive CAF with features rendering them a potential target for future immunotherapies.