Cancer-associated fibroblasts (CAFs) are important, highly heterogeneous components of the tumor extracellular matrix that have different origins and express a diverse set of biomarkers. Different subtypes of CAFs participate in the immune regulation of the tumor microenvironment (TME). In addition to their role in supporting stromal cells, CAFs have multiple immunosuppressive functions, membrane and secretory patterns, against anti-tumor immunity. The inhibition of CAFs function and anti-TME therapy targeting CAFs provides new adjuvant means for immunotherapy. In this review, we outline the emerging understanding of CAFs with a particular emphasis on their origin and heterogeneity, different mechanisms of their regulation, as well as their direct or indirect effect on immune cells that leads to immunosuppression.