Cancer-associated fibroblasts (CAFs) play a key role in cancer progression and treatment outcome. This study dissects the intra-tumoral diversity of CAFs in basal cell carcinoma, squamous cell carcinoma, and melanoma using molecular and spatial single-cell analysis. We identify three distinct CAF subtypes: myofibroblast-like RGS5+ CAFs, matrix CAFs (mCAFs), and immunomodulatory CAFs (iCAFs). Large-cohort tissue analysis reveals significant shifts in CAF subtype patterns with increasing malignancy. Two CAF subtypes exhibit immunomodulatory properties via different mechanisms. mCAFs sythesize extracellular matrix and may restrict T cell invasion in low-grade tumors via ensheathing tumor nests, while iCAFs are enriched in late-stage tumors, and express high levels of cytokines and chemokines to aid immune cell recruitment and activation. This is supported by the induction of an iCAF-like phenotype with immunomodulatory functions in primary healthy fibroblasts exposed to skin cancer cell secretomes. Thus, targeting CAF variants holds promise to enhance immunotherapy efficacy in skin cancers.