The clinical and pathological outcomes differ between early-onset (diagnosed in men ≤55 years of age) and late-onset prostate cancer, potentially attributed to the changes in hormone levels and immune activities associated with aging. Exploring the heterogeneity therein holds potential for developing age-specific precision interventions. Here, through single-cell and spatial transcriptomic analyses of prostate cancer tissues, we identified that an androgen response-related transcriptional meta-program (AR-MP) might underlie the age-related heterogeneity of tumor cells and microenvironment. APOE tumor-associated macrophages infiltrated AR-MP-activated tumor cells in early-onset prostate cancer, potentially facilitating tumor progression and immunosuppression. By contrast, inflammatory cancer-associated fibroblasts in late-onset prostate cancer correlated with downregulation of AR-MP of tumor cells and increased epithelial-to-mesenchymal transition and pre-existing castration resistance, which may also be linked to smoking. This study provides potential insights for tailoring precision treatments by age groups, emphasizing interventions that include targeting AR and tumor-associated macrophages in young patients but anchoring epithelial-to-mesenchymal transition and inflammatory cancer-associated fibroblasts in old counterparts.