Immunotherapy has demonstrated its potential to improve the prognosis of patients with hepatocellular carcinoma (HCC); however, patients' responses to immunotherapy vary a lot. A comparative analysis of the tumor microenvironment (TME) in responders and non-responders is expected to unveil the mechanisms responsible for the immunotherapy resistance and provide potential treatment targets. We performed sequencing analyses using 10x Genomics technology on six HCC patients who responded to anti-PD-1 therapy and one HCC patient who did not respond. Additionally, we obtained single cell data from untreated, responsive, and nonresponsive HCC patients from public databases, and used part of the datasets as a validation cohort. These data were integrated using algorithms such as Harmony. An independent validation cohort was established. Furthermore, we performed spatial transcriptomic sequencing on the tumor adjacent tissues of three HCC responsive patients using 10x Genomics spatial transcriptomic technology. Additionally, we analyzed data about three HCC patients obtained from public databases. Finally, we validated our conclusions using immunofluorescence, flow cytometry, and experiments. Our findings confirmed the presence of "immune barrier" partially accounting for the limited efficacy of immunotherapy. Our analysis revealed a significant increase in TREM2 Macrophages among non-responsive patients expressing multiple immunosuppressive signals. anti-Csf1r monoclonal antibodies effectively eliminated these macrophages and augmented the therapeutic effects of anti-PD-1 therapy. TCR Macrophages possessed direct tumor-killing capabilities. IL1B cDC2 was the primary functional subtype of cDC2 cells. Absence of THEMIS CD8 T subtypes might diminish immunotherapeutic effects. Furthermore, CD8 T cells entered a state of stress after anti-PD-1 treatment, which might be associated with CD8 T cell exhaustion and senescence. The profiles of immune TMEs showed differences in HCC patients responsive, non-responsive and untreated. These differences might explain the discounted efficacy of immunotherapy in some HCC patients. The cells and molecules, which we found to carry unique capabilities, may be targeted to enhance immunotherapeutic outcomes in patients with HCC.