Immunotherapy is a powerful therapeutic strategy for end-stage hepatocellular carcinoma (HCC). It is well known that T cells, including CD8+PD-1+ T cells, play important roles involving tumor development. However, their underlying phenotypic and functional differences of T cell subsets remain unclear. We constructed single-cell immune contexture involving approximate 20,000,000 immune cells from 15 pairs of HCC tumor and non-tumor adjacent tissues and 10 blood samples (including five of HCCs and five of healthy controls) by mass cytometry. scRNA-seq and functional analysis were applied to explore the function of cells. Multi-color fluorescence staining and tissue micro-arrays were used to identify the pathological distribution of CD8+PD-1+CD161 +/- T cells and their potential clinical implication. The differential distribution of CD8+ T cells subgroups was identified in tumor and non-tumor adjacent tissues. The proportion of CD8+PD1+CD161+ T cells was significantly decreased in tumor tissues, whereas the ratio of CD8+PD1+CD161- T cells was much lower in non-tumor adjacent tissues. Diffusion analysis revealed the distinct evolutionary trajectory of CD8+PD1+CD161+ and CD8+PD1+CD161- T cells. scRNA-seq and functional study further revealed the stronger immune activity of CD8+PD1+CD161+ T cells independent of MHC class II molecules expression. Interestingly, a similar change in the ratio of CD8+CD161+/ CD8+CD161- T cells was also found in peripheral blood samples collected from HCC cases, indicating their potential usage clinically. We here identified different distribution, function, and trajectory of CD8+PD-1+CD161+ and CD8+PD-1+CD161- T cells in tumor lesions, which provided new insights for the heterogeneity of immune environment in HCCs and also shed light on the potential target for immunotherapy.