In recent years, immunotherapy has gradually become one of the main strategies for cancer treatment, with immune checkpoint inhibitors (ICIs) offering new possibilities for tumor therapy. However, some cancer patients exhibit low responses and resistance to ICIs treatment. T cell exhaustion, a process associated with tumor progression, refers to a subset of T cells that progressively lose effector functions and exhibit increased expression of inhibitory receptors. These exhausted T cells are considered key players in the therapeutic efficacy of immune checkpoint inhibitors. Therefore, understanding the impact of T cell exhaustion on tumor immunotherapy and the underlying mechanisms is critical for improving clinical treatment outcomes. Several elegant studies have provided insights into the prognostic value of exhausted T cells in cancers. In this review, we highlight the process of exhausted T cells and its predictive value in various cancers, as well as the relevant mechanisms behind it, providing new insights into the immunotherapy of cancer.