CD8+ tumor-infiltrating lymphocytes (TILs) are associated with improved survival in triple-negative breast cancer (TNBC) yet have no association with survival in estrogen receptor-positive (ER+) BC. The basis for these contrasting findings remains elusive. We identified subsets of BC tumors infiltrated by CD8+ T cells with characteristic features of exhausted T cells (TEX). Tumors with abundant CD8+ TEX exhibited a distinct tumor microenvironment marked by amplified interferon-γ signaling-related pathways and higher programmed death ligand 1 expression. Paradoxically, higher levels of tumor-infiltrating CD8+ TEX associated with decreased overall survival of patients with ER+ BC but not patients with TNBC. Moreover, high tumor expression of a CD8+ TEX signature identified dramatically reduced survival in premenopausal, but not postmenopausal, patients with ER+ BC. Finally, we demonstrated the value of a tumor TEX signature score in identifying high-risk premenopausal ER+ BC patients among those with intermediate Oncotype DX Breast Recurrence Scores. Our data highlight the complex relationship between CD8+ TILs, interferon-γ signaling, and ER status in BC patient survival. This work identifies tumor-infiltrating CD8+ TEX as a key feature of reduced survival outcomes in premenopausal patients with early-stage ER+ BC.