Human Papillomavirus (HPV)-negative head and neck squamous cell carcinoma (HNSCC) remains a challenging malignancy, with radiotherapy, alone or combined with immune checkpoint inhibitors, often failing to achieve durable disease control. Here, by conducting longitudinal multi-omic analyses of pre- and post-radiation biopsies from patients receiving a pre-operative hypofractionated radiation regimen, we uncover that radiation rapidly depletes a subpopulation of tumor-infiltrating lymphocytes (TIL), characterized by a proliferative, cytotoxic, and tissue-resident gene signature (T). We provide multi-dimensional evidence for tumor antigen-specificity of T clonotypes and show that post-radiation tumors are instead repopulated by regulatory and non-specific clones. Finally, TIL depletion correlates with radiorecurrent disease after conventional radiation, emphasizing the potential impact of radiation-induced TIL loss regardless of fractionation. Thus, this study provides key insights into radiotherapy-induced alterations in the immune microenvironment that drive immunologic radioresistance and proposes restoring tumor antigen-specific T cell clonotypes as a strategy to improve radioimmunotherapy responses in HNSCC.