CDKN2A is a tumor suppressor located in chromosome 9p21 and frequently lost in Barrett's esophagus (BE) and esophageal adenocarcinoma (EAC). How CDKN2A and other 9p21 gene co-deletions affect EAC evolution remains understudied. We explored the effects of 9p21 loss in EACs and cancer progressor and non-progressor BEs with matched genomic, transcriptomic and clinical data. Despite its cancer driver role, CDKN2A loss in BE prevents EAC initiation by counterselecting subsequent TP53 alterations. 9p21 gene co-deletions predict poor patient survival in EAC but not BE through context-dependent effects on cell cycle, oxidative phosphorylation and interferon response. Immune quantifications using bulk transcriptome, RNAscope and high-dimensional tissue imaging showed that IFNE loss reduces immune infiltration in BE, but not EAC. Mechanistically, CDKN2A loss suppresses the maintenance of squamous epithelium, contributing to a more aggressive phenotype. Our study demonstrates context-dependent roles of cancer genes during disease evolution, with consequences for cancer detection and patient management.