Many bacteria, including the model bacterium Escherichia coli can survive for years within spent media, following resource exhaustion. We carried out evolutionary experiments, followed by whole genome sequencing of hundreds of evolved clones to study the dynamics by which E. coli adapts during the first 4 months of survival under resource exhaustion. Our results reveal that bacteria evolving under resource exhaustion are subject to intense selection, manifesting in rapid mutation accumulation, enrichment in functional mutation categories and extremely convergent adaptation. In the most striking example of convergent adaptation, we found that across five independent populations adaptation to conditions of resource exhaustion occurs through mutations to the three same specific positions of the RNA polymerase core enzyme. Mutations to these three sites are strongly antagonistically pleiotropic, in that they sharply reduce exponential growth rates in fresh media. Such antagonistically pleiotropic mutations, combined with the accumulation of additional mutations, severely reduce the ability of bacteria surviving under resource exhaustion to grow exponentially in fresh media. We further demonstrate that the three positions at which these resource exhaustion mutations occur are conserved for the ancestral E. coli allele, across bacterial phyla, with the exception of nonculturable bacteria that carry the resource exhaustion allele at one of these positions, at very high frequencies. Finally, our results demonstrate that adaptation to resource exhaustion is not limited by mutational input and that bacteria are able to rapidly adapt under resource exhaustion in a temporally precise manner through allele frequency fluctuations.