Designing specific therapies for drug-resistant cancers is arguably the ultimate challenge in cancer therapy. While much emphasis has been put on the study of genetic alterations that give rise to drug resistance, much less is known about the non-genetic adaptation mechanisms that operate during the early stages of drug resistance development. Drug-tolerant persister cells have been suggested to be key players in this process. These cells are thought to have undergone non-genetic adaptations that enable survival in the presence of a drug, from which full-blown resistant cells may emerge. Such initial adaptations often involve engagement of stress response programs to maintain cancer cell viability. In this review, we discuss the nature of drug-tolerant cancer phenotypes, as well as the non-genetic adaptations involved. We also discuss how malignant cells employ homeostatic stress response pathways to mitigate the intrinsic costs of such adaptations. Lastly, we discuss which vulnerabilities are introduced by these adaptations and how these might be exploited therapeutically.