Physically triggered systems hold promise for improving drug delivery by enhancing the controllability of drug accumulation and release, lowering non-specific toxicity, and facilitating clinical translation. Several external physical stimuli including ultrasound, light, electric fields and magnetic fields have been used to control drug delivery and they share some common features such as spatial targeting, spatiotemporal control, and minimal invasiveness. At the same time, they possess several distinctive features in terms of interactions with biological entities and/or the extent of stimulus response. Here, we review the key advances of such systems with a focus on discussing their physical mechanisms, the design rationales, and translational challenges.