Human serum albumin (HSA) improves the pharmacokinetic profile of drugs attached to it, making it an attractive carrier with proven clinical success. In our previous studies, we have shown that Caveolin-1 (Cav-1) and caveolae-mediated endocytosis play important roles in the uptake of HSA and albumin-bound drugs. Doxorubicin is an FDA-approved chemotherapeutic agent that is effective against multiple cancers, but its clinical applicability has been hampered by its high toxicity levels. In this study, a doxorubicin-prodrug was developed that could independently and avidly bind HSA in circulation, called IPBA-Dox. We first developed and characterized IPBA-Dox and confirmed that it can bind albumin in vitro while retaining a potent cytotoxic effect. We then verified that it efficiently binds to HSA in circulation, leading to an improvement in the pharmacokinetic profile of the drug. In addition, we tested our prodrug for Cav-1 selectivity and found that it preferentially affects cells that express relatively higher levels of Cav-1 in vitro and in vivo. Moreover, we found that our compound was well tolerated in vivo at concentrations at which doxorubicin was lethal. Altogether, we have developed a doxorubicin-prodrug that can successfully bind HSA, retaining a strong cytotoxic effect that preferentially targets Cav-1 positive cells while improving the general tolerability of the drug.