The therapeutic effect of nanoparticles is limited in solid tumors, especially desmoplastic tumors, because the tumor matrix hinders the delivery of nanoparticles. As the most abundant cells in the tumor stroma, tumor-associated fibroblasts (TAFs) produce a dense extracellular matrix, which leads to higher tissue fluid pressure, thereby creating a physical barrier for nanoparticle delivery. Therefore, researchers focused on eliminating TAFs to combat desmoplastic tumors. In recent years, a series of methods for TAFs have been developed. In this paper, we first introduced the biological mechanism of TAFs hindering the penetration of nanoparticles. Then, the different methods of eliminating TAFs were summarized, and the mechanism of nanomedicine in eliminating TAFs was highlighted. Finally, the problems and future development directions for TAFs treatment were discussed from the perspective of the treatment of desmoplastic tumors.