Drug conjugates have emerged as a promising alternative delivery system designed to deliver an ultra-toxic payload directly to the target cancer cells, maximizing therapeutic efficacy while minimizing toxicity. Among these, antibody-drug conjugates (ADCs) have garnered significant attention from both academia and industry due to their great potential for cancer therapy. However, peptide-drug conjugates (PDCs) offer several advantages over ADCs, including more accessible industrial synthesis, versatile functionalization, high tissue penetration, and rapid clearance with low immunotoxicity. These factors position PDCs as up-and-coming drug candidates for future cancer therapy. Despite their potential, PDCs face challenges such as poor pharmacokinetic properties and low bioactivity, which hinder their clinical development. How to design PDCs to meet clinical needs is a big challenge and urgent to resolve. In this review, we first carefully analyzed the general consideration of successful PDC design learning from ADCs. Then, we summarised the basic functions of each component of a PDC construct, comprising of peptides, linkers and payloads. The peptides in PDCs were categorized into three types: tumor targeting peptides, cell penetrating peptide and self-assembling peptide. We then analyzed the potential of these peptides for drug delivery, such as overcoming drug resistance, controlling drug release and improving therapeutic efficacy with reduced non-specific toxicity. To better understand the potential druggability of PDCs, we discussed the pharmacokinetics of PDCs and also briefly introduced the current PDCs in clinical trials. Lastly, we discussed the future perspectives for the successful development of an oncology PDC. This review aimed to provide useful information for better construction of PDCs in future clinical applications.