Invasive ductal carcinomas (IDC) of the breast are associated with altered expression of hormone receptors (HR), amplification or overexpression of HER2, or a triple-negative phenotype. The most aggressive cases of IDC are characterized by a high proliferation rate, a great propensity to metastasize, and their ability to resist to standard chemotherapy, hormone therapy, or HER2-targeted therapy. Using progression tissue microarrays, we here demonstrate that the serine/threonine kinase protein kinase D3 (PKD3) is highly upregulated in estrogen receptor (ER)-negative (ER(-)) tumors. We identify direct binding of the ER to the PRKD3 gene promoter as a mechanism of inhibition of PKD3 expression. Loss of ER results in upregulation of PKD3, leading to all hallmarks of aggressive IDC, including increased cell proliferation, migration, and invasion. This identifies ER(-) breast cancers as ideal for treatment with the PKD inhibitor CRT0066101. We show that similar to a knockdown of PKD3, treatment with this inhibitor targets all tumorigenic processes in vitro and decreases growth of primary tumors and metastasis in vivo. Our data strongly support the development of PKD inhibitors for clinical use for ER(-) breast cancers, including the triple-negative phenotype.