We studied the correlation between epidermal growth factor receptor (EGFR) and the tumor stem cell markers CD44/CD24 in breast invasive ductal carcinoma (BIDC) and their relationship with prognosis. We analyzed the clinical data of 139 BIDC cases retrospectively, detecting EGFR, CD44, and CD24 expressions in tumor tissue using immunohistochemistry. The proportion of EGFR-, CD44-, and CD24-positive cases was 59.0, 62.3, and 30.9 %, respectively. The proportion of CD44-positive [76.9 % (p < 0.05)] and EGFR-positive [67.2 % (p = 0.108)] cases in the triple-negative breast cancer (TNBC) group was higher than that of the non-TNBC group. In the non-TNBC group, 36.5 % was CD24-positive, higher than that in the TNBC group but not statistically significant. The proportion of CD44-positive cases was significantly higher in the EGFR-positive group than in the EGFR-negative group (p = 0.017). EGFR-positive cases were significantly correlated with premenopausal status (p = 0.036), distant metastasis (p = 0.018), and estrogen receptor-negative status (p = 0.020). CD44-positive status was significantly correlated with human epidermal growth receptor 2 (HER2)-negative (p = 0.023), estrogen receptor-negative (p = 0.021), and progesterone receptor-negative status (p = 0.004). CD24-positive status was significantly correlated with HER2-positive status (p = 0.001). Kaplan-Meier survival analysis showed that TNBC patients had shorter survival. EGFR-positive and CD44-positive status were both correlated with shorter survival in the lymph node- and HR-negative groups, while CD24 positive was significantly correlated with poor survival in lymph node-negative and HR-positive patients. EGFR and CD44 expressions have a significantly positive correlation (p = 0.017) in BIDC. Patients both EGFR and CD44 positive had the worst outcome.