Breast cancer (BCa) incidence increases following aberrant hormone exposure, which has been linked to direct effects on estrogen receptor (ER) mammary epithelium. While estrogen exposure during mammary involution has been shown to drive tumour growth via neutrophils, the potential for the ER + immune microenvironment to mediate part (in addition to mammary epithelial cells) of hormonally controlled BCa risk during normal development has not been assessed. We collected mammary tissue, lymph nodes and blood from tumour naïve mice treated with, oophorectomy, estrogen (17β estradiol) or Fulvestrant. Flow cytometry was used to examine the impact on the frequency of innate and adaptive immune cells. Oophorectomy and fulvestrant decreased the proportion of macrophages, particularly pro-tumour polarized M2 macrophages and neutrophils. Conversely, dendritic cells were increased by these therapies, as were eosinophils. Estrogen increased the proportion of M2 macrophages and to a lesser extent CD4-CD8- double negative and FoxP3 regulatory T cells but decreased CD8 + T cells and B cells. Excluding eosinophils, these changes were restricted to the mammary tissue. This suggests that inhibiting estrogen action lowers the immune suppressive myeloid cells, increases in antigen presentation and eosinophil-mediated direct or indirect cytotoxic effects. In contrast, estrogen exposure, which drives BCa risk, increases the suppressive myeloid cells and reduces anti-tumour cytotoxic T cells. The impact of hormonal exposure on BCa risk, may in part be linked to its immune modulatory activity.