The role of estrogen receptor (ER) α as a target in treatment of breast cancer is clear, but those of ERβ1 and ERβ2 in the breast remain unclear. We have examined expression of all three receptors in surgically excised breast samples from two archives: (i): 187 invasive ductal breast cancer from a Japanese study; and (ii) 20 lobular and 24 ductal cancers from the Imperial College. Samples contained normal areas, areas of hyperplasia, and in situ and invasive cancer. In the normal areas, ERα was expressed in not more than 10% of epithelium, whereas approximately 80% of epithelial cells expressed ERβ. We found that whereas ductal cancer is a highly proliferative, ERα-positive, ERβ-negative disease, lobular cancer expresses both ERα and ERβ but with very few Ki67-positive cells. ERβ2 was expressed in 32% of the ductal cancers, of which 83% were postmenopausal. In all ERβ2-positive cancers the interductal space was filled with dense collagen, and cell nuclei expressed hypoxia-inducible factor 1α. ERβ2 expression was not confined to malignant cells but was strong in stromal, immune, and endothelial cells. In most of the high-grade invasive ductal cancers neither ERα nor ERβ was expressed, but in the high-grade lobular cancer ERβ was lost and ERα and Ki67 expression were abundant. The data show a clear difference in ER expression between lobular and ductal breast cancer and suggest (i) that tamoxifen may be more effective in late than in early lobular cancer and (ii) a potential role for ERβ agonists in preventing in situ ductal cancers from becoming invasive.