The immune system functions in a sexually dimorphic manner, with females exhibiting more robust immune responses than males. However, how female sex hormones affect immune function in normal homeostasis and in autoimmunity is poorly understood. In this review, we discuss how estrogens affect innate and adaptive immune cell activity and how dysregulation of estrogen signaling underlies the pathobiology of some autoimmune diseases and cancers. The potential roles of the major circulating estrogens, and each of the 3 estrogen receptors (ERα, ERβ, and G-protein coupled receptor) in the regulation of the activity of different immune cells are considered. This provides the framework for a discussion of the impact of ER modulators (aromatase inhibitors, selective estrogen receptor modulators, and selective estrogen receptor downregulators) on immunity. Synthesis of this information is timely given the considerable interest of late in defining the mechanistic basis of sex-biased responses/outcomes in patients with different cancers treated with immune checkpoint blockade. It will also be instructive with respect to the further development of ER modulators that modulate immunity in a therapeutically useful manner.