The rise and fall of estrogen and progesterone across menstrual cycles and during pregnancy regulates breast development and modifies cancer risk. How these hormones impact each cell type in the breast remains poorly understood because they act indirectly through paracrine networks. Using single-cell analysis of premenopausal breast tissue, we reveal a network of coordinated transcriptional programs representing the tissue-level response to changing hormone levels. Our computational approach, DECIPHER-seq, leverages person-to-person variability in breast composition and cell state to uncover programs that co-vary across individuals. We use differences in cell-type proportions to infer a subset of programs that arise from direct cell-cell interactions regulated by hormones. Further, we demonstrate that prior pregnancy and obesity modify hormone responsiveness through distinct mechanisms: obesity reduces the proportion of hormone-responsive cells, whereas pregnancy dampens the direct response of these cells to hormones. Together, these results provide a comprehensive map of the cycling human breast.