Desmoid fibromatosis of the breast (also known as desmoid tumor of the breast) is a rare entity infrequently encountered by oncologists and surgeons caring for patients with breast disease. The current body of literature is highly reliant on case series and extrapolations from other sites of desmoid tumor-related disease. Much remains unclear regarding the pathological origins, natural history, and response to treatment of this condition. Traditional treatment strategies have centered on surgical resection, which may result in significantly disfiguring cosmetic and functional outcomes, frequent need for re-operation, and associated morbidity. There are limited data to support the superiority of upfront surgical resection when compared to medical therapy or watchful waiting strategies. Current treatment guidelines for desmoid tumors do not focus on the breast as a site of disease and are purposefully ambiguous due to the paucity of evidence available. We aim to review the literature concerning desmoid fibromatosis of the breast and propose an algorithm for current evidence-based management of this rare disease in the context of our experience with this pathology at a high-volume quaternary referral center.