To elucidate the effects of neoadjuvant chemotherapy (NAC), we conduct whole transcriptome profiling coupled with histopathology analyses of a longitudinal breast cancer cohort of 146 patients including 110 pairs of serial tumor biopsies collected before treatment, after the first cycle of treatment and at the time of surgery. Here, we show that cytotoxic chemotherapies induce dynamic changes in the tumor immune microenvironment that vary by subtype and pathologic response. Just one cycle of treatment induces an immune stimulatory microenvironment harboring more tumor infiltrating lymphocytes (TILs) and up-regulation of inflammatory signatures predictive of response to anti-PD1 therapies while residual tumors are immune suppressed at end-of-treatment compared to the baseline. Increases in TILs and CD8+ T cell proportions in response to NAC are independently associated with pathologic complete response. Further, on-treatment immune response is more predictive of treatment outcome than immune features in paired baseline samples although these are strongly correlated.