A stroke can disrupt the finely tuned language network resulting in aphasia, a language impairment. Though many stroke survivors with aphasia recover within the first 6 months, a significant proportion have lasting deficits. The factors contributing to optimal treatment response remain unclear. Some evidence suggests that increased modularity or fragmentation of brain networks may underlie post-stroke aphasia severity and the extent of recovery. We examined associations between network organization and aphasia recovery in sixteen chronic stroke survivors with non-fluent aphasia following 35 h of Multi-Modality Aphasia Therapy over 10 days and 20 healthy controls who underwent imaging at a single timepoint. Using diffusion-weighted scans obtained before and after treatment, we constructed whole-brain structural connectomes representing the number of probabilistic streamlines between brain regions. Graph theory metrics were quantified for each connectome using the Brain Connectivity Toolbox. Correlations were examined between graph metrics and speech performance measured using the Boston Naming Test (BNT) at pre-, post- and 3-months post-intervention. Compared to controls, participants with stroke demonstrated higher whole-brain modularity at pre-treatment. Modularity did not differ between pre- and post-treatment. In individuals who responded to therapy, higher pre-treatment modularity was associated with worse performance on the BNT. Moreover, higher pre-treatment participation coefficients (i.e., how well a region is connected outside its own module) for the left IFG, planum temporale, and posterior temporal gyri were associated with greater improvements at post-treatment. These results suggest that pre-treatment network topology may impact therapeutic gains, highlighting the influence of network organization on post-stroke aphasia recovery.