A major challenge in the cognitive training field is inducing broad, far-transfer training effects. Thus far, little is known about the neural mechanisms underlying broad training effects. Here, we tested a set of competitive hypotheses regarding the role of brain integration versus segregation underlying the broad training effect. We retrospectively analyzed data from a randomized controlled trial comparing neurocognitive effects of vision-based speed of processing training (VSOP) and an active control consisting of mental leisure activities (MLA) in older adults with MCI. We classified a subset of participants in the VSOP as learners, who showed improvement in executive function and episodic memory. The other participants in the VSOP (i.e., VSOP non-learners) and a subset of participants in the MLA (i.e., MLA non-learners) served as controls. Structural brain networks were constructed from diffusion tensor imaging. Clustering coefficients (CCs) and characteristic path lengths were computed as measures of segregation and integration, respectively. Learners showed significantly greater global CCs after intervention than controls. Nodal CCs were selectively enhanced in cingulate cortex, parietal regions, striatum, and thalamus. Among VSOP learners, those with more severe baseline neurodegeneration had greater improvement in segregation after training. Our findings suggest broad training effects are related to enhanced segregation in selective brain networks, providing insight into cognitive training related neuroplasticity.