Parkinson's disease (PD) is a progressive neurodegenerative disease characterized by both motor and non-motor symptoms. A convergent pathophysiological hallmark of PD is an early selective vulnerability within the basal ganglia circuit. However, the causal interactions between basal ganglia atrophy and progressive structural network alterations in PD remain unaddressed. Here, we adopted voxel-based morphometry method to measure gray matter (GM) volume for each participant (n = 84 PD patients and n = 70 matched healthy controls). Patients were first divided into three stages according to the Hoehn and Yahr (H&Y) and the Part III of Unified Parkinson's Disease Rating Scale scores respectively to analyze the stage-specific GM atrophy patterns. Then, the modulation of early caudate atrophy over other brain structures was evaluated using the whole-brain voxel-wise and region-of-interest-wise causal structural covariance network approaches. We found that GM atrophy progressively expands from the basal ganglia to the angular gyrus, temporal areas, and eventually spreads through the subcortical-cortical networks as PD progresses. Notably, we identified a shared caudate-associated degeneration network including the basal ganglia, thalamus, cerebellum, sensorimotor cortex, and cortical association areas with the PD progressive factors. These findings suggest that the early structural vulnerability of basal ganglia in PD may play a pivotal role in the modulation of motor and non-motor circuits at the structural level. Our work provides evidence for a novel mechanism of network degeneration that underlies the pathology of PD and may have potential clinical applications in the development of early predictors of PD onset and progress.