Disability in multiple sclerosis (MS) patients is associated with white matter (WM) and gray matter (GM) pathology, and both processes contribute differently over the disease course. Total and regional GM volume loss can be imaged via voxel-based morphometry (VBM). Here, we retrospectively analyzed a group of 213 MS patients [163 relapsing remitting (RR) and 50 secondary progressive (SP)] using semi-automated white matter (WM) lesion mapping and voxel-based morphometry (VBM). Our aim was to assess the association of increasing disability with decreasing total and regional GM volume. As expected, total GM volume and WM lesion load were associated with patients disability, measured with the Expanded Disability Status Scale (EDSS). The more impaired the patients, the greater the statistical association to the total GM volume. Regional volume loss in the cerebellar gray matter was associated with increasing EDSS and WM lesion volume. Furthermore, SPMS patients had significantly more gray matter volume loss in the cerebellum and the hippocampus compared to RRMS patients. Our results confirm histopathological studies emphasizing the important role of the cerebellum and the hippocampus in MS patients' disability.