Visual Hallucinations (VH) are among the core features of Dementia with Lewy Bodies (DLB), but are also very frequent in demented patients with Parkinson's Disease (PDD). The purpose of this study was to investigate the pattern of gray matter and cognitive impairment underlying VH in DLB and PDD. We applied voxel-based morphometry and behavioral assessment to 12 clinically diagnosed DLB patients and 15 PDD patients. Subjects with VH showed greater gray matter loss than non-hallucinators, specifically in the right inferior frontal gyrus (BA 45) in the DLB patients and in the left orbitofrontal lobe (BA 10) in the PDD patients. Comparing the two subgroups with VH, DLB patients had greater decrease of the bilateral premotor area (BA 6) than PDD patients. Furthermore, decreased volume in associative visual areas, namely left precuneus and inferior frontal lobe, correlated with VH in the DLB but not in PDD patients. VH were related to impaired verbal fluency, inhibitory control of attention and visuoperception in the DLB group and to visual memory in the PDD group. In conclusion, DLB and PDD patients with VH had more frontal gray matter atrophy than non-hallucinators, the impairment being greater in the DLB group. The patterns of structural and functional correlations were different in both pathologies.