The objective of this study was to evaluate susceptibility changes caused by iron accumulation in cognitive normal (CN) elderly, those with amnestic mild cognitive impairment (aMCI), and those with early state AD, and to compare the findings with gray matter volume (GMV) changes caused by neuronal loss. The participants included 19 elderly CN, 19 aMCI, and 19 AD subjects. The voxel-based quantitative susceptibility map (QSM) and GMV in the brain were calculated and the differences of those insides were compared among the three groups. The differences of the QSM data and GMVs among the three groups were investigated by voxel-based and region of interest (ROI)-based comparisons using a one-way analysis of covariance (ANCOVA) test with the gender and age as covariates. Finally, a receiver-operating-characteristic (ROC) curve analysis was performed. The voxel-based results showed that QSM demonstrated more areas with significant difference between the CN and AD groups compared to GMV. GMVs were decreased, but QSM values were increased in aMCI and AD groups compared with the CN group. QSM better differentiated aMCI from CN than GMV in the precuneus and allocortex regions. In the accumulation regions of iron and amyloid β, QSM can be used to differentiate between CN and aMCI groups, indicating a useful an auxiliary imaging for early diagnosis of AD.