A family history (FH) of depression significantly influences the progress of major depressive disorder (MDD). However, the underlying neural mechanism of FH remains unclear. This study examined the association between brain structural and connectivity alterations, inflammation, and FH in MDD. A total of 134 MDD patients with (FH group, n = 43) and without (NFH group, n = 91) first-degree FH and 96 demographic-matched healthy controls (HCs) were recruited. Voxel-based morphometry (VBM) and sliding-window dynamic functional connectivity (dFC) analyses were performed, and inflammatory biomarkers (C-reactive protein (CRP) and interleukin-6 (IL-6)) were detected. Compared with HCs, FH and NFH groups showed decreased gray matter volume (GMV) in the left cerebellum posterior lobe and increased dFC between this region and the left inferior parietal lobule. The FH group showed increased dFC between the cerebellum region and medial prefrontal cortex (mPFC) compared to NFH and HCs. The combination of these brain measurements further differentiated between FH and NFH. Moreover, the GMV of the cerebellum was positively correlated with CRP in the NFH group, while the dFC between the cerebellum and mPFC was positively correlated with IL-6 in the FH group. The present findings indicate that cerebellar structure and dynamic function vary according to FH of MDD and are related to inflammatory factors, potentially offering novel insights into the underlying pathogenic mechanisms of MDD.