Mild cognitive impairment (MCI) is recognized as the prodromal phase of dementia, a condition that can be either maintained or reversed through timely medical interventions to prevent cognitive decline. Considerable studies using functional magnetic resonance imaging (fMRI) have indicated that altered activity in the medial prefrontal cortex (mPFC) serves as an indicator of various cognitive stages of aging. However, the impacts of intrinsic functional connectivity in the mPFC as a mediator on cognitive performance in individuals with and without MCI have not been fully understood. In this study, we recruited 42 MCI patients and 57 healthy controls, assessing their cognitive abilities and functional brain connectivity patterns through neuropsychological evaluations and resting-state fMRI, respectively. The MCI patients exhibited poorer performance on multiple neuropsychological tests compared to the healthy controls. At the neural level, functional connectivity between the mPFC and the anterior cingulate cortex (ACC) was significantly weaker in the MCI group and correlated with multiple neuropsychological test scores. The result of the mediation analysis further demonstrated that functional connectivity between the mPFC and ACC notably mediated the relationship between the MCI and semantic fluency performance. These findings suggest that altered mPFC-ACC connectivity may have a plausible causal influence on cognitive decline and provide implications for early identifications of neurodegenerative diseases and precise monitoring of disease progression.