This narrative review follows the JBI approach and comprehensively explores the effects and mechanisms of acute exercise on cognitive function in Alzheimer's disease (AD) and Mild cognitive impairment (MCI) patients. The results showed that the combination of acute exercise and cognitive training improved the cognitive function of AD patients better than aerobic exercise or resistance training alone. For patients with MCI, moderate intensity acute aerobic exercise and resistance exercise were beneficial to enhance Inhibitory control (IC), but high-intensity acute exercise was adverse to improve IC; Brain-derived neurotrophic factor (BDNF) and Insulin-like growth factor 1 (IGF-1) may assume the potential mediating mechanism of acute exercise on cognitive function in AD and MCI patients, but more research is needed to further confirm this mechanism.