Effective cognitive performance often requires the allocation of additional neural resources (i.e. blood-oxygen-level-dependent [BOLD] activation) as task demands increase, and this demand-related modulation is affected by amyloid-beta deposition and normal aging. The present study investigated these complex relationships between amyloid, modulation, and cognitive function (i.e. fluid ability). Participants from the Dallas Lifespan Brain Study (DLBS, n = 252, ages 50-89) completed a semantic judgment task during functional magnetic resonance imaging (fMRI) where the judgments differed in classification difficulty. Amyloid burden was assessed via positron emission tomography (PET) using 18F-florbetapir. A quadratic relationship between amyloid standardized value uptake ratios (SUVRs) and BOLD modulation was observed such that modulation was weaker in those with moderately elevated SUVRs (e.g. just reaching amyloid-positivity), whereas those with very high SUVRs (e.g. SUVR > 1.5) showed strong modulation. Greater modulation was related to better fluid ability, and this relationship was strongest in younger participants and those with lower amyloid burden. These results support the theory that effective demand-related modulation contributes to healthy cognitive aging, especially in the transition from middle age to older adulthood, whereas high modulation may be dysfunctional in those with substantial amyloid deposition.