Speech-in-noise (SIN) understanding often becomes difficult for older adults because of impaired hearing and aging-related changes in central auditory processing. Central auditory processing depends on a fine balance between excitatory and inhibitory neural mechanisms, which may be upset in older age by a change in the level of the inhibitory neurotransmitter gamma-aminobutyric acid (GABA). In this study, we used MEGA-PRESS magnetic resonance spectroscopy (MRS) to estimate GABA levels in both the left and right auditory cortices of young and older adults. We found that total auditory GABA levels were lower in older compared to young adults. To understand the relationship between GABA and hearing function, we correlated GABA levels with hearing loss and SIN performance. In older adults, the GABA level in the right auditory cortex was correlated with age and SIN performance. The relationship between chronological age and SIN loss was partially mediated by the GABA level in the right auditory cortex. These findings support the hypothesis that inhibitory mechanisms in the auditory system are reduced in aging, and this reduction relates to functional impairments.