Aging listeners often experience difficulties in perceiving temporally complex acoustic cues in noisy environments. These difficulties likely have neurophysiological contributors from various levels of auditory processing. Cochlear synapses between inner hair cells and auditory nerve fibers exhibit a progressive decline with age which is not reflected in the threshold audiogram. The functional consequences of this loss for the coding of suprathreshold sound remain poorly understood. Recent studies suggest that cochlear synaptopathy results in degraded representations of temporal envelope cues at the earliest levels of the auditory pathway. Central nuclei downstream of the auditory nerve exhibit a compensatory plasticity in response to this deafferentation, in the form of altered gain. This results in a modulation frequency selective increase in the representation of envelope cues at the level of the auditory midbrain and cortex. These changes may be shaped by mechanisms such as decreased inhibitory neurotransmission occurring with age across various central auditory nuclei. Altered representations of the differing temporal components of speech due to these interactions between multiple levels of the auditory pathway may contribute to the age-related difficulties hearing speech in noisy environments.