The horizontal direction of a sound source (i.e., azimuth) is perceptually determined in a frequency-dependent manner: low- and high-frequency sounds are localized via differences in the arrival time and intensity of the sound at the two ears, respectively, called interaural time and level differences (ITDs and ILDs). In the central auditory system, these binaural cues to direction are thought to be separately encoded by neurons tuned to low and high characteristic frequencies (CFs). However, at high sound levels a neuron often responds to frequencies far from its CF, raising the possibility that individual neurons may encode the azimuths of both low- and high-frequency sounds using both binaural cues. We tested this possibility by measuring auditory-driven single-unit responses in the central nucleus of the inferior colliculus (ICC) of unanesthetized female Dutch Belted rabbits with a multitetrode drive. At 70 dB SPL, ICC neurons across the cochleotopic map transmitted information in their firing rates about the direction of both low- and high-frequency noise stimuli. We independently manipulated ITD and ILD cues in virtual acoustic space and found that sensitivity to ITD and ILD, respectively, shaped the directional sensitivity of ICC neurons to low (<1.5 kHz)- and high (>3 kHz)-pass stimuli, regardless of the neuron's CF. We also found evidence that high-CF neurons transmit information about both the fine-structure and envelope ITD of low-frequency sound. Our results indicate that at conversational sound levels the majority of the cochleotopic map is engaged in transmitting directional information, even for sources with narrowband spectra. A "division of labor" has previously been assumed in which the directions of low- and high-frequency sound sources are thought to be encoded by neurons preferentially sensitive to low and high frequencies, respectively. Contrary to this, we found that auditory midbrain neurons encode the directions of both low- and high-frequency sounds regardless of their preferred frequencies. Neural responses were shaped by different sound localization cues depending on the stimulus spectrum-even within the same neuron.