Sensory systems across the brain are specialized for their input, yet some principles of neural organization are conserved across modalities. The pattern of anatomical connections from the primate auditory cortex to the temporal, parietal, and prefrontal lobes suggests a possible division into dorsal and ventral auditory processing streams, with the dorsal stream originating from more caudal areas of the auditory cortex, and the ventral stream originating from more rostral areas. These streams are hypothesized to be analogous to the well-established dorsal and ventral streams of visual processing. In the visual system, the dorsal processing stream shows substantially faster neural response latencies than does the ventral stream. However, the relative timing of putative dorsal and ventral stream processing has yet to be explored in other sensory modalities. Here, we compare distributions of neural response latencies from 10 different areas of macaque auditory cortex, confirmed by individual anatomical reconstructions, to determine whether a similar timing advantage is found for the hypothesized dorsal auditory stream. Across three varieties of auditory stimuli (clicks, noise, and pure tones), we find that latencies increase with hierarchical level, as predicted by anatomical connectivity. Critically, we also find a pronounced timing differential along the caudal-to-rostral axis within the same hierarchical level, with caudal (dorsal stream) latencies being faster than rostral (ventral stream) latencies. This observed timing differential mirrors that found for the dorsal stream of the visual system, suggestive of a common timing advantage for the dorsal stream across sensory modalities.