Early-life experiences with signals used in communication are instrumental in shaping an animal's social interactions. In songbirds, which use vocalizations for guiding social interactions and mate choice, recent studies show that sensory effects on development occur earlier than previously expected, even in embryos and nestlings. Here, we explored the neural dynamics underlying experience-dependent song categorization in young birds prior to the traditionally studied sensitive period of vocal learning that begins around 3 weeks post-hatch. We raised zebra finches either with their biological parents, cross-fostered by Bengalese finches beginning at embryonic day 9, or with only the non-singing mother from 2 days post-hatch. Then, 1-5 days after fledging, we conducted behavioral experiments and extracellular recordings in the auditory forebrain to test responses to zebra finch and Bengalese finch songs. Auditory forebrain neurons in cross-fostered and isolated birds showed increases in firing rate and decreases in responsiveness and selectivity. In cross-fostered birds, decreases in responsiveness and selectivity relative to white noise were specific to conspecific song stimuli, which paralleled behavioral attentiveness to conspecific songs in those same birds. This study shows that auditory and social experience can already impact song 'type' processing in the brains of nestlings, and that brain changes at this age can portend the effects of natal experience in adults.