The Social Determinants of Health, a set of social factors including socioeconomic status, community context, and neighborhood safety among others, are well-known predictors of mental and physical health across the lifespan. Recent research has begun to establish the importance of these social factors at the earliest points of brain development, including during the prenatal period. Prenatal socioeconomic status, perceived stress, and neighborhood safety have all been reported to impact neonatal brain structure and function, with exploratory work suggesting subsequent effects on infant and child behavior. Secondary effects of the Social Determinants of Health, such as maternal sleep and psychopathology during pregnancy, have also been established as important predictors of infant brain development. This research not only establishes prenatal Social Determinants of Health as important predictors of future outcomes but may be effectively applied even before birth. Future research replicating and extending the effects in this nascent literature has great potential to produce more specific and mechanistic understanding of the social factors that shape early neurobehavioral development. IMPACT: This review synthesizes the research to date examining the effects of the Social Determinants of Health during the prenatal period and neonatal brain outcomes. Structural, functional, and diffusion-based imaging methodologies are included along with the limited literature assessing subsequent infant behavior. The degree to which results converge between studies is discussed, in combination with the methodological and sampling considerations that may contribute to divergence in study results. Several future directions are identified, including new theoretical approaches to assessing the impact of the Social Determinants of Health during the perinatal period.