Early life stress (ELS) is defined as an acute or chronic stressor that negatively impacts a child's development. ELS is associated with substance use and mental health problems. This narrative literature review focuses on sex and gender differences in the effects of ELS on 1) adolescent neuroendocrine development; 2) pubertal brain maturation; and 3) development of internalizing symptoms and subsequent substance use. We posit that ELS may generate larger hormonal dysregulation in females than males during puberty, increasing internalizing symptoms and substance use. Future research should consider sex and gender differences in neuroendocrine developmental processes when studying the link between ELS and negative health outcomes.