Social-emotional problems occurring early in life can place children at future risk of adverse health, social and economic outcomes. Determinants of social-emotional problems are multi-layered and originate from different contexts surrounding children, though few studies consider them simultaneously. We adopted a holistic approach by using Bronfenbrenner's process-person-context-time model as a structuring device. We aimed to assess what characteristics of families and children from pregnancy, over birth, and up to 3 years of age are associated with social-emotional problems in boys and girls. This study used regional data from the Salut Programme, a universal health promotion programme implemented in Antenatal and Child Health Care, and data from national Swedish registers. The study population included 6033 3-year-olds and their parents during the period 2010-2018. Distinct logistic regression models for boys and girls were used to assess associations between the family social context, parents' lifestyle, parent's mental health, children's birth characteristics, and indicators of proximal processes (the independent variables); and children's social-emotional problems as measured by the parent-completed Ages and Stages Questionnaire: Social-Emotional between 33 and 41 months of age (the outcome). Overall, a less favourable family social context, detrimental lifestyle of the parents during pregnancy, and parents' mental illness from pregnancy onwards were associated with higher odds of social-emotional problems in 3-year-olds. Higher screentime and infrequent shared book-reading were associated with higher odds of social-emotional problems. The multifaceted determinants of children's social-emotional problems imply that many diverse targets for intervention exist. Additionally, this study suggests that Bronfenbrenner's process-person-context-time theoretical framework could be relevant for public health research and policy.