The first 1000 days of life lay the foundations for subsequent growth. This ambidirectional study, including prenatal, perinatal and postnatal factors, aimed to identify exposure variables affecting body size and composition and corresponding Z-score outcomes at age 18 months in infants born to women at low risk of adverse pregnancy outcomes in a peri-urban area of South Africa. Prenatal factors (maternal age, HIV status, anthropometry, parity, food insecurity and umbilical artery resistance index Z-score (UmA-RIAZ) as a measure of placental function, with higher UmA-RIAZ indicating poorer placental function); perinatal factors (infant sex, gestational age and birth anthropometry) and postnatal factors (infant feeding) were included as exposure variables, with infant anthropometry and body composition at 18 months as outcomes. Simple linear regression analysis was used to investigate associations between exposure variables and infant outcomes, and variables with p < 0.10 were included in the subsequent multiple regression analyses. Multiple regression analysis showed that higher UmA-RIAZ predicted lower birthweight [-0.11 kg (95% CI: -0.17, -0.04 kg)], birthweight-for-age Z-score [-0.24 (95% CI: -0.39, -0.09)] and 18-month infant length [-0.9 cm (95% CI: -1.4, -0.4 cm)] and length-for-age Z-score [-0.28 (95% CI: -0.45, -0.11)]. Maternal HIV infection predicted reduced 18-month infant length-for-age Z-score [-0.46 (95% CI: -0.83, -0.09)]. Household food insecurity predicted reduced fat-free mass-for-age Z-score at 18 months [-0.27 (95% CI: -0.51, -0.03)]. Infant anthropometry and body composition outcomes, therefore, are greatly affected by pre- and postnatal nutrition-related factors, such as placental insufficiency in utero and household food insecurity, with long-term consequences including stunting, which impact the individual, future generations and society.