Interactions between the immune system and the microbiome play a crucial role on the human health. These interactions start in the prenatal period and are critical for the maturation of the immune system in newborns and infants. Several factors influence the composition of the infant's microbiota and subsequently the development of the immune system. They include maternal infection, antibiotic treatment, environmental exposure, mode of delivery, breastfeeding, and food introduction. In this review, we focus on the ontogeny of the immune system and its association to microbial colonization from conception to food diversification. In this context, we give an overview of the mother-fetus interactions during pregnancy, the impact of the time of birth and the mode of delivery, the neonate gastrointestinal colonization and the role of breastfeeding, weaning, and food diversification. We further review the impact of the vaccination on the infant's microbiota and the reciprocal case. Finally, we discuss several potential therapeutic interventions that might help to improve the newborn and infant's health and their responses to vaccination. Throughout the review, we underline the main scientific questions that are left to be answered and how the non-human primate model could help enlighten the path.