Inflammatory bowel diseases (IBDs) develop in genetically predisposed individuals in response to environmental factors. IBDs are concomitant conditions of industrialized societies, and diet is a potential culprit. Consumption of ultra-processed food has increased over the last decade in industrialized countries, and epidemiological studies have found associations between ultra-processed food consumption and chronic diseases. Further studies are now required to identify the potential culprit in ultra-processed food, such as a poor nutritional composition or the presence of food additives. In our review, we will focus on food additives, i.e., substances from packaging in contact with food, and compounds formed during production, processing, and storage. A literature search using PubMed from inception to January 2019 was performed to identify relevant studies on diet and/or food additive and their role in IBDs. Manuscripts published in English from basic science, epidemiological studies, or clinical trials were selected and reviewed. We found numerous experimental studies highlighting the key role of food additives in IBD exacerbation but epidemiological studies on food additives on IBD risk are still limited. As diet is a modifiable environmental risk factor, this may offer a scientific rationale for providing dietary advice for IBD patients.