It is important to monitor body composition longitudinally, especially in children with atypical body composition trajectories. Dual-energy X-ray absorptiometry (DXA) can be used and reference values are available. Air-displacement plethysmography (ADP) is a relatively new technique, but reference values are lacking. In addition, estimates of fat-free mass density (Dffm), needed in ADP calculations, are based on children aged >8 years and may not be valid for younger children. We, therefore, aimed to investigate whether DXA and ADP results were comparable in young children aged 3−5 years, either born full-term or preterm, and if Dffm estimates in the ADP algorithm could be improved. In 154 healthy children born full-term and 67 born < 30 weeks of the inverse pressure-volume gestation, aged 3−5 years, body composition was measured using ADP (BODPOD, with default Lohman Dffm estimates) and DXA (Lunar Prodigy). We compared fat mass (FM), fat mass percentage (FM%) and fat-free mass (FFM), between ADP and DXA using Bland−Altman analyses, in both groups. Using a 3-compartment model as reference method, we revised the Dffm estimates for ADP. In full-term-born children, Bland−Altman analyses showed considerable fixed and proportional bias for FM, FM%, and FFM. After revising the Dffm estimates, agreement between ADP and DXA improved, with mean differences (LoA) for FM, FM%, and FFM of −0.67 kg (−2.38; 1.04), −3.54% (−13.44; 6.36), and 0.5 kg (−1.30; 2.30), respectively, but a small fixed and proportional bias remained. The differences between ADP and DXA were larger in preterm-born children, even after revising Dffm estimates. So, despite revised and improved sex and age-specific Dffm estimates, results of ADP and DXA remained not comparable and should not be used interchangeably in the longitudinal assessment of body composition in children aged 3−5 years, and especially not in very preterm-born children of that age.