Microbial and microeukaryotic communities are extremely abundant and diverse in soil habitats where they play critical roles in ecosystem functioning and services that are essential to soil health. Soil biodiversity is influenced by above-ground (vegetation) and below-ground factors (soil properties), which together create habitat-specific conditions. However, the compound effects of vegetation and soil properties on soil communities are less studied or often focused on one component of the soil biota. Here, we integrate metabarcoding (16S and 18S rRNA genes) and nematode morphology to assess the effects of habitat and soil properties shaping microbial and microeukaryotic communities as well as nematode-associated microbiomes. We show that both vegetation and soil properties (soil bulk density) were major factors structuring microbial and microeukaryotic communities in semi-arid soil habitats. Despite having lower nutrients and lower pH, denser soils displayed significantly higher alpha diversity than less dense soils across datasets. Nematode-associated microbiomes have lower microbial diversity, strongly differ from soil microbes and are more likely to respond to microscale variations among samples than to vegetation or soil bulk density. Consequently, different nematode lineages and trophic groups are likely to display similar associated microbiomes when sharing the same microhabitat. Different microbiome taxa were enriched within specific nematode lineages (e.g. Mycobacterium, Candidatus Cardinium) highlighting potentially new species-specific associations that may confer benefits to their soil nematode hosts. Our findings highlight the importance of exploring above- and below-ground effects to assess community structure in terrestrial habitats, and how fine-scale analyses are critical for understanding patterns of host-associated microbiomes.