During the early stages of local adaptation and speciation, genetic differences tend to accumulate at certain regions of the genome leading to the formation of genomic islands of divergence (GIDs). This pattern may be due to selection and/or difference in the rate of recombination. Here, we investigate the possible causes of GIDs in Drosophila yakuba mayottensis, and reconfirm using field collection its association with toxic noni (Morinda citrifolia) fruits on the Mayotte island. Population genomics revealed lack of genetic structure on the island and identified 23 GIDs distinguishing D. y. mayottensis from generalist mainland populations of D. y. yakuba. The GIDs were enriched with gene families involved in the metabolism of lipids, sugars, peptides and xenobiotics, suggesting a role in host shift. We assembled a new genome for D. y. mayottensis and identified five novel chromosomal inversions. Twenty one GIDs (~99% of outlier windows) fell in low recombining regions or subspecies-specific inversions. However, only two GIDs were in collinear, normally recombining regions suggesting a signal of hard selective sweeps. Unlike D. y. mayottensis, D. sechellia, the only other noni-specialist, is known to be homosequential with its generalist relatives. Thus, whereas structural variation may disproportionally shape GIDs in some species, striking parallel adaptations can occur between species despite distinct genomic architectures.