The global decline of coral reefs heightens the need to understand how corals respond to changing environmental conditions. Corals are metaorganisms, so-called holobionts, and restructuring of the associated bacterial community has been suggested as a means of holobiont adaptation. However, the potential for restructuring of bacterial communities across coral species in different environments has not been systematically investigated. Here we show that bacterial community structure responds in a coral host-specific manner upon cross-transplantation between reef sites with differing levels of anthropogenic impact. The coral Acropora hemprichii harbors a highly flexible microbiome that differs between each level of anthropogenic impact to which the corals had been transplanted. In contrast, the microbiome of the coral Pocillopora verrucosa remains remarkably stable. Interestingly, upon cross-transplantation to unaffected sites, we find that microbiomes become indistinguishable from back-transplanted controls, suggesting the ability of microbiomes to recover. It remains unclear whether differences to associate with bacteria flexibly reflects different holobiont adaptation mechanisms to respond to environmental change.