Oysters play an important role in coastal ecology and are a globally popular seafood source. However, their filter-feeding lifestyle enables coastal pathogens, toxins, and pollutants to accumulate in their tissues, potentially endangering human health. While pathogen concentrations in coastal waters are often linked to environmental conditions and runoff events, these do not always correlate with pathogen concentrations in oysters. Additional factors related to the microbial ecology of pathogenic bacteria and their relationship with oyster hosts likely play a role in accumulation but are poorly understood. In this study, we investigated whether microbial communities in water and oysters were linked to accumulation of Vibrio parahaemolyticus, Vibrio vulnificus, or fecal indicator bacteria. Site-specific environmental conditions significantly influenced microbial communities and potential pathogen concentrations in water. Oyster microbial communities, however, exhibited less variability in microbial community diversity and accumulation of target bacteria overall and were less impacted by environmental differences between sites. Instead, changes in specific microbial taxa in oyster and water samples, particularly in oyster digestive glands, were linked to elevated levels of potential pathogens. For example, increased levels of V. parahaemolyticus were associated with higher relative abundances of cyanobacteria, which could represent an environmental vector for spp. transport, and with decreased relative abundance of and other key members of the oyster digestive gland microbiota. These findings suggest that host and microbial factors, in addition to environmental variables, may influence pathogen accumulation in oysters. Bacteria in the marine environment cause thousands of human illnesses annually. Bivalves are a popular seafood source and are important in coastal ecology, but their ability to concentrate pathogens from the water can cause human illness, threatening seafood safety and security. To predict and prevent disease, it is critical to understand what causes pathogenic bacteria to accumulate in bivalves. In this study, we examined how environmental factors and host and water microbial communities were linked to potential human pathogen accumulation in oysters. Oyster microbial communities were more stable than water communities, and both contained the highest concentrations of Vibrio parahaemolyticus at sites with warmer temperatures and lower salinities. High oyster V. parahaemolyticus concentrations corresponded with abundant cyanobacteria, a potential vector for transmission, and a decrease in potentially beneficial oyster microbes. Our study suggests that poorly understood factors, including host and water microbiota, likely play a role in pathogen distribution and pathogen transmission.