Bacteria acquire novel DNA through horizontal gene transfer (HGT), a process that enables an organism to rapidly adapt to changing environmental conditions, provides a competitive edge and potentially alters its relationship with its host. Although the HGT process is routinely exploited in laboratories, there is a surprising disconnect between what we know from laboratory experiments and what we know from natural environments, such as the human gut microbiome. Owing to a suite of newly available computational algorithms and experimental approaches, we have a broader understanding of the genes that are being transferred and are starting to understand the ecology of HGT in natural microbial communities. This Review focuses on these new technologies, the questions they can address and their limitations. As these methods are applied more broadly, we are beginning to recognize the full extent of HGT possible within a microbiome and the punctuated dynamics of HGT, specifically in response to external stimuli. Furthermore, we are better characterizing the complex selective pressures on mobile genetic elements and the mechanisms by which they interact with the bacterial host genome.