Oxygen deficient zones (ODZs) are subsurface marine systems that harbour distinct microbial communities, including populations of the picocyanobacteria Prochlorococcus that can form a secondary chlorophyll maximum (SCM), and low-oxygen tolerant strains of the globally abundant heterotroph Pelagibacter (SAR11). Yet, the small labile molecules (metabolites) responsible for maintaining these ODZ communities are unknown. Here, we compared the metabolome of an ODZ to that of an oxygenated site by quantifying 87 metabolites across depth profiles in the eastern tropical North Pacific ODZ and the oxygenated waters of the North Pacific Gyre. Metabolomes were largely consistent between anoxic and oxic water columns. However, the osmolyte glycine betaine (GBT) was enriched in the oxycline and SCM of the ETNP, comprising as much as 1.2% of particulate organic carbon. Transcriptomes revealed two active GBT production pathways, glycine methylation (SDMT/bsmB) expressed by Prochlorococcus and choline oxidation (betB) expressed by Gammaproteobacteria. GBT consumption through demethylation involved diverse microbial taxa, with SAR11 contributing nearly half of the transcripts for the initial step of GBT demethylation (BHMT), which is predicted to convert GBT and homocysteine into dimethylglycine and methionine, a compound SAR11 cannot otherwise produce. Thus, GBT connects the metabolisms of the dominant phototroph and heterotroph in the oceans.