By consuming ammonium and nitrite, anammox bacteria form an important functional guild in nitrogen cycling in many environments, including marine sediments. However, their distribution and impact on the important substrate nitrite has not been well characterized. Here we combined biogeochemical, microbiological, and genomic approaches to study anammox bacteria and other nitrogen cycling groups in two sediment cores retrieved from the Arctic Mid-Ocean Ridge (AMOR). We observed nitrite accumulation in these cores, a phenomenon also recorded at 28 other marine sediment sites and in analogous aquatic environments. The nitrite maximum coincides with reduced abundance of anammox bacteria. Anammox bacterial abundances were at least one order of magnitude higher than those of nitrite reducers and the anammox abundance maxima were detected in the layers above and below the nitrite maximum. Nitrite accumulation in the two AMOR cores co-occurs with a niche partitioning between two anammox bacterial families (Candidatus Bathyanammoxibiaceae and Candidatus Scalinduaceae), likely dependent on ammonium availability. Through reconstructing and comparing the dominant anammox genomes (Ca. Bathyanammoxibius amoris and Ca. Scalindua sediminis), we revealed that Ca. B. amoris has fewer high-affinity ammonium transporters than Ca. S. sediminis and lacks the capacity to access alternative substrates and/or energy sources such as urea and cyanate. These features may restrict Ca. Bathyanammoxibiaceae to conditions of higher ammonium concentrations. These findings improve our understanding about nitrogen cycling in marine sediments by revealing coincident nitrite accumulation and niche partitioning of anammox bacteria.