Repressive H3K27me3 and active H3K4me2/3 together form bivalent chromatin domains, molecular hallmarks of developmental potential. In the male germline, these domains are thought to persist into sperm to establish totipotency in the next generation. However, it remains unknown how H3K27me3 is established on specific targets in the male germline. Here, we demonstrate that a germline-specific Polycomb protein, SCML2, binds to H3K4me2/3-rich hypomethylated promoters in undifferentiated spermatogonia to facilitate H3K27me3. Thus, SCML2 establishes bivalent domains in the male germline of mice. SCML2 regulates two major classes of bivalent domains: Class I domains are established on developmental regulator genes that are silent throughout spermatogenesis, while class II domains are established on somatic genes silenced during late spermatogenesis. We propose that SCML2-dependent H3K27me3 in the male germline prepares the expression of developmental regulator and somatic genes in embryonic development.