The four mammalian Argonaute family members are thought to share redundant functions in the microRNA pathway, yet only AGO2 possesses the catalytic "slicer" function required for RNAi. Whether AGO1, AGO3, or AGO4 possesses specialized functions remains unclear. Here we show that AGO4 localizes to spermatocyte nuclei during meiotic prophase I, specifically at sites of asynapsis and the transcriptionally silenced XY subdomain, the sex body. We generated Ago4 knockout mice and show that Ago4(-/-) spermatogonia initiate meiosis early, resulting from premature induction of retinoic acid-response genes. During prophase I, the sex body assembles incorrectly in Ago4(-/-) mice, leading to disrupted meiotic sex chromosome inactivation (MSCI). This is associated with a dramatic loss of microRNAs, >20% of which arises from the X chromosome. Thus, AGO4 regulates meiotic entry and MSCI in mammalian germ cells, implicating small RNA pathways in these processes.