ASH1L histone methyltransferase plays a crucial role in the pathogenesis of different diseases, including acute leukemia. While ASH1L represents an attractive drug target, developing ASH1L inhibitors is challenging, as the catalytic SET domain adapts an inactive conformation with autoinhibitory loop blocking the access to the active site. Here, by applying fragment-based screening followed by medicinal chemistry and a structure-based design, we developed first-in-class small molecule inhibitors of the ASH1L SET domain. The crystal structures of ASH1L-inhibitor complexes reveal compound binding to the autoinhibitory loop region in the SET domain. When tested in MLL leukemia models, our lead compound, AS-99, blocks cell proliferation, induces apoptosis and differentiation, downregulates MLL fusion target genes, and reduces the leukemia burden in vivo. This work validates the ASH1L SET domain as a druggable target and provides a chemical probe to further study the biological functions of ASH1L as well as to develop therapeutic agents.