O-GlcNAcylation is an essential protein glycosylation governed by two O-GlcNAc cycling enzymes: O-GlcNAc transferase (OGT) installs a single sugar moiety N-acetylglucosamine (GlcNAc) on protein serine and threonine residues, and O-GlcNAcase (OGA) removes them. Aberrant O-GlcNAcylation has been implicated in various diseases. However, the large repertoire of more than 1000 O-GlcNAcylated proteins and the elusive mechanisms of OGT/OGA in substrate recognition present significant challenges in targeting the dysregulated O-GlcNAcylation for therapeutic development. Recently, emerging evidence suggested that the non-catalytic domains play critical roles in regulating the functional specificity of OGT/OGA via modulating their protein interactions and substrate recognition. Here, we discuss recent studies on the structures, mechanisms, and related tools of the OGT/OGA non-catalytic domains, highlighting new opportunities for function-specific control.