Poly(ADP-ribosyl)ation (PAR) is a versatile and complex posttranslational modification composed of repeating units of ADP-ribose arranged into linear or branched polymers. This scaffold is linked to the regulation of many of cellular processes including the DNA damage response, alteration of chromatin structure and Wnt signalling. Despite decades of research, the principles and mechanisms underlying all steps of PAR removal remain actively studied. In this work, we synthesise well-defined PAR branch point molecules and demonstrate that PARG, but not ARH3, can resolve this distinct PAR architecture. Structural analysis of ARH3 in complex with dimeric ADP-ribose as well as an ADP-ribosylated peptide reveal the molecular basis for the hydrolysis of linear and terminal ADP-ribose linkages. We find that ARH3-dependent hydrolysis requires both rearrangement of a catalytic glutamate and induction of an unusual, square-pyramidal magnesium coordination geometry.