Glycogen branching enzyme (GBE) catalyzes the formation of α-1,6-branching points during glycogenesis by cleaving α-1,4 bonds and making new α-1,6 bonds. Most GBEs belong to the glycoside hydrolase 13 family (GH13), but new GBEs in the GH57 family have been isolated from Archaea. Here, we determined the crystal structure of a GH57 GBE from the hyperthermophilic archaeon Pyrococcus horikoshii (PhGBE) at a resolution of 2.3 Å. PhGBE exhibits both α-1,6-branching activity and endo-α-1,4 hydrolytic activity. PhGBE has a central (β/α)-barrel domain that contains an embedded helix domain and an α-helix-rich C-terminal domain. The active-site cleft is located at the interface of the central and C-terminal domains. Amino acid substitution at Trp22, which is separate from the catalytic nucleophilic residue, abolished both enzymatic activities, indicating that Trp22 might be responsible for substrate recognition. We also observed that shortening of the flexible loop near the catalytic residue changed branched chain lengths of the reaction products with increased hydrolytic activity. Taken together, our findings propose a molecular mechanism for how GH57 GBEs exhibit the two activities and where the substrate binds the enzyme.