Glycosylation reactions in the Golgi complex and the endoplasmic reticulum utilize nucleotide sugars as donors and produce inorganic phosphate (Pi) and acid (H) as byproducts. Here we show that homologs of mammalian XPR1 and TMEM165 (termed Erd1 and Gdt1) recycle luminal Pi and exchange luminal H for cytoplasmic Ca, respectively, thereby promoting growth of yeast cells in low Pi and low Ca environments. As expected for reversible H/Ca exchangers, Gdt1 also promoted growth in high Ca environments when the Golgi-localized V-ATPase was operational but had the opposite effect when the V-ATPase was eliminated. Gdt1 activities were negatively regulated by calcineurin signaling and by Erd1, which recycled the Pi byproduct of glycosylation reactions and prevented the loss of this nutrient to the environment via exocytosis. Thus, Erd1 transports Pi in the opposite direction from XPR1 and other EXS family proteins and facilitates byproduct removal from the Golgi complex together with Gdt1.