The retrograde transport pathways from early/recycling endosomes are critical for recycling a range of endogenous cargo, as well as internalisation of bacterial and plant toxins. We have previously shown that the retrograde transport of the two model cargos, TGN38 and Shiga toxin, differs in the requirement for TGN golgins; transport of TGN38 requires the TGN golgin GCC88 whereas that of Shiga toxin requires GCC185. Here we have further defined the retrograde transport requirements of these two cargos. Tracking the transport of these cargos demonstrated that the bulk of Shiga toxin is transported from early endosomes to recycling endosomes en route to the TGN whereas the bulk of TGN38 is transported from early endosomes to the TGN with only low levels detected in recycling endosomes. In cells depleted of the TGN t-SNARE syntaxin 16, TGN38 accumulated predominantly in early endosomes whereas Shiga toxin accumulated in Rab11-positive recycling endosomes, suggesting distinct routes for each cargo. Retrograde transport of Shiga toxin and TGN38 requires retromer, however, whereas sorting nexin 1 (SNX1) is specifically required for transport of Shiga toxin, sorting nexin 2 (SNX2) is required for the transport of TGN38. Overall, our data have identified different itineraries for the retrograde transport of Shiga toxin and TGN38 and distinct retromer components that regulate the transport of these cargos.