The protein kinase mechanistic target of rapamycin complex 1 (mTORC1) is a master regulator of cell growth and proliferation, supporting anabolic reactions and inhibiting catabolic pathways like autophagy. Its hyperactivation is a frequent event in cancer promoting tumor cell proliferation. Several intracellular membrane-associated mTORC1 pools have been identified, linking its function to distinct subcellular localizations. Here, we characterize the N-terminal kinase-like protein SCYL1 as a Golgi-localized target through which mTORC1 controls organelle distribution and extracellular vesicle secretion in breast cancer cells. Under growth conditions, SCYL1 is phosphorylated by mTORC1 on Ser754, supporting Golgi localization. Upon mTORC1 inhibition, Ser754 dephosphorylation leads to SCYL1 displacement to endosomes. Peripheral, dephosphorylated SCYL1 causes Golgi enlargement, redistribution of early and late endosomes and increased extracellular vesicle release. Thus, the mTORC1-controlled phosphorylation status of SCYL1 is an important determinant regulating subcellular distribution and function of endolysosomal compartments. It may also explain the pathophysiology underlying human genetic diseases such as CALFAN syndrome, which is caused by loss-of-function of SCYL1.