The primary cilium is a microtubule-based sensory cell organelle templated by a modified parent centriole that mediates mechanotransduction and response to biochemical cues such as morphogens to regulate organismal development and homeostasis. Given that the cilium is a specialized microdomain devoid of its translation machinery, it relies on the endomembrane pathway for the delivery of proteins and other biomolecules to it. This review provides a comprehensive insight into how membrane trafficking modulators such as Rab and Rab-like proteins, and the exocyst complex control transport to the primary cilia, in turn regulating various aspects of their assembly and function. We integrate findings from and animal models and draw on human diseases associated with the dysfunction of Rabs or exocyst that exhibit phenotypes overlapping with those of ciliopathies, which further support their relevance to cilia biogenesis and maintenance.