More than 60 highly conserved proteins appear sequentially at sites of clathrin-mediated endocytosis in yeast and mammals. The yeast Eps15-related proteins Pan1 and End3 and the CIN85-related protein Sla1 are known to interact with each other in vitro, and they all appear after endocytic-site initiation but before endocytic actin assembly, which facilitates membrane invagination/scission. Here we used live-cell imaging in parallel with genetics and biochemistry to explore comprehensively the dynamic interactions and functions of Pan1, End3, and Sla1. Our results indicate that Pan1 and End3 associate in a stable manner and appear at endocytic sites before Sla1. The End3 C-terminus is necessary and sufficient for its cortical localization via interaction with Pan1, whereas the End3 N-terminus plays a crucial role in Sla1 recruitment. We systematically examined the dynamic behaviors of endocytic proteins in cells in which Pan1 and End3 were simultaneously eliminated, using the auxin-inducible degron system. The results lead us to propose that endocytic-site initiation and actin assembly are separable processes linked by a Pan1/End3/Sla1 complex. Finally, our study provides mechanistic insights into how Pan1 and End3 function with Sla1 to coordinate cargo capture with actin assembly.