At the trans-Golgi network, clathrin coats containing AP-1 adaptor complexes are formed in an ARF1-dependent manner, generating vesicles transporting cargo proteins to endosomes. The mechanism of site-specific targeting of AP-1 and the role of cargo are poorly understood. We have developed an in vitro assay to study the recruitment of purified AP-1 adaptors to chemically defined liposomes presenting peptides corresponding to tyrosine-based sorting motifs. AP-1 recruitment was found to be dependent on myristoylated ARF1, GTP or nonhydrolyzable GTP-analogs, tyrosine signals, and small amounts of phosphoinositides, most prominently phosphatidylinositol 4,5-bisphosphate, in the absence of any additional cytosolic or membrane bound proteins. AP-1 from cytosol could be recruited to a tyrosine signal independently of the lipid composition, but the rate of recruitment was increased by phosphatidylinositol 4,5-bisphosphate. The results thus indicate that cargo proteins are involved in coat recruitment and that the local lipid composition contributes to specifying the site of vesicle formation.