Human immunodeficiency virus 1 (HIV-1) infects T cells, macrophages and dendritic cells and can manipulate their cytoskeleton structures at multiple steps during its replication cycle. Based on pharmacological and genetic targeting of cytoskeleton modulators, new imaging approaches and primary cell culture models, important roles for actin and microtubules during entry and cell-to-cell transfer have been established. Virological synapses and actin-containing membrane extensions can mediate HIV-1 transfer from dendritic cells or macrophage cells to T cells and between T cells. We will review the role of the cytoskeleton in HIV-1 entry, cellular trafficking and cell-to-cell transfer between primary cells.