Membrane compartments are amongst the most fascinating markers of cell evolution from prokaryotes to eukaryotes, some being conserved and the others having emerged via a series of primary and secondary endosymbiosis events. Membrane compartments comprise the system limiting cells (one or two membranes in bacteria, a unique plasma membrane in eukaryotes) and a variety of internal vesicular, subspherical, tubular, or reticulated organelles. In eukaryotes, the internal membranes comprise on the one hand the general endomembrane system, a dynamic network including organelles like the endoplasmic reticulum, the Golgi apparatus, the nuclear envelope, etc. and also the plasma membrane, which are linked via direct lateral connectivity (e.g. between the endoplasmic reticulum and the nuclear outer envelope membrane) or indirectly via vesicular trafficking. On the other hand, semi-autonomous organelles, i.e. mitochondria and chloroplasts, are disconnected from the endomembrane system and request vertical transmission following cell division. Membranes are organized as lipid bilayers in which proteins are embedded. The budding of some of these membranes, leading to the formation of the so-called lipid droplets (LDs) loaded with hydrophobic molecules, most notably triacylglycerol, is conserved in all clades. The evolution of eukaryotes is marked by the acquisition of mitochondria and simple plastids from Gram-positive bacteria by primary endosymbiosis events and the emergence of extremely complex plastids, collectively called secondary plastids, bounded by three to four membranes, following multiple and independent secondary endosymbiosis events. There is currently no consensus view of the evolution of LDs in the Tree of Life. Some features are conserved; others show a striking level of diversification. Here, we summarize the current knowledge on the architecture, dynamics, and multitude of functions of the lipid droplets in prokaryotes and in eukaryotes deriving from primary and secondary endosymbiosis events.