Sesame is an ancient oilseed crop with high oil content and quality. However, the evolutionary history and genetic mechanisms of its valuable agronomic traits remain unclear. Here, we report chromosome-scale genomes of cultivated sesame (Sesamum indicum L.) and six wild Sesamum species, representing all three karyotypes within this genus. Karyotyping and genome-based phylogenic analysis revealed the evolutionary route of Sesamum species from n = 13 to n = 16 and revealed that allotetraploidization occurred in the wild species Sesamum radiatum. Early divergence of the Sesamum genus (48.5-19.7 million years ago) during the Tertiary period and its ancient phylogenic position within eudicots were observed. Pan-genome analysis revealed 9164 core gene families in the 7 Sesamum species. These families are significantly enriched in various metabolic pathways, including fatty acid (FA) metabolism and FA biosynthesis. Structural variations in SiPT1 and SiDT1 within the phosphatidyl ethanolamine-binding protein gene family lead to the genomic evolution of plant-architecture and inflorescence-development phenotypes in Sesamum. A genome-wide association study (GWAS) of an interspecific population and genome comparisons revealed a long terminal repeat insertion and a sequence deletion in DIR genes of wild Sesamum angustifolium and cultivated sesame, respectively; both variations independently cause high susceptibility to Fusarium wilt disease. A GWAS of 560 sesame accessions combined with an overexpression study confirmed that the NAC1 and PPO genes play an important role in upregulating oil content of sesame. Our study provides high-quality genomic resources for cultivated and wild Sesamum species and insights that can improve molecular breeding strategies for sesame and other oilseed crops.