The origins of agriculture were key events in human history, during which people came to depend for their food on small numbers of animal and plant species. However, the biological traits determining which species were domesticated for food provision, and which were not, are unclear. Here, we investigate the phylogenetic distribution of livestock and crops, and compare their phenotypic traits with those of wild species. Our results indicate that phylogenetic clustering is modest for crop species but more intense for livestock. Domesticated species explore a reduced portion of the phenotypic space occupied by their wild counterparts and have particular traits in common. For example, herbaceous crops are globally characterized by traits including high leaf nitrogen concentration and tall canopies, which make them fast-growing species and proficient competitors. Livestock species are relatively large mammals with low basal metabolic rates, which indicate moderate to slow life histories. Our study therefore reveals ecological differences in domestication potential between plants and mammals. Domesticated plants belong to clades with traits that are advantageous in intensively managed high-resource habitats, whereas domesticated mammals are from clades adapted to moderately productive environments. Combining comparative phylogenetic methods with ecologically relevant traits has proven useful to unravel the causes and consequences of domestication.