If we restrict the use of Homo sapiens in the fossil record to specimens which share a significant number of derived features in the skeleton with extant H. sapiens, the origin of our species would be placed in the African late middle Pleistocene, based on fossils such as Omo Kibish 1, Herto 1 and 2, and the Levantine material from Skhul and Qafzeh. However, genetic data suggest that we and our sister species Homo neanderthalensis shared a last common ancestor in the middle Pleistocene approximately 400-700 ka, which is at least 200 000 years earlier than the species origin indicated from the fossils already mentioned. Thus, it is likely that the African fossil record will document early members of the sapiens lineage showing only some of the derived features of late members of the lineage. On that basis, I argue that human fossils such as those from Jebel Irhoud, Florisbad, Eliye Springs and Omo Kibish 2 do represent early members of the species, but variation across the African later middle Pleistocene/early Middle Stone Age fossils shows that there was not a simple linear progression towards later sapiens morphology, and there was chronological overlap between different 'archaic' and 'modern' morphs. Even in the late Pleistocene within and outside Africa, we find H. sapiens specimens which are clearly outside the range of Holocene members of the species, showing the complexity of recent human evolution. The impact on species recognition of late Pleistocene gene flow between the lineages of modern humans, Neanderthals and Denisovans is also discussed, and finally, I reconsider the nature of the middle Pleistocene ancestor of these lineages, based on recent morphological and genetic data.This article is part of the themed issue 'Major transitions in human evolution'.