Contingency tests of neutrality using intra/interspecific gene trees: the rejection of neutrality for the evolution of the mitochondrial cytochrome oxidase II gene in the hominoid primates.

Contingency tests of neutrality are performed using mitochondrial cytochrome oxidase II (COII) DNA sequences from hominoid primates, including humans. An intra-/interspecific haplotype tree is estimated, including a statistical assessment of ambiguities in tree topology and branch lengths. Four functional mutational categories are considered: silent and replacement substitutions in the transmembrane portion of the COII molecule, and silent and replacement substitutions in the cytosolic portion. Three tree topological mutational categories are used: intraspecific tips, intraspecific interiors, and interspecific fixed mutations. A full contingency analysis is performed, followed by nested contingency analyses. The analyses indicate that replacement mutations in the cytosolic portion are deleterious, and replacement mutations in the transmembrane portion and silent mutations throughout tend to be neutral. These conclusions are robust to ambiguities in tree topology and branch lengths. These inferences would have been impossible with an analysis that only contrasts silent and replacement vs. polymorphic and fixed. Also, intraspecific interior mutations have similar evolutionary dynamics to fixed mutations, so pooling tip and interior mutations into a single "polymorphic" class reduces power. Finally, the detected deleterious selection causes lowered inbreeding effective sizes, so arguments for small effective sizes in recent human evolutionary history based upon mitochondrial DNA may be invalid.