Group selection for a polygenic behavioral trait: estimating the degree of population subdivision.

For assessing the degree of population subdivision, and therefore the extent to which group selection might favor an altruistic trait, an appropriate measure is Nei's GST, defined by (F0-F)/(1-F). F0 is the probability that two alleles drawn from the same group are identical in state and F is the probability for two alleles drawn at random from the entire population. These probabilities can be assessed from molecular polymorphisms. GST has a number of properties that make it useful for empirical studies. When the mutation rate is small relative to the migration rate and the reciprocal of the group size, GST depends mainly on the absolute number of migrants per generation, moves rapidly to near equilibrium, and is independent of the number of alleles. The relative homogenizing effect of migration in the island and stepping-stone models is not as different as might be expected; one immigrant chosen randomly from the rest of the population is only one to two times as effective as one from a neighboring group, appreciably exceeding 2 only when there are 1000 or more groups. The use of molecular data to estimate the degree of population subdivision may permit testable predictions of the extent of altruistic behavior.