Inferring a major gene for quantitative traits by using segregation analysis with tests on transmission probabilities: how often do we miss?

In an effort to safeguard against false inference of a major gene in segregation analysis, it has become common practice to require nonrejection of the Mendelian-transmission hypothesis (Mendelian tau's) and rejection of the no-transmission hypothesis (equal tau's). However, it is not known how often one would actually infer a major gene, when one exists, by using these criteria. A simulation study was undertaken to investigate this issue. Segregation of a Mendelian gene under a variety of models was simulated in families with both parents and three children. The data were analyzed by using POINTER; the assumptions under the generating and analysis models were identical. By design, the power to reject the no-major-effect hypothesis (q = 0) was > 60% for all models considered; tests on the transmission probabilities were carried out only when q = 0 was rejected, using alpha = 0.05 for all tests. The rates of Mendelian inference were mostly in the range of 22%-50% under recessive inheritance, versus 60%-99% under dominant inheritance. Notably, it was not possible to resolve the transmission (from among Mendelian tau's, equal tau's, and general unconstrained tau's) in approximately 20%-70% of the cases under recessive models, versus 3%-15% under dominant models. Therefore, while tests on transmission probabilities can serve to reduce rates of false inference of a major gene, it is also possible to fail to infer a major gene when one indeed exists, especially under recessive inheritance.