The extreme diversity of the human immune system, forged and maintained throughout evolutionary history, provides a potent defense against opportunistic pathogens. At the same time, this immune variation is the substrate upon which a plethora of immune-associated diseases develop. Genetic analysis suggests that thousands of individually weak loci together drive up to half of the observed immune variation. Intense selection maintains this genetic diversity, even selecting for the introgressed Neanderthal or Denisovan alleles that have reintroduced variation lost during the out-of-Africa migration. Variations in age, sex, diet, environmental exposure, and microbiome each potentially explain the residual variation, with proof-of-concept studies demonstrating both plausible mechanisms and correlative associations. The confounding interaction of many of these variables currently makes it difficult to assign definitive contributions. Here, we review the current state of play in the field, identify the key unknowns in the causality of immune variation, and identify the multidisciplinary pathways toward an improved understanding.