Loss of fine skin patterning is a sign of both aging and photoaging. Studies investigating the genetic contribution to skin patterning offer an opportunity to better understand a trait that influences both physical appearance and risk of keratinocyte skin cancer. We undertook a meta-analysis of genome-wide association studies of a measure of skin pattern (microtopography score) damage in 1,671 twin pairs and 1,745 singletons (N = 5,087) drawn from three independent cohorts. We identified that rs185146 near SLC45A2 is associated with a skin aging trait at genome-wide significance (P = 4.1 × 10); to our knowledge this is previously unreported. We also confirm previously identified loci, rs12203592 near IRF4 (P = 8.8 × 10) and rs4268748 near MC1R (P = 1.2 × 10). At all three loci we highlight putative functionally relevant SNPs. There are a number of red hair/low pigmentation alleles of MC1R; we found that together these MC1R alleles explained 4.1% of variance in skin pattern damage. We also show that skin aging and reported experience of sunburns was proportional to the degree of penetrance for red hair of alleles of MC1R. Our work has uncovered genetic contributions to skin aging and confirmed previous findings, showing that pigmentation is a critical determinant of skin aging.