Associations between larger size at birth and increased rates of adult cancer have been proposed but few empirical studies have examined this hypothesis. We investigated overall and site-specific cancer incidence in relation to birth characteristics in a Swedish population-based cohort of 11,166 singletons born in 1915-1929 for whom we have detailed obstetric data and who were alive in 1960. A total of 2,685 first primary cancers were registered during follow-up from 1960 to 2001. A standard deviation (SD) increase in birth weight for gestational age (GA) was associated with (sex-adjusted) increases of 13% (95% CI = 0.03-0.23) in the rates of digestive cancers and of 17% (95% CI = 0.01-0.35) in the rates of lymphatic cancers. Women who had higher birth weights also had increased rates of breast cancer under age 50 years (by 39% per SD increase; 95% CI = 0.09-0.79), but reduced rates (by 24%; 95% CI = 0.07-0.38) of endometrial (corpus uteri) cancer at all ages. There was no evidence of associations with other cancer sites. For overall cancer incidence, men had an 8% increased risk at all ages per SD increase in birth weight for GA while women only had an increased risk under age 50 years (mainly driven by the association with breast cancer). These findings provide evidence of a modest association of birth size and adult cancer risk, resulting from positive associations with a few cancer sites and a possible inverse association with endometrial cancer.