High red and processed meat intake and genetic predisposition are risk factors of colorectal cancer (CRC). However, evidence of their independent and joint associations on the risk of colorectal neoplasms is limited. We assessed these associations among 4774 men and women undergoing screening colonoscopy. Polygenic risk scores (PRSs) were calculated based on 140 loci related to CRC. We used multiple logistic regression models to evaluate the associations of red and processed meat intake and PRS with the risk of colorectal neoplasms. Adjusted odds ratios (aORs) were translated to genetic risk equivalents (GREs) to compare the strength of the associations with colorectal neoplasm risk of both factors. Compared to ≤1 time/week, processed meat intake >1 time/week was associated with a significantly increased risk of colorectal neoplasm [aOR (95% CI): 1.28 (1.12-1.46)]. This risk increase was equivalent to the risk increase associated with a 19 percentile higher PRS. The association of red meat intake with colorectal neoplasm was weaker and did not reach statistical significance. High processed meat intake and PRS contribute to colorectal neoplasm risk independently. Limiting processed meat intake may offset a substantial proportion of the genetically increased risk of colorectal neoplasms.