Physical frailty and genetic factors are both risk factors for increased dementia; nevertheless, the joint effect remains unclear. This study aimed to investigated the long-term relationship between physical frailty, genetic risk, and dementia incidence. A total of 274,194 participants from the UK Biobank were included. We applied Cox proportional hazards regression models to estimate the association between physical frailty and genetic and dementia risks. Among the participants (146,574 females [53.45%]; mean age, 57.24 years), 3,353 (1.22%) new-onset dementia events were recorded. Compared to non-frailty, the hazard ratio (HR) for dementia incidence in prefrailty and frailty was 1.396 (95% confidence interval [CI], 1.294-1.506, P < 0.001) and 2.304 (95% CI, 2.030-2.616, P < 0.001), respectively. Compared to non-frailty and low polygenic risk score (PRS), the HR for dementia risk was 3.908 (95% CI, 3.051-5.006, P < 0.001) for frailty and high PRS. Furthermore, among the participants, slow walking speed (HR, 1.817; 95% CI, 1.640-2.014, P < 0.001), low physical activity (HR, 1.719; 95% CI, 1.545-1.912, P < 0.001), exhaustion (HR, 1.670; 95% CI, 1.502-1.856, P < 0.001), low grip strength (HR, 1.606; 95% CI, 1.479-1.744, P < 0.001), and weight loss (HR, 1.464; 95% CI, 1.328-1.615, P < 0.001) were independently associated with dementia risk compared to non-frailty. Particularly, precise modulation for different dementia genetic risk populations can also be identified due to differences in dementia risk resulting from the constitutive pattern of frailty in different genetic risk populations. In conclusion, both physical frailty and high genetic risk are significantly associated with higher dementia risk. Early intervention to modify frailty is beneficial for achieving primary and precise prevention of dementia, especially in those at high genetic risk.