Background Studies have shown significant associations of chronic kidney disease (CKD) and diabetes mellitus (DM) with cardiovascular disease (CVD) mortality. The impact on the general population is an important public health issue. However, the population attributable fraction (PAF) of CVD because of CKD and/or DM in a general population is uncertain. Methods We followed 7229 participants (age: ≥30 years) with no history of CVD in a Japanese community. We divided participants into four categories according to CKD and/or DM and calculated hazard ratios (HRs) of CVD mortality for each category using a Cox proportional hazards model adjusted for age, dyslipidemia, smoking and alcohol consumption. In addition, PAFs of CVD were estimated among populations with CKD and/or DM. Results During 20-year follow-up, 488 participants died. HRs for CVD were 1.63 [95% confidence interval (CI): 1.16-2.30] with DM only, 1.42 (95% CI: 1.08-1.86) with CKD only and 2.37 (95% CI: 1.40-4.01) with CKD + DM. In men, the corresponding HRs for CVD were 1.88 (95% CI: 1.19-2.97), 1.71 (95% CI: 1.15-2.56) and 3.26 (95% CI: 1.69-6.30), respectively; the corresponding PAFs of CVD were 4.1%, 5.1% and 2.9%, respectively. PAFs for CVD among women were lower than those in men, 1.6% for DM only, 2.0% for CKD only and 0.7% for CKD + DM. Conclusions PAFs of CVD mortality due to CKD and/or DM were not so high in past 20 years; however, they might increase in the future because of recent increase in prevalence of these in Japanese population.