F-box and WD repeat domain containing 7 (FBXW7) is a tumor suppressor gene frequently inactivated in several human malignancies. The present study aimed to investigate the role of FBXW7 in the invasion, migration and angiogenesis of ovarian cancer (OC) cells, and to identify its potential molecular mechanisms. First, the expression levels of FBXW7 and vascular endothelial growth factor (VEGF) were detected in several human OC cell lines using western blotting. Subsequently, FBXW7 was overexpressed to determine VEGF expression in SKOV3 cells. Transwell, wound healing and tube formation assays were performed following transfection with FBXW7 and VEGF overexpression plasmids to assess invasion, migration and angiogenesis in SKOV3 cells, respectively. Western blot analysis was performed to detect the expression levels of epithelial-to-mesenchymal transition and angiogenesis-associated proteins. In addition, the expression levels of β-catenin and c-Myc were assessed, and lithium chloride (LiCl), an agonist of β-catenin signaling, was used to elucidate the molecular mechanisms by which FBXW7 mediates its antitumor activity in OC. The results demonstrated that FBXW7 expression was markedly downregulated, whilst VEGF expression was markedly upregulated in OC cell lines compared with that in normal ovarian cells. Overexpression of FBXW7 significantly decreased VEGF expression in SKOV3 cells. Notably, overexpression of VEGF reversed the inhibitory effects of FBXW7 overexpression on the invasion, migration and angiogenesis of OC cells, accompanied by upregulated expression levels of N-cadherin, slug, CD31, VEGF receptor 1 (VEGFR1) and VEGFR2, and downregulated expression levels of E-cadherin. Furthermore, overexpression of FBXW7 markedly suppressed β-catenin and c-Myc expression, whereas the decreased expression levels of VEGF, VEGFR1 and VEGFR2 following overexpression of FBXW7 were increased after treatment of SKOV3 cells with LiCl. Overall, the results of the present study suggested that FBXW7 inhibited invasion, migration and angiogenesis of OC cells by suppressing VEGF expression through inactivation of β-catenin signaling. Thus, FBXW7 may be used as a novel therapeutic target for the treatment of OC.