Histone post-translational modifications regulate chromatin structure and function largely through interactions with effector proteins that often contain multiple histone-binding domains. PHF1 [plant homeodomain (PHD) finger protein 1], which contains two kinds of histone reader modules, a Tudor domain and two PHD fingers, is an essential factor for epigenetic regulation and genome maintenance. While significant progress has been made in characterizing the function of the Tudor domain, the roles of the two PHD fingers are poorly defined. Here, we demonstrated that the N-terminal PHD finger of PHF1 recognizes symmetric dimethylation of H4R3 (H4R3me2s) catalyzed by PRMT5-WDR77. However, the C-terminal PHD finger of PHF1, instead of binding to modified histones, directly interacts with DDB1, the main component of the CUL4B-Ring E3 ligase complex (CRL4B), which is responsible for H2AK119 mono-ubiquitination (H2AK119ub1). We showed that PHF1, PRMT5-WDR77, and CRL4B reciprocally interact with one another and collaborate as a functional unit. Genome-wide analysis of PHF1/PRMT5/CUL4B targets identified a cohort of genes including E-cadherin and FBXW7, which are critically involved in cell growth and migration. We demonstrated that PHF1 promotes cell proliferation, invasion, and tumorigenesis in vivo and in vitro and found that its expression is markedly upregulated in a variety of human cancers. Our data identified a new reader for H4R3me2s and provided a molecular basis for the functional interplay between histone arginine methylation and ubiquitination. The results also indicated that PHF1 is a key factor in cancer progression, supporting the pursuit of PHF1 as a target for cancer therapy.