KDM5B is a histone H3K4me2/3 demethylase. The PHD1 domain of KDM5B is critical for demethylation, but the mechanism underlying the action of this domain is unclear. In this paper, we observed that PHD1KDM5B interacts with unmethylated H3K4me0. Our NMR structure of PHD1KDM5B in complex with H3K4me0 revealed that the binding mode is slightly different from that of other reported PHD fingers. The disruption of this interaction by double mutations on the residues in the interface (L325A/D328A) decreases the H3K4me2/3 demethylation activity of KDM5B in cells by approximately 50% and increases the transcriptional repression of tumor suppressor genes by approximately twofold. These findings imply that PHD1KDM5B may help maintain KDM5B at target genes to mediate the demethylation activities of KDM5B.