Histone lysine methylation has been linked to the recruitment of mammalian DNA repair factor 53BP1 and putative fission yeast homolog Crb2 to DNA double-strand breaks (DSBs), but how histone recognition is achieved has not been established. Here we demonstrate that this link occurs through direct binding of 53BP1 and Crb2 to histone H4. Using X-ray crystallography and nuclear magnetic resonance (NMR) spectroscopy, we show that, despite low amino acid sequence conservation, both 53BP1 and Crb2 contain tandem tudor domains that interact with histone H4 specifically dimethylated at Lys20 (H4-K20me2). The structure of 53BP1/H4-K20me2 complex uncovers a unique five-residue 53BP1 binding cage, remarkably conserved in the structure of Crb2, that best accommodates a dimethyllysine but excludes a trimethyllysine, thus explaining the methylation state-specific recognition of H4-K20. This study reveals an evolutionarily conserved molecular mechanism of targeting DNA repair proteins to DSBs by direct recognition of H4-K20me2.