BACKGROUND: Validation studies on an ICD-10-based algorithm to identify major bleeding events are scarce, and mostly focused on positive predictive values. OBJECTIVE: To evaluate the sensitivity and specificity of an ICD-10-based algorithm in adult patients referred to hospital. METHODS: This was a cross-sectional, retrospective analysis. Among all hospital stays of adult patients referred to Rennes University Hospital, France, through the emergency ward in 2014, we identified major bleeding events according to an index test based on a list of ICD-10 diagnoses. As a reference, a two-step process was applied: firstly, a computerized request for electronic health records from the emergency ward, using several hemorrhage-related diagnostic codes and specific emergency therapies so as to discard stays with a very low probability of bleeding; secondly, a chart review of selected records was conducted by a medical expert blinded to the index test results and each hospital stay was classified into one of two exclusive categories: major bleeding or no major bleeding, according to pre-specified criteria. RESULTS: Out of 16,012 hospital stays, the reference identified 736 major bleeding events and left 15,276 stays considered as without the target condition. The index test identified 637 bleeding events: 293 intracranial hemorrhages, 197 gastrointestinal hemorrhages and 147 other bleeding events. Overall, sensitivity was 65% (95%CI, 62 to 69), and specificity was 99.0%. We observed differential sensitivity and specificity across bleeding types, with the highest values for intracranial hemorrhage. Positive predictive values ranged from 59% for "other" bleeding events, to 71% (95%CI, 65 to 78) for gastrointestinal hemorrhage, and 96% for intracranial hemorrhage. CONCLUSIONS: Low sensitivity and differential measures of accuracy across bleeding types support the need for specific data collection and medical validation rather than using an ICD-10-based algorithm for assessing the incidence of major bleeding.