BACKGROUND: Emotion recognition dysfunction has been reported in both autism spectrum disorders (ASD) and attention-deficit/hyperactivity disorder (ADHD). This suggests that emotion recognition is a cross-disorder trait that may be utilised to understand the heterogeneous psychopathology of ASD and ADHD. We aimed to identify emotion recognition subtypes and to examine their relation with quantitative and diagnostic measures of ASD and ADHD to gain further insight into disorder comorbidity and heterogeneity. METHODS: Factor mixture modelling was used on speed and accuracy measures of auditory and visual emotion recognition tasks. These were administered to children and adolescents with ASD (N = 89), comorbid ASD + ADHD (N = 64), their unaffected siblings (N = 122), ADHD (N = 111), their unaffected siblings (N = 69), and controls (N = 220). Identified classes were compared on diagnostic and quantitative symptom measures. RESULTS: A four-class solution was revealed, with the following emotion recognition abilities: (1) average visual, impulsive auditory; (2) average-strong visual and auditory; (3) impulsive/imprecise visual, average auditory; (4) weak visual and auditory. The weakest performing class (4) contained the highest percentage of patients (66.07%) and the lowest percentage controls (10.09%), scoring the highest on ASD/ADHD measures. The best performing class (2) demonstrated the opposite: 48.98% patients, 15.26% controls with relatively low scores on ASD/ADHD measures. CONCLUSIONS: Subgroups of youths can be identified that differ both in quantitative and qualitative aspects of emotion recognition abilities. Weak emotion recognition abilities across sensory domains are linked to an increased risk for ASD as well as ADHD, although emotion recognition impairments alone are neither necessary nor sufficient parts of these disorders.