IMPORTANCE

Suicide is a leading cause of death among young people. Accurate detection of self-injurious thoughts and behaviors (SITB) underpins equity in youth suicide prevention.

OBJECTIVES

To compare methods of detecting SITB using structured electronic health information and measure algorithmic performance across demographics.

DESIGN, SETTING, AND PARTICIPANTS: This cross-sectional study used medical records among youths aged 6 to 17 years with at least 1 mental health-related emergency department (ED) visit in 2017 to 2019 to an academic health system in Southern California serving 787 000 unique individuals each year. Analyses were conducted between January and September 2023.

EXPOSURES

Multiexpert electronic health record review ascertained the presence of SITB using the Columbia Classification Algorithm of Suicide Assessment. Random forest classifiers with nested cross-validation were developed using (1) International Statistical Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) codes for nonfatal suicide attempt and self-harm and chief concern and (2) all available structured data, including diagnoses, medications, and laboratory tests.

MAIN OUTCOME AND MEASURES

Detection performance was assessed overall and stratified by age group, sex, and race and ethnicity.

RESULTS

The sample comprised 2702 unique youths with an MH-related ED visit (1384 youths who identified as female [51.2%]; 131 Asian [4.8%], 266 Black [9.8%], 719 Hispanic [26.6%], 1319 White [48.8%], and 233 other race [8.6%]; median [IQR] age, 14 [12-16] years), including 898 children and 1804 adolescents. Approximately half of visits were related to SITB (1286 visits [47.6%]). Sensitivity of SITB detection using only codes and chief concern varied by age group and increased until age 15 years (6-9 years: 59.3% [95% CI, 48.5%-69.5%]; 10-12 years: 69.0% [95% CI, 63.8%-73.9%]; 13-15 years: 88.4% [95% CI, 85.1%-91.2%]; 16-17 years: 83.1% [95% CI, 79.1%-86.6%]), while specificity remained constant. The area under the receiver operating characteristic curve (AUROC) was lower among preadolescents (0.841 [95% CI, 0.815-0.867]) and male (0.869 [95% CI, 0.848-0.890]), Black (0.859 [95% CI, 0.813-0.905]), and Hispanic (0.861 [95% CI, 0.831-0.891]) youths compared with adolescents (0.925 [95% CI, 0.912-0.938]), female youths (0.923 [95% CI, 0.909-0.937]), and youths of other races and ethnicities (eg, White: 0.901 [95% CI, 0.884-0.918]). Augmented classification (ie, using all available structured data) outperformed classification with codes and chief concern alone (AUROC, 0.975 [95% CI, 0.968-0.980] vs 0.894 [95% CI, 0.882-0.905]; P < .001).

CONCLUSIONS AND RELEVANCE

In this study, diagnostic codes and chief concern underestimated SITB prevalence, particularly among minoritized youths. These results suggest that priority on algorithmic fairness in suicide prevention strategies must extend to accurate detection of youths with suicide-related emergencies.