Congenital heart disease (CHD) survivors are at risk for neurodevelopmental disability (NDD), and recent studies identify genes associated with both disorders, suggesting that NDD in CHD survivors may be of genetic origin. Genes contributing to neurogenesis, dendritic development and synaptogenesis organize neural elements into networks known as the connectome. We hypothesized that NDD in CHD may be attributable to genes altering both neural connectivity and cardiac patterning. To assess the contribution of de novo variants (DNVs) in connectome genes, we annotated 229 published NDD genes for connectome status and analyzed data from 3,684 CHD subjects and 1,789 controls for connectome gene mutations. CHD cases had more protein truncating and deleterious missense DNVs among connectome genes compared to controls (OR = 5.08, 95%CI:2.81-9.20, Fisher's exact test P = 6.30E-11). When removing three known syndromic CHD genes, the findings remained significant (OR = 3.69, 95%CI:2.02-6.73, Fisher's exact test P = 1.06E-06). In CHD subjects, the top 12 NDD genes with damaging DNVs that met statistical significance after Bonferroni correction (PTPN11, CHD7, CHD4, KMT2A, NOTCH1, ADNP, SMAD2, KDM5B, NSD2, FOXP1, MED13L, DYRK1A; one-tailed binomial test P ≤ 4.08E-05) contributed to the connectome. These data suggest that NDD in CHD patients may be attributable to genes that alter both cardiac patterning and the connectome.