Temporal lobe epilepsy (TLE) has been conceptualized as focal disease with a discrete neurobiological focus and can respond well to targeted resection or ablation. In contrast, the neuro-cognitive deficits resulting from TLE can be widespread involving regions beyond the primary epileptic network. We hypothesize that this seemingly paradoxical findings can be explained by differences in connectivity between the primary epileptic region which is hyper-connected and its secondary influence on global connectome organization. This hypothesis is tested using regional and global graph theory metrics where we anticipate that regional mesial-temporal hyperconnectivity will be found and correlate with seizure frequency while global networks will be disorganized and be more closely associated with neuro-cognitive deficits. Resting-state fMRI was used to examine temporal lobe regional connectivity and global functional connectivity from 102 patients with TLE and 55 controls. Connectivity matrices were calculated for subcortical volumes and cortical parcellations. Graph theory metrics (global clustering coefficient (GCC), degree, closeness) were compared between groups and in relation to neuropsychological profiles and disease covariates using permutation testing and causal analysis. In TLE there was a decrease in GCC (p = 0.0345) associated with a worse neuropsychological profile (p = 0.0134). There was increased connectivity in the left hippocampus/amygdala (degree p = 0.0103, closeness p = 0.0104) and a decrease in connectivity in the right lateral temporal lobe (degree p = 0.0186, closeness p = 0.0122). A ratio between the hippocampus/amygdala and lateral temporal lobe-temporal lobe connectivity ratio (TLCR) revealed differences between TLE and controls for closeness (left p = 0.00149, right p = 0.0494) and for degree on left p = 0.00169; with trend on right p = 0.0567. Causal analysis suggested that "Epilepsy Activity" (seizure frequency, anti-seizure medications) was associated with increase in TLCR but not in GCC, while cognitive decline was associated with decreased GCC. These findings support the hypothesis that in TLE there is hyperconnectivity in the hippocampus/amygdala and hypoconnectivity in the lateral temporal lobe associated with "Epilepsy Activity." While, global connectome disorganization was associated with worse neuropsychological phenotype.