Perisomatic inhibition of pyramidal neurons (PNs) coordinates cortical network activity during sensory processing, and this role is mainly attributed to parvalbumin-expressing basket cells (BCs). However, cannabinoid receptor type 1 (CB1)-expressing interneurons are also BCs, but the connectivity and function of these elusive but prominent neocortical inhibitory neurons are unclear. We find that their connectivity pattern is visual area specific. Persistently active CB1 signaling suppresses GABA release from CB1 BCs in the medial secondary visual cortex (V2M), but not in the primary visual cortex (V1). Accordingly, in vivo, tonic CB1 signaling is responsible for higher but less coordinated PN activity in the V2M than in the V1. These differential firing dynamics in the V1 and V2M can be captured by a computational network model that incorporates visual-area-specific properties. Our results indicate a differential CB1-mediated mechanism controlling PN activity, suggesting an alternative connectivity scheme of a specific GABAergic circuit in different cortical areas.