Receptive fields of primary auditory cortex (A1) neurons show excitatory neuronal frequency preference and diverse inhibitory sidebands. While the frequency preferences of excitatory neurons in local A1 areas can be heterogeneous, those of inhibitory neurons are more homogeneous. To date, the diversity and the origin of inhibitory sidebands in local neuronal populations and the relation between local cellular frequency preference and inhibitory sidebands are unknown. To reveal both excitatory and inhibitory subfields, we presented two-tone and pure tone stimuli while imaging excitatory neurons (Thy1) and two types of inhibitory neurons (parvalbumin and somatostatin) in L2/3 of mice A1. We classified neurons into six classes based on frequency response area (FRA) shapes and sideband inhibition depended both on FRA shapes and cell types. Sideband inhibition showed higher local heterogeneity than frequency tuning, suggesting that sideband inhibition originates from diverse sources of local and distant neurons. Two-tone interactions depended on neuron subclasses with excitatory neurons showing the most nonlinearity. Onset and offset neurons showed dissimilar spectral integration, suggesting differing circuits processing sound onset and offset. These results suggest that excitatory neurons integrate complex and nonuniform inhibitory input. Thalamocortical terminals also exhibited sideband inhibition, but with different properties from those of cortical neurons. Thus, some components of sideband inhibition are inherited from thalamocortical inputs and are further modified by converging intracortical circuits. The combined heterogeneity of frequency tuning and diverse sideband inhibition facilitates complex spectral shape encoding and allows for rapid and extensive plasticity. Sensory systems recognize and differentiate between different stimuli through selectivity for different features. Sideband inhibition serves as an important mechanism to sharpen stimulus selectivity, but its cortical mechanisms are not entirely resolved. We imaged pyramidal neurons and two common classes of interneurons suggested to mediate sideband inhibition (parvalbumin and somatostatin positive) in the auditory cortex and inferred their inhibitory sidebands. We observed a higher degree of variability in the inhibitory sideband than in the local frequency tuning, which cannot be predicted from the relative high homogeneity of responses by inhibitory interneurons. This suggests that cortical sideband inhibition is nonuniform and likely results from a complex interplay between existing functional inhibition in the feedforward input and cortical refinement.