Working memory (WM) is a latent cognitive structure that serves to store and manipulate a limited amount of information over a short time period. How information is maintained in WM remains a debated issue: it is unclear whether stimuli from different sensory domains are maintained under distinct mechanisms or maintained under the same mechanism. Previous neuroimaging research on this issue to date has focused on individual brain regions and has not provided a comprehensive view of the functional networks underlying multi-domain WM. To study the functional networks involved in visual and auditory WM, we applied constrained principal component analysis (CPCA) to a functional magnetic resonance imaging (fMRI) dataset acquired when participants performed a change-detection task requiring them to remember only visual, only auditory, or both visual and auditory stimuli. Analysis revealed evidence of both [1] domain-specific networks responsive to either visual or auditory WM (but not both), and [2] domain-general networks responsive to both visual and auditory WM. The domain-specific networks showed load-dependent activations during only encoding, whereas a domain-general network was sensitive to WM load across encoding, maintenance, and retrieval. The latter domain-general network likely reflected attentional processes involved in WM encoding, retrieval, and possibly maintenance as well. These results do not support the domain-specific account of WM maintenance but instead favor the domain-general theory that items from different sensory domains are maintained under the same mechanism.