Cocaine craving, seeking, and relapse are mediated, in part, by cocaine-induced adaptive changes in the brain reward circuits. The nucleus accumbens (NAc) integrates and prioritizes different emotional and motivational inputs to the reward system by processing convergent glutamatergic projections from the medial prefrontal cortex, basolateral amygdala, ventral hippocampus, and other limbic and paralimbic brain regions. Medium spiny neurons (MSNs) are the principal projection neurons in the NAc, which can be divided into two major subpopulations, namely dopamine receptor D1- versus D2-expressing MSNs, with complementing roles in reward-associated behaviors. After cocaine experience, NAc MSNs exhibit complex and differential adaptations dependent on cocaine regimen, withdrawal time, cell type, location (NAc core versus shell), and related input and output projections, or any combination of these factors. Detailed characterization of these cellular adaptations has been greatly facilitated by the recent development of optogenetic/chemogenetic techniques combined with transgenic tools. In this review, we discuss such cell type- and projection-specific adaptations induced by cocaine experience. Specifically, (1) D1 and D2 NAc MSNs frequently exhibit differential adaptations in spinogenesis, glutamatergic receptor trafficking, and intrinsic membrane excitability, (2) cocaine experience differentially changes the synaptic transmission at different afferent projections onto NAc MSNs, (3) cocaine-induced NAc adaptations exhibit output specificity, e.g., being different at NAc-ventral pallidum versus NAc-ventral tegmental area synapses, and (4) the input, output, subregion, and D1/D2 cell type may together determine cocaine-induced circuit plasticity in the NAc. In light of the projection- and cell-type specificity, we also briefly discuss ensemble and circuit mechanisms contributing to cocaine craving and relapse after drug withdrawal.