There is ongoing debate concerning the mechanisms of feature binding in working memory. In particular, there is controversy regarding the extent to which these binding processes are automatic. The present article demonstrates that binding mechanisms differ depending on whether the to-be-integrated features are perceived as forming a coherent object. We presented a series of experiments that investigated the binding of color and shape, whereby color was either an intrinsic feature of the shape or an extrinsic feature of the shape's background. Results show that intrinsic color affected shape recognition, even when it was incidentally studied and irrelevant for the recognition task. In contrast, extrinsic color did not affect shape recognition, even when the association of color and shape was encoded and retrievable on demand. This strongly suggests that binding of intrinsic intra-item information but not extrinsic contextual information is obligatory in visual working memory. We highlight links to perception as well as implicit and explicit long-term memory, which suggest that the intrinsic-extrinsic dimension is a principle relevant to multiple domains of human cognition.