Theories of attention postulate the existence of an attentional template containing target features in working or long-term memory. Previous research has shown that these internal representations of target features in memory are shifted away from nontarget features and that attention is tuned to the shifted feature especially when the target appeared with similar nontarget items. While previous studies have shown that the target-nontarget relationship has influence on the attentional selection and the representation shift when attentional template is maintained in long-term memory, there is little evidence for such effects when attentional template is stored in working memory. To address this issue, we asked participants to search for a target, which varied from trial to trial (working memory attentional template), or look for the target being stable across trials (long-term memory attentional template). We found that the shifted target features captured attention and that the representations of target features were deviated away from nontarget features when the target template was stored in either working memory or long-term memory. However, such effects were found to be greater for the attentional template in long-term memory. The present results provide evidence that one can encode the target-nontarget relationship even though the target varies from trial to trial, and such contextual information influences attentional selection and target representation shift even under this dynamically changing environment.