Here we show that the pupillary light response reflects exogenous (involuntary) shifts of attention and inhibition of return. Participants fixated in the center of a display that was divided into a bright and a dark half. An exogenous cue attracted attention to the bright or dark side of the display. Initially, the pupil constricted when the bright, as compared to the dark, side of the display was cued, reflecting a shift of attention toward the exogenous cue. Crucially, this pattern reversed about 1 s after cue presentation. This later-occurring, relative dilation (when the bright side was cued) reflected disengagement from the previously attended location, analogous to the behavioral phenomenon of inhibition of return. Indeed, we observed a reliable correlation between "pupillary inhibition" and behavioral inhibition of return. Our results support the view that inhibition of return results from habituation to (or short-term depression of) visual input. We conclude that the pupillary light response is a complex eye movement that reflects how we selectively parse and interpret visual input.