Smooth pursuit operates over perceived not physical positions of the double-drift stimulus.

The double-drift illusion produces a large deviation in perceived direction that strongly dissociates physical position from perceived position. Surprisingly, saccades do not seem to be affected by the illusion (Lisi & Cavanagh, 2015). When targeting a double-drift stimulus, the saccade system is driven by retinal rather than perceived position. Here, using paired double-drift targets, we test whether the smooth pursuit system is driven by perceived or physical position. Participants (n = 7) smoothly pursued the inferred midpoint (Steinbach, 1976) between two horizontally aligned Gabor patches that were separated by 20° and moving on parallel, oblique paths. On the first half of each trial, the Gabors' internal textures were static while both drifted obliquely downward. On the second half of each trial, while the envelope moved obliquely upward, the internal texture drifted orthogonally to the envelope's motion, producing a large perceived deviation from the downward path even though the upward and downward trajectories always followed the same physical path but in opposite directions. We find that smooth pursuit eye movements accurately followed the nonillusory downward path of the midpoint between the two Gabors, but then followed the illusory rather than the physical trajectory on the upward return. Thus, virtual targets for smooth pursuit are derived from perceived rather than retinal coordinates.