Eye growth is regulated by the visual input. Many studies suggest that the retina can detect whether a visual image is focused in front of or behind the back of the eye and modulate eye growth to bring it back to focus. How can the retina distinguish between these two types of defocus? Here, we simulated how eye optics transforms natural images and recorded how the isolated retina responds to different types of simulated defocus. We found that some ganglion cell types could distinguish between an image focused in front of or behind the retina by estimating spatial contrast. Aberrations in the eye optics made spatial contrast, but not luminance, a reliable cue to distinguish these two types of defocus. Our results suggest a mechanism for how the retina can estimate the sign of defocus and provide an explanation for several results aiming at mitigating strong myopia by slowing down eye growth.