How are three-dimensional objects represented in the brain?

In this report we discuss a variety of psychophysical experiments that explore different aspects of the problem of object recognition and representation in human vision. In all experiments, subjects were presented with realistically rendered images of computer-generated 3D objects, with tight control over stimulus shape, surface properties, illumination, and viewpoint, as well as subjects' prior exposure to the stimulus objects. Contrary to the predictions of the paradigmatic theory of recognition, which holds that object representations are viewpoint invariant, performance in all experiments was consistently viewpoint dependent, was only partially aided by binocular stereo and other depth information, was specific to viewpoints that were familiar, and was systematically disrupted by rotation in depth more than by deforming the 2D images of the stimuli. The emerging concept of multiple-views representation supported by these results is consistent with recently advanced computational theories of recognition based on view interpolation. Moreover, in several simulated experiments employing the same stimuli used in experiments with human subjects, models based on multiple-views representations replicated many of the psychophysical results concerning the observed pattern of human performance.