In the present study we explore how social factors (group contact, individuating experience, implicit racial bias) influence the eye movements made during the visual exploration when judging their aesthetic merit of figurative paintings depicting White and Black sitters. An opportunity sample of participants visiting a gallery in Liverpool viewed ten artworks while their eye movements were recorded and completed a set of individual difference measures. The individual difference measures indicated self-report of art interest, social contact and individuating experience with both Black and White communities, and implicitly held racial bias. The results showed that, despite viewing the paintings for less time, the majority of participants reported paintings showing Black sitters as more interesting, emotionally moving, and pleasurable then those depicting White sitters. However, if a participant reported limited social contact with Black community, and a negative implicit racial bias against them, their rating of aesthetic merit of paintings showing Black sitters was reduced, viewing time increased, and fixations became more focused on faces. The influence of social factors on the viewing of paintings showing White sitters was limited to aesthetic rating. The results are discussed in terms of how social factors influence eye movements when viewing paintings in a real-world setting.