One feature of autism spectrum disorder (ASD) is a deficit in verbal reference production, that is, providing an appropriate amount of verbal information for the listener to refer to things, people, and events. However, very few studies have manipulated whether individuals with ASD can take a speaker's perspective to interpret verbal reference. A critical limitation of all interpretation studies is that comprehension of another's verbal reference required the participant to represent only the other's visual perspective. Yet, many everyday interpretations of verbal reference require knowledge of social perspective (i.e., a consideration of which experiences one has shared with which interlocutor). We investigated whether 22 5;0-7;11-year-old children with ASD and 22 well-matched typically developing (TD) children used social perspective to comprehend (Study 1) and produce (Study 2) verbal reference. Social perspective-taking was manipulated by having children collaboratively complete activities with one of two interlocutors such that for a given activity, one interlocutor was Knowledgeable and one was Naïve. Study 1 found no between-group differences for the interpretation of ambiguous references based on social perspective. In Study 2, when producing referring terms, the ASD group made modifications based on listener needs, but this effect was significantly stronger in the TD group. Overall, the findings suggest that high-functioning children with ASD know with which interlocutor they have previously shared a given experience and can take this information into account to steer verbal reference. Nonetheless, they show clear performance limitations in this regard relative to well-matched controls. Autism Res 2018, 11: 1366-1375. © 2018 International Society for Autism Research, Wiley Periodicals, Inc. LAY SUMMARY: No one had studied if young children with autism spectrum disorder (ASD) could take into account previous collaboration with particular conversation partners to drive how well they communicate with others. In both their language understanding and spoken language, we found that five to 7-year-olds with ASD were able to consider what they had previously shared with the conversation partner. However, they were impaired when compared to typically developing children in the degree to which they tailored their spoken language for a specific listener.