Caffeine is a widely consumed stimulant with dose-dependent effects on behavior. Across species, lower doses tend be anxiolytic and increase activity, while higher doses tend to be anxiogenic and decrease activity. Given the importance of the social environment on stress responses, we investigated how social stimuli modulate behavioral responses to caffeine. We exposed adult zebrafish to low (25 mg/L) or moderate (60 mg/L) caffeine either in isolation or within view of a stimulus group. Zebrafish are highly social, and social stimuli are known to buffer responses to environmental stressors. As predicted, we found that isolated fish were more sensitive to caffeine's anxiogenic effects and less sensitive to caffeine's stimulant effects. Among socially-exposed individuals, caffeine decreased social approach. Our data illustrate the importance of social context. Understanding how social cues shape pharmacological responses is important for understanding the external validity of studies that investigate drug effects in social species.