Artificial intelligence vs. human expert: Licensed mental health clinicians' blinded evaluation of AI-generated and expert psychological advice on quality, empathy, and perceived authorship.

BACKGROUND

The use of artificial intelligence for psychological advice shows promise for enhancing accessibility and reducing costs, but it remains unclear whether AI-generated advice can match the quality and empathy of experts.

METHOD

In a blinded, comparative cross-sectional design, licensed psychologists and psychotherapists assessed the quality, empathy, and authorship of psychological advice, which was either AI-generated or authored by experts.

RESULTS

AI-generated responses were rated significantly more favorable for emotional (OR = 1.79, 95 % CI [1.1, 2.93],  = .02) and motivational empathy (OR = 1.84, 95 % CI [1.12, 3.04],  = .02). Ratings for scientific quality ( = .10) and cognitive empathy ( = .08) were comparable to expert advice. Participants could not distinguish between AI- and expert-authored advice ( = .27), but expert authorship was associated with more favorable ratings across these measures (ORs for perceived AI vs. perceived expert ranging from 0.03 to 0.15, all  < .001). For overall preference, AI-authored advice was favored when assessed blindly based on its actual source ( = 6.96,  = .002). Nevertheless, advice as expert-authored was also strongly preferred ( = 6.26,  = .001), with 93.55 % of participants preferring the advice they believed came from an expert, irrespective of its true origin.

CONCLUSIONS

AI demonstrates potential to match expert performance in asynchronous written psychological advice, but biases favoring perceived expert authorship may hinder its broader acceptance. Mitigating these biases and evaluating AI's trustworthiness and empathy are important next steps for safe and effective integration of AI in clinical practice.