The objective of this study was to investigate the relation between self-report and objective assessment of motivational interviewing (MI) skills following training and supervision. After an MI workshop, 96 clinicians from 26 community programs (age 21-68, 65% female, 40.8% Black, 29.6% Caucasian, 24.5% Hispanic, 2.0% Asian, 3.1% other) were randomized to supervision (tele-conferencing or tape-based), or workshop only. At four time points, trainees completed a self-report of MI skill, using items from the MI understanding questionnaire (MIU), and were objectively assessed by raters using the Motivational Interviewing Treatment Integrity (MITI) system. Correlations were calculated between MIU and MITI scores. A generalized linear mixed model was tested on MIU scores, with MITI scores, supervision condition and time as independent variables. MIU scores increased from pre-workshop (mean = 4.74, SD = 1.79) to post-workshop (mean = 6.31, SD = 1.03) (t = 8.69, p < .0001). With supervision, scores continued to increase, from post-workshop to week 8 (mean = 7.07, SD = 0.91, t = 5.60, p < .0001) and from week 8 to week 20 (mean = 7.28, SD = 0.94, t = 2.43, p = .02). However, MIU scores did not significantly correlate with MITI scores, with or without supervision. Self-reported ability increased with supervision, but self-report was not an indicator of objectively measured skill. This suggests that training does not increase correspondence between self-report and objective assessment, so community treatment programs should not rely on clinician self-report to assess the need for ongoing training and supervision and it may be necessary to train clinicians to accurately assess their own skill.