BACKGROUND AND STUDY AIMS: Probe-based confocal laser endomicroscopy (pCLE) enables real-time microscopic visualization of the duodenal mucosa and has shown acute food-triggered disruption of the duodenal epithelial barrier of patients with irritable bowel syndrome (IBS). The interpretation of the recordings is subjective, with unknown agreement rates. The aim of this study was to investigate the intra- and interobserver variability of this technique. PATIENTS AND METHODS: An international multicenter study was performed, including pCLE recordings from three centers. Recordings were randomized and re-evaluated by five blinded experienced assessors. Low-quality recordings were excluded. The mucosa was considered altered if both fluorescein leakage and luminal particles were observed. Agreement was quantified using Fleiss' and Cohen's kappa (κ). Reference videos (i.e., videos with 100% agreement) were used to assess the optimal characteristics of videos needed to make a judgment based on the optimal receiver operating characteristic curve cutoff. RESULTS: Of the 119 individual recordings, 87 could be used for analyses (total of 86,408 frames). Intraindividual agreement rate was 80%-100%, whereas the interindividual agreement rate was 85% (κ = 0.68). The agreement rate with the endoscopist ranged 54%-95% (κ = 0.15-0.89). The optimal cutoff to distinguish altered from unaltered was by observing alterations in ≥ 2 out of 6 mucosal spots (100% sensitivity and specificity). CONCLUSION: Our study showed a substantial to perfect intraobserver agreement and a substantial interobserver agreement for the judgment of acute food-triggered disruption of the duodenal epithelial barrier by pCLE, confirming that this real-time readout is reliable and reproducible.