School of Animal and Veterinary Sciences, Charles Sturt University, Wagga Wagga, New South Wales, Australia.
Quantitative Consultant Unit, Charles Sturt University, Wagga Wagga, New South Wales, Australia.
J Vet Intern Med. 2021 Jan;35(1):571-579. doi: 10.1111/jvim.15987. Epub 2020 Dec 7.
Grading of equine gastric ulcer syndrome (EGUS) is undertaken in clinical and research settings, but the reliability of EGUS grading systems is poorly understood.
HYPOTHESIS/OBJECTIVES: Investigate interobserver and intraobserver reliability of an established ordinal grading system and a novel visual analog scale (VAS), and assess the influence of observer experience.
Sixty deidentified gastroscopy videos.
Six observers (3 specialists and 3 residents) graded videos using the EGUS Council (EGUC) system and VAS. Observers graded the videos three 3 for each system, using a cross-over design with at least 1 week between each phase. The order of videos was randomized for each phase.
Interobserver and intraobserver reliability were estimated using Gwet's agreement coefficient with ordinal weights applied (AC2) for the EGUC system and the intraclass correlation coefficient (ICC) for the VAS.
Using the EGUC system, interobserver reliability was substantial for squamous (AC2 = 0.69; 95% confidence interval [CI], 0.57-0.80) and glandular mucosa (AC2 = 0.72; 95% CI, 0.70-0.75), and intraobserver reliability was substantial for squamous (AC2 = 0.80; 95% CI, 0.71-0.90) and glandular mucosa (AC2 = 0.80; 95% CI, 0.74-0.86). Interobserver reliability using the VAS was moderate for squamous (ICC = 0.64; 95% CI, 0.31-0.96) and poor for glandular mucosa (ICC = 0.35; 95% CI, 0.06-0.64), and intraobserver reliability was moderate for squamous (ICC = 0.74; 95% CI, 0.62-0.86) and glandular mucosa (ICC = 0.56; 95% CI, 0.39-0.72).
The EGUC system had acceptable intraobserver and interobserver reliability and performed well regardless of observer experience. Familiarity and observer experience improved reliability of the VAS.
在临床和研究环境中进行马属动物胃溃疡综合征（EGUS）分级，但 EGUS 分级系统的可靠性知之甚少。
假设/目的：研究一种既定的等级评分系统和一种新的视觉模拟评分（VAS）的观察者间和观察者内可靠性，并评估观察者经验的影响。
60 个未经识别的胃镜检查视频。
六名观察者（3 名专家和 3 名住院医师）使用 EGUS 委员会（EGUC）系统和 VAS 对视频进行分级。观察者使用交叉设计，每个系统至少在每个阶段之间间隔 1 周，对每个阶段的视频进行 3 次分级。每个阶段的视频顺序都是随机的。
使用 EGUC 系统，通过应用有序权重（AC2）估计观察者间和观察者内可靠性，用于鳞状（AC2=0.69；95%置信区间[CI]，0.57-0.80）和腺黏膜（AC2=0.72；95%CI，0.70-0.75），并且对于鳞状（AC2=0.80；95%CI，0.71-0.90）和腺黏膜（AC2=0.80；95%CI，0.74-0.86），观察者内可靠性为高度可靠。使用 VAS 时，对于鳞状（ICC=0.64；95%CI，0.31-0.96），观察者间可靠性为中度，对于腺黏膜（ICC=0.35；95%CI，0.06-0.64），观察者间可靠性为低度，对于鳞状（ICC=0.74；95%CI，0.62-0.86）和腺黏膜（ICC=0.56；95%CI，0.39-0.72），观察者内可靠性为中度。
无论观察者的经验如何，EGUC 系统都具有可接受的观察者内和观察者间可靠性，并且表现良好。熟悉程度和观察者经验提高了 VAS 的可靠性。
