Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan.
Sleep Medicine Center, National Cheng Kung University Hospital, College of Medicine, National Cheng Kung University, Tainan, Taiwan.
J Clin Sleep Med. 2024 Feb 1;20(2):271-278. doi: 10.5664/jcsm.10852.
To efficiently improve the scoring competency of scorers with varying levels of experience across regions in Taiwan, we developed a training program with a cloud-based polysomnography scoring platform to evaluate and improve interscorer agreement.
A total of 70 scorers from 34 sleep centers in Taiwan (job tenure: 0.5-39.0 years) completed a scoring test. All scorers scored a 742-epoch (30 s/epoch) overnight polysomnography recording of a patient with a moderate apnea-hypopnea index. Subsequently, 8 scoring experts delivered 8 interactive online lectures (each lasting 30 minutes). The training program included identifying scoring weaknesses, highlighting the latest scoring rules, and providing physicians' perspectives. Afterward, the scorers completed the second scoring test on the same participant. Changes in agreement from the first to second scoring test were identified. Sleep staging, sleep parameters, and respiratory events were considered for evaluating scoring agreement.
The scorers' agreement in overall sleep stage scoring significantly increased from 74.6 to 82.3% (median score). The proportion of scorers with an agreement of ≥ 80% increased from 20.0% (14/70) to 58.6% (41/70) after the online training program. In addition, the scorers' agreement in overall respiratory-event scoring increased to 88.8% (median score) after training. The scorers with a job tenure of 2.0-4.9 years exhibited the highest level of improvement in overall sleep staging (their median agreement increased from 72.8 to 84.9%; < .001).
Our interactive online training program efficiently targeted the scorers' scoring weaknesses identified in the first scoring test, leading to substantial improvements in scoring proficiency.
Liao Y-S, Wu M-C, Li C-X, Lin W-K, Lin C-Y, Liang S-F. Polysomnography scoring-related training and quantitative assessment for improving interscorer agreement. . 2024;20(2):271-278.
为了有效提高台湾不同地区、不同经验水平评分员的评分能力，我们开发了一个基于云的多导睡眠图评分平台的培训计划，以评估和提高评分员之间的一致性。
来自台湾 34 个睡眠中心的 70 名评分员（工作年限：0.5-39.0 年）完成了评分测试。所有评分员都对一名中度睡眠呼吸暂停低通气指数患者的 742 个时段（30 秒/时段）的整夜多导睡眠图记录进行了评分。随后，8 名评分专家进行了 8 次在线互动讲座（每次持续 30 分钟）。培训计划包括识别评分弱点、强调最新的评分规则，并提供医生的观点。之后，评分员对同一名参与者进行了第二次评分测试。从第一次评分测试到第二次评分测试的一致性变化被确定。评估评分一致性时考虑了睡眠分期、睡眠参数和呼吸事件。
整体睡眠分期评分的评分员一致性从 74.6%显著提高到 82.3%（中位数评分）。在线培训计划后，具有≥80%一致性的评分员比例从 20.0%（14/70）增加到 58.6%（41/70）。此外，培训后，整体呼吸事件评分的评分员一致性提高到 88.8%（中位数评分）。工作年限为 2.0-4.9 年的评分员在整体睡眠分期方面表现出最高水平的提高（他们的中位数一致性从 72.8%提高到 84.9%；<.001）。
我们的互动在线培训计划有效地针对在第一次评分测试中确定的评分员评分弱点，从而大大提高了评分能力。
廖 Y-S，吴 M-C，李 C-X，林 W-K，林 C-Y，梁 S-F。多导睡眠图评分相关培训和定量评估以提高评分者之间的一致性。 2024；20（2）：271-278。
