Department of Health Technology, Technical University of Denmark, Kgs Lyngby, Denmark; Stanford University Center for Sleep Sciences and Medicine, Palo Alto, CA, USA.
Department of Health Technology, Technical University of Denmark, Kgs Lyngby, Denmark; Stanford University Center for Sleep Sciences and Medicine, Palo Alto, CA, USA; Danish Center for Sleep Medicine, Department of Clinical Neurophysiology, Rigshospitalet, Glostrup, Denmark.
Sleep Med. 2020 May;69:109-119. doi: 10.1016/j.sleep.2019.12.032. Epub 2020 Jan 23.
Currently, manual scoring is the gold standard of leg movement scoring (LMs) and periodic LMs (PLMS) in overnight polysomnography (PSG) studies, which is subject to inter-scorer variability. The objective of this study is to design and validate an end-to-end deep learning system for the automatic scoring of LMs and PLMS in sleep.
The deep learning system was developed, validated and tested, with respect to manual annotations by expert technicians on 800 overnight PSGs using a leg electromyography channel. The study includes data from three cohorts, namely, the Wisconsin Sleep Cohort (WSC), Stanford Sleep Cohort (SSC) and MrOS Sleep Study. The performance of the system was further compared against individual expert technicians and existing PLM detectors.
The system achieved an F1 score of 0.83, 0.71, and 0.77 for the WSC, SSC, and an ancillary study (Osteoporotic Fractures in Men Study, MrOS) cohorts, respectively. In a total of 60 PSGs from the WSC and the SSC scored by nine expert technicians, the system performed better than two and comparable to seven of the individual scorers with respect to a majority-voting consensus of the remaining scorers. In 60 PSGs from the WSC scored accurately for PLMS, the system outperformed four previous PLM detectors, which were all evaluated on the same data, with an F1 score of 0.85.
The proposed system performs better or comparable to individual expert technicians while outperforming previous automatic detectors. Thereby, the study validates fully automatic methods for scoring LMs in sleep.
目前，手动评分是夜间多导睡眠图（PSG）研究中腿部运动评分（LMs）和周期性 LMs（PLMS）的金标准，但存在评分者间的变异性。本研究旨在设计和验证一种用于睡眠中 LMs 和 PLMS 自动评分的端到端深度学习系统。
该深度学习系统是针对 800 例夜间 PSG 进行开发、验证和测试的，使用腿部肌电图通道由专家技师进行手动注释。该研究包括三个队列的数据，即威斯康星州睡眠队列（WSC）、斯坦福睡眠队列（SSC）和 MrOS 睡眠研究。该系统的性能还与个别专家技师和现有的 PLM 探测器进行了比较。
该系统在 WSC、SSC 和辅助研究（男性骨质疏松性骨折研究，MrOS）队列中的 F1 评分分别为 0.83、0.71 和 0.77。在来自 WSC 和 SSC 的总共 60 例 PSG 中，由 9 名专家技师进行评分，该系统在多数票共识的情况下，表现优于两名评分者，与 7 名评分者中的 7 名评分者相当。在对 WSC 中 60 例 PSG 进行准确的 PLMS 评分中，该系统在相同数据上评估的四个先前的 PLM 探测器中表现更好，其 F1 评分均为 0.85。
所提出的系统在性能上优于或等同于个别专家技师，同时优于先前的自动探测器。因此，该研究验证了睡眠中 LMs 评分的全自动方法。
