Multilevel Inter-modal and Intra-modal Transformer network with domain adversarial learning for multimodal sleep staging.

Sleep staging identification is a fundamental task for the diagnosis of sleep disorders. With the development of biosensing technology and deep learning technology, it is possible to automatically decode sleep process through electroencephalogram signals. However, most sleep staging methods do not consider multimodal sleep signals such as electroencephalogram and electrooculograms signals simultaneously for sleep staging due to the limitation of performance improvement. To this regard, we design a Multilevel Inter-modal and Intra-modal Transformer network with domain adversarial learning for multimodal sleep staging, we introduce a multilevel Transformer structure to fully capture the temporal dependencies within sleep signals of each modality and the interdependencies among different modalities. Simultaneously, we strive for the multi-scale CNNs to learn time and frequency features separately. Our research promotes the application of Transformer models in the field of sleep staging identification. Moreover, considering individual differences among subjects, models trained on one group's data often perform poorly when applied to another group, known as the domain generalization problem. While domain adaptation methods are commonly used, fine-tuning on the target domain each time is cumbersome and impractical. To effectively address these issues without using target domain information, we introduce domain adversarial learning to help the model learn domain-invariant features for better generalization across domains. We validated the superiority of our model on two commonly used datasets, significantly outperforming other baseline models. Our model efficiently extracts dependencies of intra-modal level and inter-modal level from multimodal sleep data, making it suitable for scenarios requiring high accuracy.