Jayachitra V P, Nivetha S, Nivetha R, Harini R
Department of Computer Technology, MIT campus, Anna University, Chennai, India.
Biomed Signal Process Control. 2021 Sep;70:102960. doi: 10.1016/j.bspc.2021.102960. Epub 2021 Jul 7.
The COVID-19 emerged at the end of 2019 and has become a global pandemic. There are many methods for COVID-19 prediction using a single modality. However, none of them predicts with 100% accuracy, as each individual exhibits varied symptoms for the disease. To decrease the rate of misdiagnosis, multiple modalities can be used for prediction. Besides, there is also a need for a self-diagnosis system to narrow down the risk of virus spread in testing centres. Therefore, we propose a robust IoT and deep learning-based multi-modal data classification method for the accurate prediction of COVID-19. Generally, highly accurate models require deep architectures. In this work, we introduce two lightweight models, namely CovParaNet for audio (cough, speech, breathing) classification and CovTinyNet for image (X-rays, CT scans) classification. These two models were identified as the best unimodal models after comparative analysis with the existing benchmark models. Finally, the obtained results of the five independently trained unimodal models are integrated by a novel dynamic multimodal Random Forest classifier. The lightweight CovParaNet and CovTinyNet models attain a maximum accuracy of 97.45% and 99.19% respectively even with a small dataset. The proposed dynamic multimodal fusion model predicts the final result with 100% accuracy, precision, and recall, and the online retraining mechanism enables it to extend its support even in a noisy environment. Furthermore, the computational complexity of all the unimodal models is minimized tremendously and the system functions effectively with 100% reliability even in the absence of any one of the input modalities during testing.
2019年末出现的新型冠状病毒肺炎已成为全球大流行病。有许多使用单一模态进行新型冠状病毒肺炎预测的方法。然而，由于每个人感染该疾病后表现出的症状各不相同，没有一种方法能达到100%的准确率。为了降低误诊率，可以使用多种模态进行预测。此外，还需要一个自我诊断系统来降低病毒在检测中心传播的风险。因此，我们提出了一种基于物联网和深度学习的强大多模态数据分类方法，用于准确预测新型冠状病毒肺炎。一般来说，高精度模型需要深度架构。在这项工作中，我们引入了两个轻量级模型，即用于音频（咳嗽、语音、呼吸）分类的CovParaNet和用于图像（X光、CT扫描）分类的CovTinyNet。在与现有基准模型进行对比分析后，这两个模型被确定为最佳单模态模型。最后，通过一种新颖的动态多模态随机森林分类器对五个独立训练的单模态模型的结果进行整合。即使在数据集较小的情况下，轻量级的CovParaNet和CovTinyNet模型分别达到了97.45%和99.19%的最高准确率。所提出的动态多模态融合模型以100%的准确率、精确率和召回率预测最终结果，并且在线再训练机制使其即使在嘈杂环境中也能提供支持。此外，所有单模态模型的计算复杂度都大幅降低，并且即使在测试期间缺少任何一种输入模态的情况下，系统也能以100%的可靠性有效运行。
