A Comprehensive Drift-Adaptive Framework for Sustaining Model Performance in COVID-19 Detection From Dynamic Cough Audio Data: Model Development and Validation.

BACKGROUND

The COVID-19 pandemic has highlighted the need for robust and adaptable diagnostic tools capable of detecting the disease from diverse and continuously evolving data sources. Machine learning models, particularly convolutional neural networks, are promising in this regard. However, the dynamic nature of real-world data can lead to model drift, where the model's performance degrades over time, as the underlying data distribution changes due to evolving disease characteristics, demographic shifts, and variations in recording conditions. Addressing this challenge is crucial to maintaining the accuracy and reliability of these models in ongoing diagnostic applications.

OBJECTIVE

This study aims to develop a comprehensive framework that not only monitors model drift over time but also uses adaptation mechanisms to mitigate performance fluctuations in COVID-19 detection models trained on dynamic cough audio data.

METHODS

Two crowdsourced COVID-19 audio datasets, namely COVID-19 Sounds and Coswara, were used for development and evaluation purposes. Each dataset was divided into 2 distinct periods, namely the development period and postdevelopment period. A baseline convolutional neural network model was initially trained and evaluated using data (ie, coughs from COVID-19 Sounds and shallow coughs from Coswara dataset) from the development period. To detect changes in data distributions and the model's performance between these periods, the maximum mean discrepancy distance was used. Upon detecting significant drift, a retraining procedure was triggered to update the baseline model. The study explored 2 model adaptation approaches, unsupervised domain adaptation and active learning, both of which were comparatively assessed.

RESULTS

The baseline model achieved an area under the receiver operating characteristic curve of 69.13% and a balanced accuracy of 63.38% on the development test set of the COVID-19 Sounds dataset, while for the Coswara dataset, the corresponding values were 66.8% and 61.64%. A decline in performance was observed when the model was evaluated on data from the postdevelopment period, indicating the presence of model drift. The application of the unsupervised domain adaptation approach led to performance improvement in terms of balanced accuracy by up to 22% and 24% for the COVID-19 Sounds and Coswara datasets, respectively. The active learning approach yielded even greater improvement, corresponding to a balanced accuracy increase of up to 30% and 60% for the 2 datasets, respectively.

CONCLUSIONS

The proposed framework successfully addresses the challenge of model drift in COVID-19 detection by enabling continuous adaptation to evolving data distributions. This approach ensures sustained model performance over time, contributing to the development of robust and adaptable diagnostic tools for COVID-19 and potentially other infectious diseases.