Kapalaga Geofrey, Kivunike Florence N, Kerfua Susan, Jjingo Daudi, Biryomumaisho Savino, Rutaisire Justus, Ssajjakambwe Paul, Mugerwa Swidiq, Abbey Seguya, Aaron Mulindwa H, Kiwala Yusuf
Department of Information Technology, College of Computing and Information Sciences, Makerere University, Kampala, Uganda.
Department of Vaccinology, National Livestock Resources Research Institute, Kampala, Uganda.
Front Artif Intell. 2024 Nov 1;7:1455331. doi: 10.3389/frai.2024.1455331. eCollection 2024.
Foot-and-mouth disease poses a significant threat to both domestic and wild cloven-hoofed animals, leading to severe economic losses and jeopardizing food security. While machine learning models have become essential for predicting foot-and-mouth disease outbreaks, their effectiveness is often compromised by distribution shifts between training and target datasets, especially in non-stationary environments. Despite the critical impact of these shifts, their implications in foot-and-mouth disease outbreak prediction have been largely overlooked. This study introduces the Calibrated Uncertainty Prediction approach, designed to enhance the performance of Random Forest models in predicting foot-and-mouth disease outbreaks across varying distributions. The Calibrated Uncertainty Prediction approach effectively addresses distribution shifts by calibrating uncertain instances for pseudo-label annotation, allowing the active learner to generalize more effectively to the target domain. By utilizing a probabilistic calibration model, Calibrated Uncertainty Prediction pseudo-annotates the most informative instances, refining the active learner iteratively and minimizing the need for human annotation and outperforming existing methods known to mitigate distribution shifts. This reduces costs, saves time, and lessens the dependence on domain experts while achieving outstanding predictive performance. The results demonstrate that Calibrated Uncertainty Prediction significantly enhances predictive performance in non-stationary environments, achieving an accuracy of 98.5%, Area Under the Curve of 0.842, recall of 0.743, precision of 0.855, and an F1 score of 0.791. These findings underscore Calibrated Uncertainty Prediction's ability to overcome the vulnerabilities of existing ML models, offering a robust solution for foot-and-mouth disease outbreak prediction and contributing to the broader field of predictive modeling in infectious disease management.
口蹄疫对家养和野生偶蹄类动物都构成了重大威胁，会导致严重的经济损失并危及粮食安全。虽然机器学习模型已成为预测口蹄疫疫情的关键，但由于训练数据集和目标数据集之间的分布变化，尤其是在非平稳环境中，其有效性常常受到影响。尽管这些变化具有重大影响，但它们在口蹄疫疫情预测中的影响在很大程度上被忽视了。本研究引入了校准不确定性预测方法，旨在提高随机森林模型在预测不同分布情况下口蹄疫疫情的性能。校准不确定性预测方法通过校准用于伪标签标注的不确定实例来有效应对分布变化，使主动学习器能够更有效地推广到目标领域。通过使用概率校准模型，校准不确定性预测方法对信息量最大的实例进行伪标注，迭代优化主动学习器，并最大限度地减少对人工标注的需求，其性能优于已知的减轻分布变化的现有方法。这降低了成本、节省了时间并减少了对领域专家的依赖，同时实现了出色的预测性能。结果表明，校准不确定性预测在非平稳环境中显著提高了预测性能，准确率达到98.5%，曲线下面积为0.842，召回率为0.743，精确率为0.855，F1分数为0.791。这些发现突出了校准不确定性预测克服现有机器学习模型弱点的能力，为口蹄疫疫情预测提供了一个强大的解决方案，并为传染病管理中更广泛的预测建模领域做出了贡献。
