Department of Computer Science, Faculty of Electronics and Information Technologies, Sumy State University, Sumy, Ukraine.
Department of Computer Systems, Network and Cybersecurity, Faculty of Radio-Electronics, Computer Systems and Infocommunications, National Aerospace University "KhAI", Kharkiv, Ukraine.
Front Public Health. 2024 Mar 27;12:1342937. doi: 10.3389/fpubh.2024.1342937. eCollection 2024.
The healthcare sector demands a higher degree of responsibility, trustworthiness, and accountability when implementing Artificial Intelligence (AI) systems. Machine learning operations (MLOps) for AI-based medical diagnostic systems are primarily focused on aspects such as data quality and confidentiality, bias reduction, model deployment, performance monitoring, and continuous improvement. However, so far, MLOps techniques do not take into account the need to provide resilience to disturbances such as adversarial attacks, including fault injections, and drift, including out-of-distribution. This article is concerned with the MLOps methodology that incorporates the steps necessary to increase the resilience of an AI-based medical diagnostic system against various kinds of disruptive influences.
resilience optimization, predictive uncertainty calibration, uncertainty monitoring, and graceful degradation are incorporated as additional stages in MLOps. To optimize the resilience of the AI based medical diagnostic system, additional components in the form of adapters and meta-adapters are utilized. These components are fine-tuned during meta-training based on the results of adaptation to synthetic disturbances. Furthermore, an additional model is introduced for calibration of predictive uncertainty. This model is trained using both in-distribution and out-of-distribution data to refine predictive confidence during the inference mode.
The structure of resilience-aware MLOps for medical diagnostic systems has been proposed. Experimentally confirmed increase of robustness and speed of adaptation for medical image recognition system during several intervals of the system's life cycle due to the use of resilience optimization and uncertainty calibration stages. The experiments were performed on the DermaMNIST dataset, BloodMNIST and PathMNIST. ResNet-18 as a representative of convolutional networks and MedViT-T as a representative of visual transformers are considered. It is worth noting that transformers exhibited lower resilience than convolutional networks, although this observation may be attributed to potential imperfections in the architecture of adapters and meta-adapters.
СONCLUSION: The main novelty of the suggested resilience-aware MLOps methodology and structure lie in the separating possibilities and activities on creating a basic model for normal operating conditions and ensuring its resilience and trustworthiness. This is significant for the medical applications as the developer of the basic model should devote more time to comprehending medical field and the diagnostic task at hand, rather than specializing in system resilience. Resilience optimization increases robustness to disturbances and speed of adaptation. Calibrated confidences ensure the recognition of a portion of unabsorbed disturbances to mitigate their impact, thereby enhancing trustworthiness.
在实施人工智能 (AI) 系统时，医疗保健领域需要更高的责任感、可信度和问责制。基于人工智能的医疗诊断系统的机器学习操作 (MLOps) 主要侧重于数据质量和保密性、减少偏差、模型部署、性能监控和持续改进等方面。然而，到目前为止，MLOps 技术并未考虑到提供对干扰（如对抗性攻击、故障注入和漂移，包括分布外）的弹性的需求。本文关注的是将增加基于人工智能的医疗诊断系统对各种干扰的弹性的步骤纳入 MLOps 方法学中。
弹性优化、预测不确定性校准、不确定性监控和优雅降级被纳入 MLOps 作为附加阶段。为了优化基于人工智能的医疗诊断系统的弹性，使用适配器和元适配器的形式增加了额外的组件。这些组件在基于对合成干扰的适应结果进行元训练期间进行微调。此外，引入了一个用于预测不确定性校准的附加模型。该模型使用分布内和分布外数据进行训练，以在推理模式下细化预测置信度。
提出了用于医疗诊断系统的弹性感知 MLOps 结构。通过在系统生命周期的几个间隔内使用弹性优化和不确定性校准阶段，实验证实了医学图像识别系统的稳健性和适应速度的提高。实验在 DermaMNIST 数据集、BloodMNIST 和 PathMNIST 上进行。考虑了 ResNet-18 作为卷积网络的代表和 MedViT-T 作为视觉转换器的代表。值得注意的是，尽管这一观察结果可能归因于适配器和元适配器架构中的潜在缺陷，但与卷积网络相比，转换器的弹性较低。
所提出的弹性感知 MLOps 方法学和结构的主要新颖之处在于，为正常运行条件下的基本模型创建可能性和活动与确保其弹性和可信度分开。这对于医疗应用非常重要，因为基本模型的开发人员应该花更多的时间来理解医学领域和手头的诊断任务，而不是专门研究系统的弹性。弹性优化提高了对干扰的稳健性和适应速度。校准的置信度确保识别未吸收的干扰部分，以减轻其影响，从而提高可信度。
