Faddi Zakaria, da Mata Karen, Silva Priscila, Nagaraju Vidhyashree, Ghosh Susmita, Kul Gokhan, Fiondella Lance
Department of Electrical and Computer Engineering, University of Massachusetts Dartmouth, Dartmouth, Massachusetts, USA.
Department of Computer Science, Stonehill College, Easton, Massachusetts, USA.
Risk Anal. 2025 Apr;45(4):790-807. doi: 10.1111/risa.14666. Epub 2024 Jul 23.
Advances in machine learning (ML) have led to applications in safety-critical domains, including security, defense, and healthcare. These ML models are confronted with dynamically changing and actively hostile conditions characteristic of real-world applications, requiring systems incorporating ML to be reliable and resilient. Many studies propose techniques to improve the robustness of ML algorithms. However, fewer consider quantitative techniques to assess changes in the reliability and resilience of these systems over time. To address this gap, this study demonstrates how to collect relevant data during the training and testing of ML suitable for the application of software reliability, with and without covariates, and resilience models and the subsequent interpretation of these analyses. The proposed approach promotes quantitative risk assessment of ML technologies, providing the ability to track and predict degradation and improvement in the ML model performance and assisting ML and system engineers with an objective approach to compare the relative effectiveness of alternative training and testing methods. The approach is illustrated in the context of an image recognition model, which is subjected to two generative adversarial attacks and then iteratively retrained to improve the system's performance. Our results indicate that software reliability models incorporating covariates characterized the misclassification discovery process more accurately than models without covariates. Moreover, the resilience model based on multiple linear regression incorporating interactions between covariates tracks and predicts degradation and recovery of performance best. Thus, software reliability and resilience models offer rigorous quantitative assurance methods for ML-enabled systems and processes.
机器学习（ML）的进展已使其在安全关键领域得到应用，包括安全、国防和医疗保健。这些机器学习模型面临着现实世界应用中动态变化且充满敌意的条件，这就要求包含机器学习的系统具备可靠性和弹性。许多研究提出了提高机器学习算法鲁棒性的技术。然而，较少有人考虑用定量技术来评估这些系统的可靠性和弹性随时间的变化。为了填补这一空白，本研究展示了如何在机器学习的训练和测试过程中收集适用于软件可靠性应用的数据，包括有无协变量的情况，以及弹性模型和对这些分析的后续解读。所提出的方法促进了对机器学习技术的定量风险评估，能够跟踪和预测机器学习模型性能的退化和提升，并协助机器学习和系统工程师以一种客观的方法来比较替代训练和测试方法的相对有效性。该方法在一个图像识别模型的背景下进行了说明，该模型遭受了两次生成对抗攻击，然后进行迭代重新训练以提高系统性能。我们的结果表明，包含协变量的软件可靠性模型比不包含协变量的模型更准确地刻画了误分类发现过程。此外，基于包含协变量之间相互作用的多元线性回归的弹性模型能最好地跟踪和预测性能的退化和恢复。因此，软件可靠性和弹性模型为基于机器学习的系统和流程提供了严格的定量保证方法。
