1 Department of Data Science, IBM Thomas J. Watson Research Center , Yorktown Heights, New York.
2 Department of Electrical and Computer Engineering, University of Virginia , Charlottesville, Virginia.
Big Data. 2017 Sep;5(3):246-255.
Machine learning algorithms increasingly influence our decisions and interact with us in all parts of our daily lives. Therefore, just as we consider the safety of power plants, highways, and a variety of other engineered socio-technical systems, we must also take into account the safety of systems involving machine learning. Heretofore, the definition of safety has not been formalized in a machine learning context. In this article, we do so by defining machine learning safety in terms of risk, epistemic uncertainty, and the harm incurred by unwanted outcomes. We then use this definition to examine safety in all sorts of applications in cyber-physical systems, decision sciences, and data products. We find that the foundational principle of modern statistical machine learning, empirical risk minimization, is not always a sufficient objective. We discuss how four different categories of strategies for achieving safety in engineering, including inherently safe design, safety reserves, safe fail, and procedural safeguards can be mapped to a machine learning context. We then discuss example techniques that can be adopted in each category, such as considering interpretability and causality of predictive models, objective functions beyond expected prediction accuracy, human involvement for labeling difficult or rare examples, and user experience design of software and open data.
机器学习算法越来越多地影响我们的决策，并在我们日常生活的各个方面与我们互动。因此，正如我们考虑发电厂、高速公路和各种其他工程社会技术系统的安全性一样，我们还必须考虑涉及机器学习的系统的安全性。迄今为止，机器学习语境中的安全性定义尚未形式化。在本文中，我们通过根据风险、认识不确定性以及意外结果造成的伤害来定义机器学习安全性。然后，我们使用此定义来检查网络物理系统、决策科学和数据产品中的各种应用程序中的安全性。我们发现，现代统计机器学习的基础原则，经验风险最小化，并不总是一个充分的目标。我们讨论了如何将工程中实现安全性的四种不同策略类别（包括固有安全设计、安全储备、安全失效和程序保护）映射到机器学习上下文中。然后，我们讨论了每个类别中可以采用的示例技术，例如考虑预测模型的可解释性和因果关系、超越预期预测精度的目标函数、人工参与标记困难或罕见示例以及软件和开放数据的用户体验设计。
