2016/10/04 by Kush R. Varshney, Varshney, Kush R., Homa Alemzadeh +1 · 7 citations
Computer Science · Decision Sciences · #Adversarial Robustness in Machine Learning #Risk and Safety Analysis #Explainable Artificial Intelligence (XAI)
paper · pdf · doi:10.48550/arxiv.1610.01256
Machine learning algorithms increasingly influence our decisions and interact\nwith us in all parts of our daily lives. Therefore, just as we consider the\nsafety of power plants, highways, and a variety of other engineered\nsocio-technical systems, we must also take into account the safety of systems\ninvolving machine learning. Heretofore, the definition of safety has not been\nformalized in a machine learning context. In this paper, we do so by defining\nmachine learning safety in terms of risk, epistemic uncertainty, and the harm\nincurred by unwanted outcomes. We then use this definition to examine safety in\nall sorts of applications in cyber-physical systems, decision sciences, and\ndata products. We find that the foundational principle of modern statistical\nmachine learning, empirical risk minimization, is not always a sufficient\nobjective. Finally, we discuss how four different categories of strategies for\nachieving safety in engineering, including inherently safe design, safety\nreserves, safe fail, and procedural safeguards can be mapped to a machine\nlearning context. We then discuss example techniques that can be adopted in\neach category, such as considering interpretability and causality of predictive\nmodels, objective functions beyond expected prediction accuracy, human\ninvolvement for labeling difficult or rare examples, and user experience design\nof software and open data.\n