2018/12/04 by Jonathan Uesato, Uesato, Jonathan, Ananya Kumar +15 · 4 citations
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Explainable Artificial Intelligence (XAI)
paper · pdf · doi:10.48550/arxiv.1812.01647
This paper addresses the problem of evaluating learning systems in safety\ncritical domains such as autonomous driving, where failures can have\ncatastrophic consequences. We focus on two problems: searching for scenarios\nwhen learned agents fail and assessing their probability of failure. The\nstandard method for agent evaluation in reinforcement learning, Vanilla Monte\nCarlo, can miss failures entirely, leading to the deployment of unsafe agents.\nWe demonstrate this is an issue for current agents, where even matching the\ncompute used for training is sometimes insufficient for evaluation. To address\nthis shortcoming, we draw upon the rare event probability estimation literature\nand propose an adversarial evaluation approach. Our approach focuses evaluation\non adversarially chosen situations, while still providing unbiased estimates of\nfailure probabilities. The key difficulty is in identifying these adversarial\nsituations -- since failures are rare there is little signal to drive\noptimization. To solve this we propose a continuation approach that learns\nfailure modes in related but less robust agents. Our approach also allows reuse\nof data already collected for training the agent. We demonstrate the efficacy\nof adversarial evaluation on two standard domains: humanoid control and\nsimulated driving. Experimental results show that our methods can find\ncatastrophic failures and estimate failures rates of agents multiple orders of\nmagnitude faster than standard evaluation schemes, in minutes to hours rather\nthan days.\n