2016/02/29 by Aviv Tamar, Yi Wu, Garrett Thomas +2 · 1 citation
Computer Science · Mathematics · #cs.AI #cs.LG #cs.NE #stat.ML
published as Advances in Neural Information Processing Systems 29 pages 2154--2162, 2016 · Fixed missing table values
arxiv created 2017/03/20 · arxiv updated 2017/03/22
We introduce the value iteration network (VIN): a fully differentiable neural network with a `planning module' embedded within. VINs can learn to plan, and are suitable for predicting outcomes that involve planning-based reasoning, such as policies for reinforcement learning. Key to our approach is a novel differentiable approximation of the value-iteration algorithm, which can be represented as a convolutional neural network, and trained end-to-end using standard backpropagation. We evaluate VIN based policies on discrete and continuous path-planning domains, and on a natural-language based search task. We show that by learning an explicit planning computation, VIN policies generalize better to new, unseen domains.