2019/07/09 by Nicholas Collins, N. Collins, Hanna Kurniawati +2
Computer Science · Mathematics · #Adversarial Robustness in Machine Learning #Algorithm #Architecture #Artificial Intelligence (cs.AI) #Artificial intelligence #Artificial neural network #Computer science #FOS: Computer and information sciences #Key (lock) #Machine Learning (cs.LG) #Machine Learning and Algorithms #Machine learning #Markov chain #Markov decision process #Markov model #Markov process #Mathematics #Observable #Partially observable Markov decision process #Reinforcement Learning in Robotics #Robotics (cs.RO) #Space (punctuation) #State (computer science) #State space #System dynamics #cs.AI #cs.LG #cs.RO
paper · pdf · doi:10.48550/arxiv.1907.04457
published in arXiv (Cornell University) (Cornell University)
arxiv created 2019/07/09 · openalex publication_date 2019/07/09 · arxiv updated 2019/07/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
We propose a neural network architecture, called TransNet, that combines planning and model learning for solving Partially Observable Markov Decision Processes (POMDPs) with non-uniform system dynamics. The past decade has seen a substantial advancement in solving POMDP problems. However, constructing a suitable POMDP model remains difficult. Recently, neural network architectures have been proposed to alleviate the difficulty in acquiring such models. Although the results are promising, existing architectures restrict the type of system dynamics that can be learned --that is, system dynamics must be the same in all parts of the state space. TransNet relaxes such a restriction. Key to this relaxation is a novel neural network module that classifies the state space into classes and then learns the system dynamics of the different classes. TransNet uses this module together with the overall architecture of QMDP-Net[1] to allow solving POMDPs that have more expressive dynamic models, while maintaining efficient data requirement. Its evaluation on typical benchmarks in robot navigation with initially unknown system and environment models indicates that TransNet substantially out-performs the quality of the generated policies and learning efficiency of the state-of-the-art method QMDP-Net.