Policy Design for Active Sequential Hypothesis Testing using Deep\n Learning

2018/10/11 by Dhruva Kartik, Ekraam Sabir, Kartik, Dhruva +5 · 3 citations
Computer Science · Mathematics · #Artificial Intelligence (cs.AI) #Artificial intelligence #Bellman equation #Computer science #Distributed Sensor Networks and Detection Algorithms #FOS: Computer and information sciences #FOS: Electrical engineering #FOS: Mathematics #Heuristic #Heuristics #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning and Algorithms #Machine learning #Markov chain #Markov decision process #Markov model #Markov process #Mathematical optimization #Mathematics #Optimization and Search Problems #Partially observable Markov decision process #Reinforcement Learning in Robotics #Reinforcement learning #Statistics Theory (math.ST) #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.1810.04859

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2018/10/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

Information theory has been very successful in obtaining performance limits\nfor various problems such as communication, compression and hypothesis testing.\nLikewise, stochastic control theory provides a characterization of optimal\npolicies for Partially Observable Markov Decision Processes (POMDPs) using\ndynamic programming. However, finding optimal policies for these problems is\ncomputationally hard in general and thus, heuristic solutions are employed in\npractice. Deep learning can be used as a tool for designing better heuristics\nin such problems. In this paper, the problem of active sequential hypothesis\ntesting is considered. The goal is to design a policy that can reliably infer\nthe true hypothesis using as few samples as possible by adaptively selecting\nappropriate queries. This problem can be modeled as a POMDP and bounds on its\nvalue function exist in literature. However, optimal policies have not been\nidentified and various heuristics are used. In this paper, two new heuristics\nare proposed: one based on deep reinforcement learning and another based on a\nKL-divergence zero-sum game. These heuristics are compared with\nstate-of-the-art solutions and it is demonstrated using numerical experiments\nthat the proposed heuristics can achieve significantly better performance than\nexisting methods in some scenarios.\n

Citations

• Multilayer feedforward networks are universal approximators
• Training feedforward networks with the Marquardt algorithm
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation

Cited by

• Control-Optimized Deep Reinforcement Learning for Artificially Intelligent Autonomous Systems
• Searching for a Hidden Markov Anomaly over Multiple Processes
• Sequential Experiment Design for Hypothesis Verification

Related

• Active hypothesis testing in unknown environments using recurrent neural networks and model free reinforcement learning
• Anomaly Detection via Learning-Based Sequential Controlled Sensing
• Sequential Experiment Design for Hypothesis Verification
• Deep anytime-valid hypothesis testing
• Active Sequential Hypothesis Testing with Non-Homogeneous Costs