Reinforcement Learning with Efficient Active Feature Acquisition

2020/11/02 by Haiyan Yin, Yin, Haiyan, Yingzhen Li +7 · 3 citations
Computer Science · #Adaptive Dynamic Programming Control #Artificial Intelligence (cs.AI) #Evolutionary Algorithms and Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2011.00825

openalex publication_date 2020/11/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Solving real-life sequential decision making problems under partial observability involves an exploration-exploitation problem. To be successful, an agent needs to efficiently gather valuable information about the state of the world for making rewarding decisions. However, in real-life, acquiring valuable information is often highly costly, e.g., in the medical domain, information acquisition might correspond to performing a medical test on a patient. This poses a significant challenge for the agent to perform optimally for the task while reducing the cost for information acquisition. In this paper, we propose a model-based reinforcement learning framework that learns an active feature acquisition policy to solve the exploration-exploitation problem during its execution. Key to the success is a novel sequential variational auto-encoder that learns high-quality representations from partially observed states, which are then used by the policy to maximize the task reward in a cost efficient manner. We demonstrate the efficacy of our proposed framework in a control domain as well as using a medical simulator. In both tasks, our proposed method outperforms conventional baselines and results in policies with greater cost efficiency.

Citations

• Playing Atari with Deep Reinforcement Learning
• Auto-Encoding Variational Bayes
• Continuous control with deep reinforcement learning
• SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning
• EDDI: Efficient Dynamic Discovery of High-Value Information with Partial\n VAE

Cited by

• PRECISE-AS: Personalized Reinforcement Learning for Efficient Point-of-Care Echocardiography in Aortic Stenosis Diagnosis
• Stochastic Encodings for Active Feature Acquisition
• NOCTA: Non-Greedy Objective Cost-Tradeoff Acquisition for Longitudinal Data

Related

• Active Measure Reinforcement Learning for Observation Cost Minimization
• Explicit Explore-Exploit Algorithms in Continuous State Spaces
• Active Inference and Reinforcement Learning: A unified inference on continuous state and action spaces under partial observability
• Active Reinforcement Learning Strategies for Offline Policy Improvement
• Sequential Cost-Sensitive Feature Acquisition