Changing Model Behavior at Test-Time Using Reinforcement Learning

2017/02/24 by Augustus Odena, Odena, Augustus, Dieterich Lawson +3 · 19 citations
Computer Science · Mathematics · #Artificial intelligence #Computer science #Data Stream Mining Techniques #Deep learning #Inference #MNIST database #Machine Learning and Algorithms #Machine Learning and Data Classification #Machine learning #Phone #Reinforcement learning #Test (biology) #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1702.07780

published in arXiv (Cornell University) (Cornell University) · Submitted to ICLR 2017 Workshop Track

arxiv created 2017/02/24 · openalex publication_date 2017/02/24 · arxiv updated 2017/02/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Machine learning models are often used at test-time subject to constraints and trade-offs not present at training-time. For example, a computer vision model operating on an embedded device may need to perform real-time inference, or a translation model operating on a cell phone may wish to bound its average compute time in order to be power-efficient. In this work we describe a mixture-of-experts model and show how to change its test-time resource-usage on a per-input basis using reinforcement learning. We test our method on a small MNIST-based example.

Cited by

• Exploring Budgeted Image Classification with Content-Sensitive Resource Allocation
• Dynamic Neural Networks: A Survey
• DS-TTS: Zero-Shot Speaker Style Adaptation from Voice Clips via Dynamic Dual-Style Feature Modulation
• Learning Time/Memory-Efficient Deep Architectures with Budgeted Super Networks
• Energy-Aware Deep Learning on Resource-Constrained Hardware

Related

• Local Mixtures of Experts: Essentially Free Test-Time Training via Model Merging
• The Greedy Miser: Learning under Test-time Budgets
• Optimizing Language Models for Inference Time Objectives using Reinforcement Learning
• Reinforcement Learning for Machine Learning Engineering Agents
• A Model-based Approach for Sample-efficient Multi-task Reinforcement Learning