Scaling Multimodal Search and Recommendation with Small Language Models via Upside-Down Reinforcement Learning

2025/02/14 by Yuchen Lin, Sanat Sharma, Lin, Yu-Chen +9 · 2 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2502.09854

openalex publication_date 2025/02/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this work, we investigate how small language models (SLMs) can be scaled to support multimodal search and recommendation use cases while remaining efficient enough for real-time, resource-constrained deployments. We present a framework that combines upside-down reinforcement learning with synthetic data distillation from a large language model (Llama-3) to train a 100M-parameter GPT-2 model for multitask prompt generation. Despite being up to 80 times smaller than state-of-the-art large language models (LLMs), our SLM achieves relevance and diversity scores within 6% of competitive baselines such as Llama-3 8B, Qwen3 8B, and Ministral 8B. These results demonstrate that SLMs can effectively handle multimodal search and recommendation tasks, while dramatically reducing inference latency and memory overhead. Our study highlights the potential of lightweight models as practical engines for scalable multimodal discovery, bridging the gap between cutting-edge research and real-world multimodal applications such as media recommendations and creative content generation.

Cited by

• Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges
• A Survey on Collaborative Mechanisms Between Large and Small Language Models

Related

• Multimodal Large Language Models with Adaptive Preference Optimization for Sequential Recommendation
• Scaling Down, Serving Fast: Compressing and Deploying Efficient LLMs for Recommendation Systems
• Rec-R1: Bridging Generative Large Language Models and User-Centric Recommendation Systems via Reinforcement Learning
• LlamaRec: Two-Stage Recommendation using Large Language Models for Ranking
• SEM: Reinforcement Learning for Search-Efficient Large Language Models