MoTiAC: Multi-Objective Actor-Critics for Real-Time Bidding

2020/02/18 by Haolin Zhou, Zhou, Haolin, Chaoqi Yang +9
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Artificial Intelligence (cs.AI) #Auction Theory and Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2002.07408

Accepted in ECML-PKDD 2022. Zhou and Yang made equal contributions

openalex publication_date 2020/02/18 · openalex created_date 2020/02/24 · arxiv created 2022/07/06 · arxiv updated 2022/07/07 · openalex updated_date 2026/07/28

Abstract

Online Real-Time Bidding (RTB) is a complex auction game among which advertisers struggle to bid for ad impressions when a user request occurs. Considering display cost, Return on Investment (ROI), and other influential Key Performance Indicators (KPIs), large ad platforms try to balance the trade-off among various goals in dynamics. To address the challenge, we propose a Multi-ObjecTive Actor-Critics algorithm based on reinforcement learning (RL), named MoTiAC, for the problem of bidding optimization with various goals. In MoTiAC, objective-specific agents update the global network asynchronously with different goals and perspectives, leading to a robust bidding policy. Unlike previous RL models, the proposed MoTiAC can simultaneously fulfill multi-objective tasks in complicated bidding environments. In addition, we mathematically prove that our model will converge to Pareto optimality. Finally, experiments on a large-scale real-world commercial dataset from Tencent verify the effectiveness of MoTiAC versus a set of recent approaches

Citations

• Playing Atari with Deep Reinforcement Learning
• Continuous control with deep reinforcement learning
• Toward negotiable reinforcement learning: shifting priorities in Pareto optimal sequential decision-making
• Hybrid Reward Architecture for Reinforcement Learning
• A Nonparametric Delayed Feedback Model for Conversion Rate Prediction
• Deep Reinforcement Learning for Sponsored Search Real-time Bidding

Related

• Hierarchical Multi-agent Meta-Reinforcement Learning for Cross-channel Bidding
• Bid Optimization using Maximum Entropy Reinforcement Learning
• RTBAgent: A LLM-based Agent System for Real-Time Bidding
• Techniques Toward Optimizing Viewability in RTB Ad Campaigns Using Reinforcement Learning
• Functional Optimization Reinforcement Learning for Real-Time Bidding