DeepFM: A Factorization-Machine based Neural Network for CTR Prediction

2017/03/13 by Huifeng Guo, Ruiming Tang, Guo, Huifeng +7 · 219 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Image Retrieval and Classification Techniques #Recommender Systems and Techniques #cs.CL #cs.IR

paper · pdf · doi:10.48550/arxiv.1703.04247

arxiv created 2017/03/13 · arxiv updated 2017/03/14

Abstract

Learning sophisticated feature interactions behind user behaviors is critical in maximizing CTR for recommender systems. Despite great progress, existing methods seem to have a strong bias towards low- or high-order interactions, or require expertise feature engineering. In this paper, we show that it is possible to derive an end-to-end learning model that emphasizes both low- and high-order feature interactions. The proposed model, DeepFM, combines the power of factorization machines for recommendation and deep learning for feature learning in a new neural network architecture. Compared to the latest Wide & Deep model from Google, DeepFM has a shared input to its "wide" and "deep" parts, with no need of feature engineering besides raw features. Comprehensive experiments are conducted to demonstrate the effectiveness and efficiency of DeepFM over the existing models for CTR prediction, on both benchmark data and commercial data.

Citations

• Deep Learning over Multi-field Categorical Data: A Case Study on User Response Prediction

Cited by

• SpecFormer: Mitigating Embedding and Attention Collapse via Spectral-Aware Transformer for Recommendation
• Tokenizing Numerical and Embedding Features for LLM RecSys
• RoleMix: Unifying Sequential and Non-Sequential Features via Semantic Tokenization for Post-Click Conversion Rate Prediction
• Bumblebee: Interleaved Mixed-Layer Building Blocks for Large-Scale Recommendation Systems
• TextBridgeGNN: Pre-training Graph Neural Network for Cross-Domain Recommendation via Text-Guided Transfer
• Embedding Samples Dispatching for Recommendation Model Training in Edge Environments
• Selective LLM-Guided Regularization for Enhancing Recommendation Models
• ReaSeq: Unleashing World Knowledge via Reasoning for Sequential Modeling
• LinkedOut: Linking World Knowledge Representation Out of Video LLM for Next-Generation Video Recommendation
• From Feature Interaction to Feature Generation: A Generative Paradigm of CTR Prediction Models
• No One Left Behind: How to Exploit the Incomplete and Skewed Multi-Label Data for Conversion Rate Prediction
• Automated Information Flow Selection for Multi-scenario Multi-task Recommendation
• xGR: Efficient Generative Recommendation Serving at Scale
• LLM-Empowered Representation Learning for Emerging Item Recommendation
• NeurIDA: Dynamic Modeling for Effective In-Database Analytics
• Exploring Test-time Scaling via Prediction Merging on Large-Scale Recommendation
• A Probabilistic Framework for Temporal Distribution Generalization in Industry-Scale Recommender Systems
• STORE: Semantic Tokenization, Orthogonal Rotation and Efficient Attention for Scaling Up Ranking Models
• AIF: Asynchronous Inference Framework for Cost-Effective Pre-Ranking
• Infer As You Train: A Symmetric Paradigm of Masked Generative for Click-Through Rate Prediction
• A Soft-partitioned Semi-supervised Collaborative Transfer Learning Approach for Multi-Domain Recommendation
• From Scaling to Structured Expressivity: Rethinking Transformers for CTR Prediction
• Adaptive Regularization for Large-Scale Sparse Feature Embedding Models
• Time Matters: A Novel Real-Time Long- and Short-term User Interest Model for Click-Through Rate Prediction
• Deep Retrieval: Learning A Retrievable Structure for Large-Scale Recommendations
• DKN: Deep Knowledge-Aware Network for News Recommendation
• NOVA: A Verification-Aware Agent Harness for Architecture Evolution in Industrial Recommender Systems
• Multi-Graph Convolution Collaborative Filtering
• Collaborative Similarity Embedding for Recommender Systems
• Learning Effective and Efficient Embedding via an Adaptively-Masked Twins-based Layer
• DeepCF: A Unified Framework of Representation Learning and Matching Function Learning in Recommender System
• MM-Rec: Multimodal News Recommendation
• Operation-aware Neural Networks for User Response Prediction
• Edge-Cloud Collaboration Enabled Video Service Enhancement: A Hybrid Human-Artificial Intelligence Scheme
• Deep Interest Network for Click-Through Rate Prediction
• Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems
• Beyond Action Imitation: Learning a Decision-Aware User Simulator for Online Advertising
• Quaternion Factorization Machines: A Lightweight Solution to Intricate Feature Interaction Modelling
• FELA: A Multi-Agent Evolutionary System for Feature Engineering of Industrial Event Log Data
• MiniOneRec: An Open-Source Framework for Scaling Generative Recommendation
• MMbeddings: Parameter-Efficient, Low-Overfitting Probabilistic Embeddings Inspired by Nonlinear Mixed Models
• Large-scale User Game Lifecycle Representation Learning
• An Efficient Framework for Whole-Page Reranking via Single-Modal Supervision
• Cross-Scenario Unified Modeling of User Interests at Billion Scale
• Beyond Static LLM Policies: Imitation-Enhanced Reinforcement Learning for Recommendation
• Controllable Multi-Interest Framework for Recommendation
• Online Interaction Detection for Click-Through Rate Prediction
• Decoupled Multimodal Fusion for User Interest Modeling in Click-Through Rate Prediction
• HoMer: Addressing Heterogeneities by Modeling Sequential and Set-wise Contexts for CTR Prediction
• Hierarchical LoRA MoE for Efficient CTR Model Scaling
• PANTHER: Generative Pretraining Beyond Language for Sequential User Behavior Modeling
• Hierarchical Semantic RL: Tackling the Problem of Dynamic Action Space for RL-based Recommendations
• MATT-CTR: Unleashing a Model-Agnostic Test-Time Paradigm for CTR Prediction with Confidence-Guided Inference Paths
• On Predicting Post-Click Conversion Rate via Counterfactual Inference
• The Role of Feature Interactions in Graph-based Tabular Deep Learning
• TriMat: Context-aware Recommendation by Tri-Matrix Factorization
• Harnessing Deep Learning in Searching Wild Relatives of Domestic Animals
• MTRec: Learning to Align with User Preferences via Mental Reward Models
• Can Increasing Input Dimensionality Improve Deep Reinforcement Learning?
• Evaluating and Pricing Advertisements in AI-Generated Responses
• Implicit Feedback Deep Collaborative Filtering Product Recommendation System
• Broad Learning for Healthcare
• Scaling Law for Recommendation Models: Towards General-purpose User Representations
• FLAME: A Serving System Optimized for Large-Scale Generative Recommendation with Efficiency
• Federated Meta-Learning with Fast Convergence and Efficient Communication
• Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems
• An Efficient Neighborhood-based Interaction Model for Recommendation on Heterogeneous Graph
• Learning Effective Representations for Person-Job Fit by Feature Fusion
• AEFS: Adaptive Early Feature Selection for Deep Recommender Systems
• VQL: An End-to-End Context-Aware Vector Quantization Attention for Ultra-Long User Behavior Modeling
• Privacy-Aware Recommender Systems Challenge on Twitter's Home Timeline
• End-to-End User Behavior Retrieval in Click-Through RatePrediction Model
• Neural News Recommendation with Attentive Multi-View Learning
• asLLR: LLM based Leads Ranking in Auto Sales
• ELEC: Efficient Large Language Model-Empowered Click-Through Rate Prediction
• MEGG: Replay via Maximally Extreme GGscore in Incremental Learning for Neural Recommendation Models
• Tackling Device Data Distribution Real-time Shift via Prototype-based Parameter Editing
• ORCA: Mitigating Over-Reliance for Multi-Task Dwell Time Prediction with Causal Decoupling
• Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterest
• OneSearch: A Preliminary Exploration of the Unified End-to-End Generative Framework for E-commerce Search
• M3TQA: Massively Multilingual Multitask Table Question Answering
• ACA-Net: Future Graph Learning for Logistical Demand-Supply Forecasting
• Spacetime-GR: A Spacetime-Aware Generative Model for Large Scale Online POI Recommendation
• An End-to-End Neighborhood-based Interaction Model for Knowledge-enhanced Recommendation
• UPRec: User-Aware Pre-training for Recommender Systems
• Modeling Long-term User Behaviors with Diffusion-driven Multi-interest Network for CTR Prediction
• DMGIN: How Multimodal LLMs Enhance Large Recommendation Models for Lifelong User Post-click Behaviors
• Graph-Based Feature Augmentation for Predictive Tasks on Relational Datasets
• GCN-Based User Representation Learning for Unifying Robust Recommendation and Fraudster Detection
• TrackRec: Iterative Alternating Feedback with Chain-of-Thought via Preference Alignment for Recommendation
• PCR-CA: Parallel Codebook Representations with Contrastive Alignment for Multiple-Category App Recommendation
• Exploring Scaling Laws of CTR Model for Online Performance Improvement
• DGenCTR: Towards a Universal Generative Paradigm for Click-Through Rate Prediction via Discrete Diffusion
• Global-Distribution Aware Scenario-Specific Variational Representation Learning Framework
• Distribution-Guided Auto-Encoder for User Multimodal Interest Cross Fusion
• ENCODE: Breaking the Trade-Off Between Performance and Efficiency in Long-Term User Behavior Modeling
• Heterogeneous Influence Maximization in User Recommendation
• A Neural Influence Diffusion Model for Social Recommendation
• Forecasting Clicks in Digital Advertising: Multimodal Inputs and Interpretable Outputs
• Personalized Product Search Ranking: A Multi-Task Learning Approach with Tabular and Non-Tabular Data
• LLM Empowered Prototype Learning for Zero and Few-Shot Tasks on Tabular Data
• An Intelligent Group Event Recommendation System in Social networks
• Selection and Exploitation of High-Quality Knowledge from Large Language Models for Recommendation
• Synthesize, Retrieve, and Propagate: A Unified Predictive Modeling Framework for Relational Databases
• MISS: Multi-Interest Self-Supervised Learning Framework for Click-Through Rate Prediction
• Graph Federated Learning for Personalized Privacy Recommendation
• KuaiLive: A Real-time Interactive Dataset for Live Streaming Recommendation
• Bidding-Aware Retrieval for Multi-Stage Consistency in Online Advertising
• G-UBS: Towards Robust Understanding of Implicit Feedback via Group-Aware User Behavior Simulation
• CTR-Sink: Attention Sink for Language Models in Click-Through Rate Prediction
• Dynamic Forgetting and Spatio-Temporal Periodic Interest Modeling for Local-Life Service Recommendation
• Training with Multi-Layer Embeddings for Model Reduction
• Random Offset Block Embedding Array (ROBE) for CriteoTB Benchmark MLPerf DLRM Model : 1000× Compression and 3.1× Faster Inference
• Res-embedding for Deep Learning Based Click-Through Rate Prediction Modeling
• Large-scale Causal Approaches to Debiasing Post-click Conversion Rate Estimation with Multi-task Learning
• Recommendation with Attribute-aware Product Networks: A Representation Learning Model
• Co-NAML-LSTUR: A Combined Model with Attentive Multi-View Learning and Long- and Short-term User Representations for News Recommendation
• An Efficient Agreement Mechanism in CapsNets By Pairwise Product
• Scaling Recommender Transformers to One Billion Parameters
• Personalized Adaptive Meta Learning for Cold-start User Preference Prediction
• DeepFM: An End-to-End Wide & Deep Learning Framework for CTR Prediction
• Image Matters: Visually modeling user behaviors using Advanced Model Server
• Memory-Efficient Factorization Machines via Binarizing both Data and Model Coefficients
• IP2: Entity-Guided Interest Probing for Personalized News Recommendation
• A Comprehensive Review on Harnessing Large Language Models to Overcome Recommender System Challenges
• Learnable Embedding Sizes for Recommender Systems
• MetaSelector: Meta-Learning for Recommendation with User-Level Adaptive Model Selection
• DeepLight: Deep Lightweight Feature Interactions for Accelerating CTR Predictions in Ad Serving
• Graph Factorization Machines for Cross-Domain Recommendation
• Explainable Student Performance Prediction With Personalized Attention for Explaining Why A Student Fails
• Generative Click-through Rate Prediction with Applications to Search Advertising
• Scenario-aware and Mutual-based approach for Multi-scenario Recommendation in E-Commerce
• Improving Conversational Recommendation System by Pretraining on Billions Scale of Knowledge Graph
• AutoRec: An Automated Recommender System
• FedCTR: Federated Native Ad CTR Prediction with Multi-Platform User Behavior Data
• Non-parametric Graph Convolution for Re-ranking in Recommendation Systems
• Towards High Supervised Learning Utility Training Data Generation: Data Pruning and Column Reordering
• CDC: Causal Domain Clustering for Multi-Domain Recommendation
• PaletteID: Prototype-Composed Semantic Identifiers for Multimodal CTR Prediction
• TayFCS: Towards Light Feature Combination Selection for Deep Recommender Systems
• Deep Learning for Click-Through Rate Estimation
• VRAgent-R1: Boosting Video Recommendation with MLLM-based Agents via Reinforcement Learning
• Practical Privacy Preserving POI Recommendation
• Reward Balancing Revisited: Enhancing Offline Reinforcement Learning for Recommender Systems
• RecLLM-R1: A Two-Stage Training Paradigm with Reinforcement Learning and Chain-of-Thought v1
• MoE-MLoRA for Multi-Domain CTR Prediction: Efficient Adaptation with Expert Specialization
• Deep Session Interest Network for Click-Through Rate Prediction
• Conditional Generative Modeling for Enhanced Credit Risk Management in Supply Chain Finance
• Similarity = Value? Consultation Value Assessment and Alignment for Personalized Search
• One Person, One Model, One World: Learning Continual User Representation without Forgetting
• OneRec Technical Report
• Click-Through Rate Prediction Using Graph Neural Networks and Online Learning
• PURS: Personalized Unexpected Recommender System for Improving User Satisfaction
• Large Scalable Cross-Domain Graph Neural Networks for Personalized Notification at LinkedIn
• AutoEmb: Automated Embedding Dimensionality Search in Streaming Recommendations
• Deep Collaborative Filtering with Multi-Aspect Information in Heterogeneous Networks
• Multimodal Topic Learning for Video Recommendation
• Explicit Semantic Cross Feature Learning via Pre-trained Graph Neural Networks for CTR Prediction
• TabFlex: Scaling Tabular Learning to Millions with Linear Attention
• FaCTR: Factorized Channel-Temporal Representation Transformers for Efficient Time Series Forecasting
• Pairwise Interactive Graph Attention Network for Context-Aware Recommendation
• Enhancing Factorization Machines with Generalized Metric Learning
• Multi-Interest-Aware User Modeling for Large-Scale Sequential Recommendations
• ContextNet: A Click-Through Rate Prediction Framework Using Contextual information to Refine Feature Embedding
• Dynamic Memory based Attention Network for Sequential Recommendation
• IP-Dialog: Evaluating Implicit Personalization in Dialogue Systems with Synthetic Data
• Open-Set Living Need Prediction with Large Language Models
• AliBoost: Ecological Boosting Framework in Alibaba Platform
• Breaker: Removing Shortcut Cues with User Clustering for Single-slot Recommendation System
• SHORING: Design Provable Conditional High-Order Interaction Network via Symbolic Testing
• Spectrum Attention Mechanism for Time Series Classification
• BiBLDR: Bidirectional Behavior Learning for Drug Repositioning
• 1^st Place Solution of WWW 2025 EReL@MIR Workshop Multimodal CTR Prediction Challenge
• CPR: Understanding and Improving Failure Tolerant Training for Deep Learning Recommendation with Partial Recovery
• Attribute-aware Collaborative Filtering: Survey and Classification
• Hierarchically Modeling Micro and Macro Behaviors via Multi-Task Learning for Conversion Rate Prediction
• GateNet: Gating-Enhanced Deep Network for Click-Through Rate Prediction
• Hierarchical Tree Search-based User Lifelong Behavior Modeling on Large Language Model
• Measure Domain's Gap: A Similar Domain Selection Principle for Multi-Domain Recommendation
• C³DRec: Cloud-Client Cooperative Deep Learning for Temporal Recommendation in the Post-GDPR Era
• Denoising User-aware Memory Network for Recommendation
• Joint Neural Collaborative Filtering for Recommender Systems
• Revisiting Feature Interactions from the Perspective of Quadratic Neural Networks for Click-through Rate Prediction
• MaskNet: Introducing Feature-Wise Multiplication to CTR Ranking Models by Instance-Guided Mask
• Adaptive Serverless Learning
• ThinkRec: Thinking-based recommendation via LLM
• Cross-Modal Alignment with Mixture Experts Neural Network for Intral-City Retail Recommendation
• Hierarchical Temporal Convolutional Networks for Dynamic Recommender Systems
• Field Matters: A lightweight LLM-enhanced Method for CTR Prediction
• MultiTab: A Comprehensive Benchmark Suite for Multi-Dimensional Evaluation in Tabular Domains
• MC²-SF: Slow-Fast Learning for Mobile-Cloud Collaborative Recommendation
• SecEmb: Sparsity-Aware Secure Federated Learning of On-Device Recommender System with Large Embedding
• DSETA: A Dual-Stage Continual Learning Framework for Travel Time Prediction in Dynamic Traffic Environments
• Information Theoretic Counterfactual Learning from Missing-Not-At-Random Feedback
• AutoRAC: Automated Processing-in-Memory Accelerator Design for Recommender Systems
• Sybil-based Virtual Data Poisoning Attacks in Federated Learning
• DARLR: Dual-Agent Offline Reinforcement Learning for Recommender Systems with Dynamic Reward
• Field-aware Neural Factorization Machine for Click-Through Rate Prediction
• Dynamic Sequential Graph Learning for Click-Through Rate Prediction
• Entire Space Multi-Task Modeling via Post-Click Behavior Decomposition for Conversion Rate Prediction
• Towards Scheduling Federated Deep Learning using Meta-Gradients for Inter-Hospital Learning
• AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems
• De-conflating Preference and Qualification: Constrained Dual-Perspective Reasoning for Job Recommendation with Large Language Models
• Efficient Click-Through Rate Prediction for Developing Countries via Tabular Learning
• Semantics-Aware Denoising: A PLM-Guided Sample Reweighting Strategy for Robust Recommendation
• Kunlun: Establishing Scaling Laws for Massive-Scale Recommendation Systems through Unified Architecture Design
• Parameter-Efficient Transfer from Sequential Behaviors for User Modeling and Recommendation
• CPM-sensitive AUC for CTR prediction
• Chromatic Learning for Sparse Datasets
• Killing Two Birds with One Stone: Unifying Retrieval and Ranking with a Single Generative Recommendation Model
• Matching Algorithms: Fundamentals, Applications and Challenges
• GAIN: Graph Attention & Interaction Network for Inductive Semi-Supervised Learning over Large-scale Graphs
• Matrix Factorization with Dynamic Multi-view Clustering for Recommender System
• Distillation based Multi-task Learning: A Candidate Generation Model for Improving Reading Duration
• Deep Neural Networks and Tabular Data: A Survey
• Representation Learning for Tabular Data: A Comprehensive Survey
• FROG: Effective Friend Recommendation in Online Games via Modality-aware User Preferences
• On the Practice of Deep Hierarchical Ensemble Network for Ad Conversion Rate Prediction
• CHIME: A Compressive Framework for Holistic Interest Modeling

Related

• DeepFM: An End-to-End Wide & Deep Learning Framework for CTR Prediction
• Memorize, Factorize, or be Naïve: Learning Optimal Feature Interaction Methods for CTR Prediction
• Field-Embedded Factorization Machines for Click-through rate prediction
• AdnFM: An Attentive DenseNet based Factorization Machine for CTR Prediction
• Towards Deeper, Lighter and Interpretable Cross Network for CTR Prediction