Towards Practical Large-scale Dynamical Heterogeneous Graph Embedding: Cold-start Resilient Recommendation

2025/12/15 by Long, Mabiao, Jiaxi Liu, Liu, Jiaxi +12
Computer Science · #Advanced Graph Neural Networks #FOS: Computer and information sciences #Graph Theory and Algorithms #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Machine Learning in Healthcare

paper · pdf · doi:10.48550/arxiv.2512.13120

openalex publication_date 2025/12/15 · openalex created_date 2025/12/17 · openalex updated_date 2026/07/28

Abstract

Deploying dynamic heterogeneous graph embeddings in production faces key challenges of scalability, data freshness, and cold-start. This paper introduces a practical, two-stage solution that balances deep graph representation with low-latency incremental updates. Our framework combines HetSGFormer, a scalable graph transformer for static learning, with Incremental Locally Linear Embedding (ILLE), a lightweight, CPU-based algorithm for real-time updates. HetSGFormer captures global structure with linear scalability, while ILLE provides rapid, targeted updates to incorporate new data, thus avoiding costly full retraining. This dual approach is cold-start resilient, leveraging the graph to create meaningful embeddings from sparse data. On billion-scale graphs, A/B tests show HetSGFormer achieved up to a 6.11% lift in Advertiser Value over previous methods, while the ILLE module added another 3.22% lift and improved embedding refresh timeliness by 83.2%. Our work provides a validated framework for deploying dynamic graph learning in production environments.

Citations

• SGFormer: Simplifying and Empowering Transformers for Large-Graph Representations
• Towards Better Dynamic Graph Learning: New Architecture and Unified Library
• PolyFormer: Referring Image Segmentation as Sequential Polygon Generation
• DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion
• How Expressive are Transformers in Spectral Domain for Graphs?
• Representing Long-Range Context for Graph Neural Networks with Global Attention
• A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions
• Rethinking Graph Transformers with Spectral Attention
• Mesh Graphormer
• A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sources
• Modelling High-Order Social Relations for Item Recommendation
• Heterogeneous Graph Transformer
• LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation
• Dynamic Graph Representation Learning via Self-Attention Networks
• Heterogeneous Information Network Embedding for Recommendation
• Graph Attention Networks
• Attention Is All You Need
• Inductive Representation Learning on Large Graphs
• Billion-scale similarity search with GPUs
• Semi-Supervised Classification with Graph Convolutional Networks
• node2vec: Scalable Feature Learning for Networks
• DeepWalk

Related

• Graph Embeddings at Scale
• Scalable and Efficient Joint Spiking Embedding Predictive Architecture for Large-Scale Dynamic Graphs
• FILDNE: A Framework for Incremental Learning of Dynamic Networks Embeddings
• DynGEM: Deep Embedding Method for Dynamic Graphs
• Staleness-Alleviated Distributed GNN Training via Online Dynamic-Embedding Prediction