THGFM: Dual-Branch Temporal Heterogeneous Graph Fusion Model

2026/07/29 by Yixin Peng, Diego Collarana, Er Jin +1
Computer Science · Mathematics · #cs.LG #cs.CL #stat.ML

paper · pdf

arxiv created 2026/07/29 · arxiv updated 2026/07/31

Abstract

Temporal heterogeneous graphs offer a natural abstraction for dynamic relational systems in which diverse node and relation types co-exist and evolve over time. Learning on such graphs requires jointly modeling cross-type structural heterogeneity and the temporal dynamics of interactions, yet existing methods still struggle to reconcile parameter-efficient cross-type transfer with relation-aware specialization, and typically inject time only as additive features outside the attention kernel. We propose THGFM, a web-scale temporal heterogeneous graph fusion model that addresses both limitations within a unified dual-path architecture. THGFM couples a Shared-Space Temporal Attention branch for parameter-efficient cross-type transfer with a Relational Type-Partitioned Temporal Attention branch for relation-aware specialization, and integrates them through Dual-Path Relational--Shared Fusion, instantiated with Type-Conditioned Non-Competitive Gated Sum Fusion: a adaptive mechanism that assigns independent, type-conditioned feature-wise gates to the shared and specialized branches, allowing both to be amplified or suppressed without zero-sum competition. To directly incorporate relative time into the attention score, THGFM further introduces Rotary Temporal Attention, which rotates queries and keys by half-phases of relative time before matching. THGFM consistently outperforms baseline graph transformer models on academic graphs benchmarks, delivering a +3.25% six-task mean gain, with peak relative gains of +12.37% on OAG-CS PV, +4.87% on PF-L₂, and +1.18% on PF-L₁, and +4.24%, +3.73%, and +4.61% on OGBN-MAG, HTAG-ArXiv, and HTAG-DBLP, respectively.

Citations

• Multi-Scale Heterogeneous Text-Attributed Graph Datasets From Diverse Domains
• Heta: Distributed Training of Heterogeneous Graph Neural Networks
• CTRL: Continuous-Time Representation Learning on Temporal Heterogeneous Information Network
• A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challenges
• Simple and Efficient Heterogeneous Graph Neural Network
• Heterogeneous Temporal Graph Neural Network
• Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks
• RoFormer: Enhanced Transformer with Rotary Position Embedding
• Temporal Graph Networks for Deep Learning on Dynamic Graphs
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• Knowledge Graphs
• Heterogeneous Graph Transformer
• Inductive Representation Learning on Temporal Graphs
• An Attention-based Graph Neural Network for Heterogeneous Structural Learning
• Composition-based Multi-Relational Graph Convolutional Networks
• XLNet: Generalized Autoregressive Pretraining for Language Understanding
• Modeling Relational Data with Graph Convolutional Networks

Related

• Heterogeneous Temporal Graph Neural Network
• T-GRAB: A Synthetic Diagnostic Benchmark for Learning on Temporal Graphs
• BG-HGNN: Toward Efficient Learning for Complex Heterogeneous Graphs
• TGM: a Modular and Efficient Library for Machine Learning on Temporal Graphs
• Heterogeneous Temporal Hypergraph Neural Network