2024/01/29 by Van Thuy Hoang, Hoang, Van Thuy, O‐Joun Lee +1 · 2 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · #Advanced Graph Neural Networks #Artificial Intelligence (cs.AI) #DNA and Biological Computing #FOS: Computer and information sciences #Graph Theory and Algorithms #Machine Learning (cs.LG)
paper · pdf · doi:10.48550/arxiv.2401.16176
openalex publication_date 2024/01/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The transformer architecture has shown remarkable success in various domains, such as natural language processing and computer vision. When it comes to graph learning, transformers are required not only to capture the interactions between pairs of nodes but also to preserve graph structures connoting the underlying relations and proximity between them, showing the expressive power to capture different graph structures. Accordingly, various structure-preserving graph transformers have been proposed and widely used for various tasks, such as graph-level tasks in bioinformatics and chemoinformatics. However, strategies related to graph structure preservation have not been well organized and systematized in the literature. In this paper, we provide a comprehensive overview of structure-preserving graph transformers and generalize these methods from the perspective of their design objective. First, we divide strategies into four main groups: node feature modulation, context node sampling, graph rewriting, and transformer architecture improvements. We then further divide the strategies according to the coverage and goals of graph structure preservation. Furthermore, we also discuss challenges and future directions for graph transformer models to preserve the graph structure and understand the nature of graphs.