2021/11/17 by Junyan Cheng, Cheng, Junyan, Iordanis Fostiropoulos +3 · 6 citations
Computer Science · #Abstract syntax #Abstract syntax tree #Advanced Malware Detection Techniques #Artificial intelligence #Automatic summarization #Computer science #FOS: Computer and information sciences #Graph #Identifier #Machine Learning (cs.LG) #Natural language processing #Programming Languages (cs.PL) #Programming language #Software Engineering Research #Source code #Syntax #Theoretical computer science #Topic Modeling #Transformer #cs.LG #cs.PL
paper · pdf · doi:10.48550/arxiv.2111.08874
published in arXiv (Cornell University) (Cornell University)
arxiv created 2021/11/17 · openalex publication_date 2021/11/17 · arxiv updated 2021/11/18 · openalex created_date 2021/11/22 · openalex updated_date 2026/08/05
As opposed to natural languages, source code understanding is influenced by grammatical relationships between tokens regardless of their identifier name. Graph representations of source code such as Abstract Syntax Tree (AST) can capture relationships between tokens that are not obvious from the source code. We propose a novel method, GN-Transformer to learn end-to-end on a fused sequence and graph modality we call Syntax-Code-Graph (SCG). GN-Transformer expands on Graph Networks (GN) framework using a self-attention mechanism. SCG is the result of the early fusion between a source code snippet and the AST representation. We perform experiments on the structure of SCG, an ablation study on the model design, and the hyper-parameters to conclude that the performance advantage is from the fused representation. The proposed methods achieve state-of-the-art performance in two code summarization datasets and across three automatic code summarization metrics (BLEU, METEOR, ROUGE-L). We further evaluate the human perceived quality of our model and previous work with an expert-user study. Our model outperforms the state-of-the-art in human perceived quality and accuracy.