2020/10/23 by David Bieber, Bieber, David, Charles Sutton +5 · 2 citations
Computer Science · Engineering · Materials Science · #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Machine Learning (cs.LG) #Machine Learning in Materials Science #Software Engineering Research
paper · pdf · doi:10.48550/arxiv.2010.12621
openalex publication_date 2020/10/23 · openalex created_date 2022/10/01 · openalex updated_date 2026/07/28
Graph neural networks (GNNs) have emerged as a powerful tool for learning\nsoftware engineering tasks including code completion, bug finding, and program\nrepair. They benefit from leveraging program structure like control flow\ngraphs, but they are not well-suited to tasks like program execution that\nrequire far more sequential reasoning steps than number of GNN propagation\nsteps. Recurrent neural networks (RNNs), on the other hand, are well-suited to\nlong sequential chains of reasoning, but they do not naturally incorporate\nprogram structure and generally perform worse on the above tasks. Our aim is to\nachieve the best of both worlds, and we do so by introducing a novel GNN\narchitecture, the Instruction Pointer Attention Graph Neural Networks\n(IPA-GNN), which achieves improved systematic generalization on the task of\nlearning to execute programs using control flow graphs. The model arises by\nconsidering RNNs operating on program traces with branch decisions as latent\nvariables. The IPA-GNN can be seen either as a continuous relaxation of the RNN\nmodel or as a GNN variant more tailored to execution. To test the models, we\npropose evaluating systematic generalization on learning to execute using\ncontrol flow graphs, which tests sequential reasoning and use of program\nstructure. More practically, we evaluate these models on the task of learning\nto execute partial programs, as might arise if using the model as a heuristic\nfunction in program synthesis. Results show that the IPA-GNN outperforms a\nvariety of RNN and GNN baselines on both tasks.\n