2019/07/17 by Alexander William Wong, Wong, Alexander William, Amir Salimi +5 · 1 citation
Computer Science · #FOS: Computer and information sciences #Online Learning and Analytics #Software Engineering (cs.SE) #Software Engineering Research
paper · pdf · doi:10.48550/arxiv.1907.07803
openalex publication_date 2019/07/17 · openalex created_date 2022/07/28 · openalex updated_date 2026/07/28
One problem when studying how to find and fix syntax errors is how to get\nnatural and representative examples of syntax errors. Most syntax error\ndatasets are not free, open, and public, or they are extracted from novice\nprogrammers and do not represent syntax errors that the general population of\ndevelopers would make. Programmers of all skill levels post questions and\nanswers to Stack Overflow which may contain snippets of source code along with\ncorresponding text and tags. Many snippets do not parse, thus they are ripe for\nforming a corpus of syntax errors and corrections. Our primary contribution is\nan approach for extracting natural syntax errors and their corresponding human\nmade fixes to help syntax error research. A Python abstract syntax tree parser\nis used to determine preliminary errors and corrections on code blocks\nextracted from the SOTorrent data set. We further analyzed our code by\nexecuting the corrections in a Python interpreter. We applied our methodology\nto produce a public data set of 62,965 Python Stack Overflow code snippets with\ncorresponding tags, errors, and stack traces. We found that errors made by\nStack Overflow users do not match errors made by student developers or random\nmutations, implying there is a serious representativeness risk within the\nfield. Finally we share our dataset openly so that future researchers can\nre-use and extend our syntax errors and fixes.\n