2022/10/28 by Anastasia Drozdova, Drozdova, Anastasia, Polina Guseva +7 · 1 citation
Computer Science · #Advanced Malware Detection Techniques #FOS: Computer and information sciences #Machine Learning and Data Classification #Software Engineering (cs.SE) #Software Engineering Research
paper · pdf · doi:10.48550/arxiv.2210.16018
openalex publication_date 2022/10/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Program code as a data source is gaining popularity in the data science community. Possible applications for models trained on such assets range from classification for data dimensionality reduction to automatic code generation. However, without annotation number of methods that could be applied is somewhat limited. To address the lack of annotated datasets, we present the Code4ML corpus. It contains code snippets, task summaries, competitions and dataset descriptions publicly available from Kaggle - the leading platform for hosting data science competitions. The corpus consists of ~2.5 million snippets of ML code collected from ~100 thousand Jupyter notebooks. A representative fraction of the snippets is annotated by human assessors through a user-friendly interface specially designed for that purpose. Code4ML dataset can potentially help address a number of software engineering or data science challenges through a data-driven approach. For example, it can be helpful for semantic code classification, code auto-completion, and code generation for an ML task specified in natural language.