STACC: Code Comment Classification using SentenceTransformers

2023/02/25 by Ali Al-Kaswan, Maliheh Izadi, Al-Kaswan, Ali +3 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Software Engineering (cs.SE) #Software Engineering Research #Software Reliability and Analysis Research

paper · pdf · doi:10.48550/arxiv.2302.13149

openalex publication_date 2023/02/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Code comments are a key resource for information about software artefacts. Depending on the use case, only some types of comments are useful. Thus, automatic approaches to classify these comments have been proposed. In this work, we address this need by proposing, STACC, a set of SentenceTransformers-based binary classifiers. These lightweight classifiers are trained and tested on the NLBSE Code Comment Classification tool competition dataset, and surpass the baseline by a significant margin, achieving an average F1 score of 0.74 against the baseline of 0.31, which is an improvement of 139%. A replication package, as well as the models themselves, are publicly available.

Cited by

• Evaluating the Performance and Efficiency of Sentence-BERT for Code Comment Classification

Related

• Performance Comparison of Binary Machine Learning Classifiers in Identifying Code Comment Types: An Exploratory Study
• Dopamin: Transformer-based Comment Classifiers through Domain Post-Training and Multi-level Layer Aggregation
• Enhancing Binary Code Comment Quality Classification: Integrating Generative AI for Improved Accuracy
• NeSy is alive and well: A LLM-driven symbolic approach for better code comment data generation and classification
• Optimizing Deep Learning Models to Address Class Imbalance in Code Comment Classification