ALANNO: An Active Learning Annotation System for Mortals

2022/11/11 by Jukić, Josip, Jelenić, Fran, Bićanić, Miroslav +1
#FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2211.06224

Abstract

Supervised machine learning has become the cornerstone of today's data-driven society, increasing the need for labeled data. However, the process of acquiring labels is often expensive and tedious. One possible remedy is to use active learning (AL) -- a special family of machine learning algorithms designed to reduce labeling costs. Although AL has been successful in practice, a number of practical challenges hinder its effectiveness and are often overlooked in existing AL annotation tools. To address these challenges, we developed ALANNO, an open-source annotation system for NLP tasks equipped with features to make AL effective in real-world annotation projects. ALANNO facilitates annotation management in a multi-annotator setup and supports a variety of AL methods and underlying models, which are easily configurable and extensible.

Related

• EASE: An Easily-Customized Annotation System Powered by Efficiency Enhancement Mechanisms
• Reassessing Active Learning Adoption in Contemporary NLP: A Community Survey
• LLMs in the Loop: Leveraging Large Language Model Annotations for Active Learning in Low-Resource Languages
• Active Learning for NLP with Large Language Models
• LLMaAA: Making Large Language Models as Active Annotators