2018/02/07 by Johannes Wagner, Tobias Baur, Wagner, Johannes +11
Computer Science · Mathematics · #Annotation #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #Artificial intelligence #Coding (social sciences) #Computer science #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Human–machine system #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #Machine learning #Modal #Multimodality #Natural language processing #Session (web analytics) #Task (project management) #Text and Document Classification Technologies #World Wide Web #cs.AI #cs.HC #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.1802.02565
arxiv created 2018/02/07 · openalex publication_date 2018/02/07 · arxiv updated 2018/02/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Scientific disciplines, such as Behavioural Psychology, Anthropology and recently Social Signal Processing are concerned with the systematic exploration of human behaviour. A typical work-flow includes the manual annotation (also called coding) of social signals in multi-modal corpora of considerable size. For the involved annotators this defines an exhausting and time-consuming task. In the article at hand we present a novel method and also provide the tools to speed up the coding procedure. To this end, we suggest and evaluate the use of Cooperative Machine Learning (CML) techniques to reduce manual labelling efforts by combining the power of computational capabilities and human intelligence. The proposed CML strategy starts with a small number of labelled instances and concentrates on predicting local parts first. Afterwards, a session-independent classification model is created to finish the remaining parts of the database. Confidence values are computed to guide the manual inspection and correction of the predictions. To bring the proposed approach into application we introduce NOVA - an open-source tool for collaborative and machine-aided annotations. In particular, it gives labellers immediate access to CML strategies and directly provides visual feedback on the results. Our experiments show that the proposed method has the potential to significantly reduce human labelling efforts.