2022/07/07 by Tsegaye Misikir Tashu, Tashu, Tsegaye Misikir, Sara Fattouh +5 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Text and Document Classification Technologies
paper · pdf · doi:10.48550/arxiv.2207.03305
openalex publication_date 2022/07/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In this work, we present a multi-modal model for commercial product classification, that combines features extracted by multiple neural network models from textual (CamemBERT and FlauBERT) and visual data (SE-ResNeXt-50), using simple fusion techniques. The proposed method significantly outperformed the unimodal models' performance and the reported performance of similar models on our specific task. We did experiments with multiple fusing techniques and found, that the best performing technique to combine the individual embedding of the unimodal network is based on combining concatenation and averaging the feature vectors. Each modality complemented the shortcomings of the other modalities, demonstrating that increasing the number of modalities can be an effective method for improving the performance of multi-label and multimodal classification problems.