2019/10/27 by Mina Basirat, Basirat, Mina, Peter M. Roth +1 · 1 citation
Computer Science · #Advanced Image and Video Retrieval Techniques #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Multimodal Machine Learning Applications
paper · pdf · doi:10.48550/arxiv.1910.12259
openalex publication_date 2019/10/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Deep neural networks paved the way for significant improvements in image\nvisual categorization during the last years. However, even though the tasks are\nhighly varying, differing in complexity and difficulty, existing solutions\nmostly build on the same architectural decisions. This also applies to the\nselection of activation functions (AFs), where most approaches build on\nRectified Linear Units (ReLUs). In this paper, however, we show that the choice\nof a proper AF has a significant impact on the classification accuracy, in\nparticular, if fine, subtle details are of relevance. Therefore, we propose to\nmodel the degree of absence and the presence of features via the AF by using\npiece-wise linear functions, which we refer to as L*ReLU. In this way, we can\nensure the required properties, while still inheriting the benefits in terms of\ncomputational efficiency from ReLUs. We demonstrate our approach for the task\nof Fine-grained Visual Categorization (FGVC), running experiments on seven\ndifferent benchmark datasets. The results do not only demonstrate superior\nresults but also that for different tasks, having different characteristics,\ndifferent AFs are selected.\n