2021/07/28 by Mattia Medina Grespan, Grespan, Mattia Medina, Ashim Gupta +3
Computer Science · #Artificial Intelligence (cs.AI) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2107.13646
openalex publication_date 2021/07/28 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28
Symbolic knowledge can provide crucial inductive bias for training neural\nmodels, especially in low data regimes. A successful strategy for incorporating\nsuch knowledge involves relaxing logical statements into sub-differentiable\nlosses for optimization. In this paper, we study the question of how best to\nrelax logical expressions that represent labeled examples and knowledge about a\nproblem; we focus on sub-differentiable t-norm relaxations of logic. We present\ntheoretical and empirical criteria for characterizing which relaxation would\nperform best in various scenarios. In our theoretical study driven by the goal\nof preserving tautologies, the Lukasiewicz t-norm performs best. However, in\nour empirical analysis on the text chunking and digit recognition tasks, the\nproduct t-norm achieves best predictive performance. We analyze this apparent\ndiscrepancy, and conclude with a list of best practices for defining loss\nfunctions via logic.\n