A Semantic Loss Function for Deep Learning with Symbolic Knowledge

2017/11/29 by Jingyi Xu, Xu, Jingyi, Zilu Zhang +7 · 20 citations
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Logic in Computer Science (cs.LO) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning and Data Classification #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1711.11157

openalex publication_date 2017/11/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper develops a novel methodology for using symbolic knowledge in deep learning. From first principles, we derive a semantic loss function that bridges between neural output vectors and logical constraints. This loss function captures how close the neural network is to satisfying the constraints on its output. An experimental evaluation shows that it effectively guides the learner to achieve (near-)state-of-the-art results on semi-supervised multi-class classification. Moreover, it significantly increases the ability of the neural network to predict structured objects, such as rankings and paths. These discrete concepts are tremendously difficult to learn, and benefit from a tight integration of deep learning and symbolic reasoning methods.

Citations

• On probabilistic inference by weighted model counting
• ImageNet classification with deep convolutional neural networks
• Neural Programmer-Interpreters
• Harnessing Deep Neural Networks with Logic Rules
• Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
• Imposing Hard Constraints on Deep Networks: Promises and Limitations
• Adversarial Sets for Regularising Neural Link Predictors
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Right for the Right Reasons: Avoiding Reasoning Shortcuts via Prototypical Neurosymbolic AI
• When Rule Violations Are Rare: Chimera Training for Logical Anomaly Detection
• Logical GANs: Adversarial Learning through Ehrenfeucht Fraisse Games
• Attack logics, not outputs: Towards efficient robustification of deep neural networks by falsifying concept-based properties
• "Stop replacing salt with sugar!'': Towards Intuitive Human-Agent Teaching
• Logic of Hypotheses: from Zero to Full Knowledge in Neurosymbolic Integration
• Understanding and Improving Adversarial Robustness of Neural Probabilistic Circuits
• Neuro-Symbolic Frameworks: Conceptual Characterization and Empirical Comparative Analysis
• Constraints-Guided Diffusion Reasoner for Neuro-Symbolic Learning
• Feasibility Guaranteed Learning-to-Optimize in Wireless Communication Resource Allocation
• The Need for Verification in AI-Driven Scientific Discovery
• T-ILR: a Neurosymbolic Integration for LTLf
• Scaling Neuro-symbolic Problem Solving: Solver-Free Learning of Constraints and Objectives
• Tuning Random Generators: Property-Based Testing as Probabilistic Programming
• Neuro-Symbolic Artificial Intelligence: Towards Improving the Reasoning Abilities of Large Language Models
• The DeepLog Neurosymbolic Machine
• Adma: A Flexible Loss Function for Neural Networks
• When Is Prior Knowledge Helpful? Exploring the Evaluation and Selection of Unsupervised Pretext Tasks from a Neuro-Symbolic Perspective
• Probabilistic Circuits for Knowledge Graph Completion with Reduced Rule Sets
• Categorical Construction of Logically Verifiable Neural Architectures

Related

• Semantic Loss Functions for Neuro-Symbolic Structured Prediction
• Semantic Objective Functions: A distribution-aware method for adding logical constraints in deep learning
• Symbolic Techniques for Deep Learning: Challenges and Opportunities
• Semantic Strengthening of Neuro-Symbolic Learning
• Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoning