Controlling Neural Networks with Rule Representations

2021/06/14 by Sungyong Seo, Sercan Ö. Arık, Seo, Sungyong +11 · 1 voice
Computer Science · Mathematics · #Adversarial Robustness in Machine Learning #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2106.07804

openalex publication_date 2021/06/14 · arxiv published 2021/06/14 · arxiv updated 2021/11/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose a novel training method that integrates rules into deep learning, in a way the strengths of the rules are controllable at inference. Deep Neural Networks with Controllable Rule Representations (DeepCTRL) incorporates a rule encoder into the model coupled with a rule-based objective, enabling a shared representation for decision making. DeepCTRL is agnostic to data type and model architecture. It can be applied to any kind of rule defined for inputs and outputs. The key aspect of DeepCTRL is that it does not require retraining to adapt the rule strength -- at inference, the user can adjust it based on the desired operation point on accuracy vs. rule verification ratio. In real-world domains where incorporating rules is critical -- such as Physics, Retail and Healthcare -- we show the effectiveness of DeepCTRL in teaching rules for deep learning. DeepCTRL improves the trust and reliability of the trained models by significantly increasing their rule verification ratio, while also providing accuracy gains at downstream tasks. Additionally, DeepCTRL enables novel use cases such as hypothesis testing of the rules on data samples, and unsupervised adaptation based on shared rules between datasets.

Citations

• Harnessing Deep Neural Networks with Logic Rules
• Towards A Rigorous Science of Interpretable Machine Learning
• Boolean Decision Rules via Column Generation
• Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning
• One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques
• Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data
• Newton versus the machine: solving the chaotic three-body problem using deep neural networks
• Underspecification Presents Challenges for Credibility in Modern Machine Learning

Discussions

• Controlling Neural Networks with Rule Representations [hn, 2 points, 0 comments]

Related

• TUTOR: Training Neural Networks Using Decision Rules as Model Priors
• Neurosymbolic Methods for Rule Mining
• Deep Explainable Learning with Graph Based Data Assessing and Rule Reasoning
• Harnessing Deep Neural Networks with Logic Rules
• An Empirical Investigation into Deep and Shallow Rule Learning