Light Differentiable Logic Gate Networks

2025/09/26 by Lukas Rüttgers, Rüttgers, Lukas, Till Aczel +5
Engineering · #Advanced Optical Network Technologies #FOS: Computer and information sciences #Machine Learning (cs.LG) #Optical Network Technologies #Performance (cs.PF) #Semiconductor Lasers and Optical Devices

paper · pdf · doi:10.48550/arxiv.2510.03250

openalex publication_date 2025/09/26 · openalex created_date 2025/10/09 · openalex updated_date 2026/07/28

Abstract

Differentiable logic gate networks (DLGNs) exhibit extraordinary efficiency at inference while sustaining competitive accuracy. But vanishing gradients, discretization errors, and high training cost impede scaling these networks. Even with dedicated parameter initialization schemes from subsequent works, increasing depth still harms accuracy. We show that the root cause of these issues lies in the underlying parametrization of logic gate neurons themselves. To overcome this issue, we propose a reparametrization that also shrinks the parameter size logarithmically in the number of inputs per gate. For binary inputs, this already reduces the model size by 4x, speeds up the backward pass by up to 1.86x, and converges in 8.5x fewer training steps. On top of that, we show that the accuracy on CIFAR-100 remains stable and sometimes superior to the original parametrization.

Citations

• Recurrent Deep Differentiable Logic Gate Networks
• NeuraLUT-Assemble: Hardware-aware Assembling of Sub-Neural Networks for Efficient LUT Inference
• PolyLUT: Ultra-low Latency Polynomial Inference with Hardware-Aware Structured Pruning
• Convolutional Differentiable Logic Gate Networks
• Deep Differentiable Logic Gate Networks
• LogicNets: Co-Designed Neural Networks and Circuits for Extreme-Throughput Applications
• Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers
• Deep Residual Learning for Image Recognition
• Deep learning in neural networks: An overview
• ImageNet classification with deep convolutional neural networks
• Analysis of Boolean Functions
• Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position

Related

• Convolutional Differentiable Logic Gate Networks
• Mind the Gap: Removing the Discretization Gap in Differentiable Logic Gate Networks
• Deep Differentiable Logic Gate Networks
• A Method for Optimizing Connections in Differentiable Logic Gate Networks
• On the Depth Scalability of Logic Gate Networks