Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Control

2025/12/01 by Fabian Kresse, Christoph H. Lampert, Kresse, Fabian +1
Computer Science · Engineering · #Advanced Memory and Neural Computing #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Hardware Architecture (cs.AR) #Machine Learning (cs.LG) #Neural Networks and Reservoir Computing #Symbolic Computation (cs.SC)

paper · pdf · doi:10.48550/arxiv.2512.01467

openalex publication_date 2025/12/01 · openalex created_date 2025/12/03 · openalex updated_date 2026/07/28

Abstract

Controlling autonomous systems under real-world conditions often requires policies that can be evaluated with low latency and minimal energy consumption. Unfortunately, these conditions are at odds with the use of high-precision deep neural networks as controllers. In this work, we introduce Differentiable Weightless Controllers (DWCs), a symbolic-differentiable architecture that learns flexible, non-linear, yet highly efficient control policies. DWCs can be trained end-to-end via gradient-based techniques, yet compile directly into FPGA-compatible circuits with few- or even single-clock-cycle latency and nanojoule-level energy cost per action. Across five MuJoCo benchmarks, including high-dimensional Humanoid, DWCs achieve returns competitive with standard deep policies (full-precision or quantized neural networks). Furthermore, DWCs exhibit structurally sparse and interpretable connectivity patterns, enabling direct inspection of which input values influence control decisions.

Citations

• LILogic Net: Compact Logic Gate Networks with Learnable Connectivity for Efficient Hardware Deployment
• Learning Quantized Continuous Controllers for Integer Hardware
• Rapid Inference of Logic Gate Neural Networks for Anomaly Detection in High Energy Physics
• WARP-LUTs -- Walsh-Assisted Relaxation for Probabilistic Look Up Tables
• Recurrent Deep Differentiable Logic Gate Networks
• A Method for Optimizing Connections in Differentiable Logic Gate Networks
• Scalable Interconnect Learning in Boolean Networks
• Differentiable Logic Cellular Automata: From Game of Life to Pattern Generation
• Logic Gate Neural Networks are Good for Verification
• High-Performance Reinforcement Learning on Spot: Optimizing Simulation Parameters with Distributional Measures
• Convolutional Differentiable Logic Gate Networks
• Towards Narrowing the Generalization Gap in Deep Boolean Networks
• The Impact of Quantization and Pruning on Deep Reinforcement Learning Models
• Learning Interpretable Differentiable Logic Networks
• Neural Network Compression for Reinforcement Learning Tasks
• Deep Differentiable Logic Gate Networks
• The State of Sparse Training in Deep Reinforcement Learning
• RLx2: Training a Sparse Deep Reinforcement Learning Model from Scratch
• Deep Binary Reinforcement Learning for Scalable Verification
• Binarized P-Network: Deep Reinforcement Learning of Robot Control from Raw Images on FPGA
• A Survey of Quantization Methods for Efficient Neural Network Inference
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Neurosymbolic Reinforcement Learning with Formally Verified Exploration
• What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Study
• Optuna: A Next-generation Hyperparameter Optimization Framework
• Verifiable Reinforcement Learning via Policy Extraction
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Proximal Policy Optimization Algorithms
• Attention Is All You Need
• Benchmarking Deep Reinforcement Learning for Continuous Control
• Playing Atari with Deep Reinforcement Learning
• ImageNet classification with deep convolutional neural networks

Related

• Towards Generalization and Simplicity in Continuous Control
• Learning Quantized Continuous Controllers for Integer Hardware
• Deep Differentiable Logic Gate Networks
• Learning with Differentiable Algorithms
• Convolutional Differentiable Logic Gate Networks