HTMA-Net: Towards Multiplication-Avoiding Neural Networks via Hadamard Transform and In-Memory Computing

2025/09/27 by Hamdan, Emadeldeen, Cetin, Ahmet Enis
#Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2509.23103

Abstract

Reducing the cost of multiplications is critical for efficient deep neural network deployment, especially in energy-constrained edge devices. In this work, we introduce HTMA-Net, a novel framework that integrates the Hadamard Transform (HT) with multiplication-avoiding (MA) SRAM-based in-memory computing to reduce arithmetic complexity while maintaining accuracy. Unlike prior methods that only target multiplications in convolutional layers or focus solely on in-memory acceleration, HTMA-Net selectively replaces intermediate convolutions with Hybrid Hadamard-based transform layers whose internal convolutions are implemented via multiplication-avoiding in-memory operations. We evaluate HTMA-Net on ResNet-18 using CIFAR-10, CIFAR-100, and Tiny ImageNet, and provide a detailed comparison against regular, MF-only, and HT-only variants. Results show that HTMA-Net eliminates up to 52% of multiplications compared to baseline ResNet-18, ResNet-20, and ResNet-32 models, while achieving comparable accuracy in evaluation and significantly reducing computational complexity and the number of parameters. Our results demonstrate that combining structured Hadamard transform layers with SRAM-based in-memory computing multiplication-avoiding operators is a promising path towards efficient deep learning architectures.

Citations

• Combating the Memory Walls: Optimization Pathways for Long-Context Agentic LLM Inference
• A Probabilistic Hadamard U-Net for MRI Bias Field Correction
• A Hybrid Quantum-Classical Approach based on the Hadamard Transform for the Convolutional Layer
• Block Walsh-Hadamard Transform Based Binary Layers in Deep Neural Networks
• Fast Walsh-Hadamard Transform and Smooth-Thresholding Based Binary Layers in Deep Neural Networks
• MF-Net: Compute-In-Memory SRAM for Multibit Precision Inference using Memory-immersed Data Conversion and Multiplication-free Operators
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Energy Efficient Hadamard Neural Networks
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural\n Networks
• Deep Residual Learning for Image Recognition
• ImageNet classification with deep convolutional neural networks
• Invariant Scattering Convolution Networks

Related

• HADES: Hardware/Algorithm Co-design in DNN accelerators using Energy-efficient Approximate Alphabet Set Multipliers
• M4BRAM: Mixed-Precision Matrix-Matrix Multiplication in FPGA Block RAMs
• No Multiplication? No Floating Point? No Problem! Training Networks for Efficient Inference
• ShiftAddNet: A Hardware-Inspired Deep Network
• BasisN: Reprogramming-Free RRAM-Based In-Memory-Computing by Basis Combination for Deep Neural Networks