Energy-Aware Deep Learning on Resource-Constrained Hardware

2025/05/18 by Millar, Josh, Haddadi, Hamed, Madhavapeddy, Anil
#FOS: Computer and information sciences #Hardware Architecture (cs.AR) #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2505.12523

Abstract

The use of deep learning (DL) on Internet of Things (IoT) and mobile devices offers numerous advantages over cloud-based processing. However, such devices face substantial energy constraints to prolong battery-life, or may even operate intermittently via energy-harvesting. Consequently, energy-aware approaches for optimizing DL inference and training on such resource-constrained devices have garnered recent interest. We present an overview of such approaches, outlining their methodologies, implications for energy consumption and system-level efficiency, and their limitations in terms of supported network types, hardware platforms, and application scenarios. We hope our review offers a clear synthesis of the evolving energy-aware DL landscape and serves as a foundation for future research in energy-constrained computing.

Citations

• Towards On-device Learning on the Edge: Ways to Select Neurons to Update under a Budget Constraint
• EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism
• LifeLearner: Hardware-Aware Meta Continual Learning System for Embedded Computing Platforms
• Domain Generalization Guided by Gradient Signal to Noise Ratio of Parameters
• Surgical Fine-Tuning Improves Adaptation to Distribution Shifts
• To update or not to update? Neurons at equilibrium in deep models
• On-Device Training Under 256KB Memory
• Predictive Exit: Prediction of Fine-Grained Early Exits for Computation- and Energy-Efficient Inference
• Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution
• AdaViT: Adaptive Vision Transformers for Efficient Image Recognition
• A Fine-Grained Analysis on Distribution Shift
• Domain Generalization via Gradient Surgery
• The Evolution of Out-of-Distribution Robustness Throughout Fine-Tuning
• Zero Time Waste: Recycling Predictions in Early Exit Neural Networks
• HW-NAS-Bench:Hardware-Aware Neural Architecture Search Benchmark
• Toward Compact Deep Neural Networks via Energy-Aware Pruning
• Sustainable Federated Learning
• AutoFreeze: Automatically Freezing Model Blocks to Accelerate Fine-tuning
• Towards Energy Efficient Federated Learning over 5G+ Mobile Devices
• In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness
• μNAS: Constrained Neural Architecture Search for Microcontrollers
• MCUNet: Tiny Deep Learning on IoT Devices
• Do Adversarially Robust ImageNet Models Transfer Better?
• Learning to Learn with Variational Information Bottleneck for Domain Generalization
• Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics
• Domain Generalization using Causal Matching
• A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions
• A Comprehensive Survey of Neural Architecture Search
• DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference
• NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural\n Architecture Search
• Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approach
• NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search
• What Would Elsa Do? Freezing Layers During Transformer Fine-Tuning
• Energy Efficient Federated Learning Over Wireless Communication Networks
• Energy-Aware Analog Aggregation for Federated Learning with Redundant Data
• Splitting Steepest Descent for Growing Neural Architectures
• Energy-Aware Neural Architecture Optimization with Fast Splitting Steepest Descent
• Energy-Efficient Radio Resource Allocation for Federated Edge Learning
• On the Convergence of FedAvg on Non-IID Data
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• DeepCaps: Going Deeper with Capsule Networks
• Robust and Communication-Efficient Federated Learning from Non-IID Data
• Broadband Analog Aggregation for Low-Latency Federated Edge Learning (Extended Version)
• ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation
• Slimmable Neural Networks
• SpotTune: Transfer Learning through Adaptive Fine-tuning
• Shallow-Deep Networks: Understanding and Mitigating Network Overthinking
• Intelligence Beyond the Edge: Inference on Intermittent Embedded Systems
• Dynamic Channel Pruning: Feature Boosting and Suppression
• ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
• ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• To Trust Or Not To Trust A Classifier
• Adaptive Federated Learning in Resource Constrained Edge Computing Systems
• Adaptive Federated Learning in Resource Constrained Edge Computing Systems
• JointDNN: An Efficient Training and Inference Engine for Intelligent\n Mobile Cloud Computing Services
• SkipNet: Learning Dynamic Routing in Convolutional Networks
• BlockDrop: Dynamic Inference Paths in Residual Networks
• Dynamic Routing Between Capsules
• Learning to Generalize: Meta-Learning for Domain Generalization
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Multi-Scale Dense Networks for Resource Efficient Image Classification
• Changing Model Behavior at Test-Time Using Reinforcement Learning
• Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
• Spatially Adaptive Computation Time for Residual Networks
• BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks
• Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning
• Training Deep Nets with Sublinear Memory Cost
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural\n Networks
• Deep Residual Learning for Image Recognition
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• Conditional Deep Learning for Energy-Efficient and Enhanced Pattern Recognition
• Learning both Weights and Connections for Efficient Neural Networks
• ImageNet classification with deep convolutional neural networks
• MLPerf Power: Benchmarking the Energy Efficiency of Machine Learning Systems from Microwatts to Megawatts for Sustainable AI
• TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edge

Related

• Enabling Fast Deep Learning on Tiny Energy-Harvesting IoT Devices
• Hardware-Aware Machine Learning: Modeling and Optimization
• Towards energy-efficient Deep Learning: An overview of energy-efficient approaches along the Deep Learning Lifecycle
• Leveraging Application-Specific Knowledge for Energy-Efficient Deep Learning Accelerators on Resource-Constrained FPGAs
• Towards Implementing Energy-aware Data-driven Intelligence for Smart Health Applications on Mobile Platforms