Minimizing Energy Consumption of Deep Learning Models by Energy-Aware Training

2023/07/01 by Dario Lazzaro, Lazzaro, Dario, Antonio Emanuele Ciná +11 · 1 citation
Computer Science · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.2307.00368

openalex publication_date 2023/07/01 · openalex created_date 2023/07/05 · openalex updated_date 2026/07/31

Abstract

Deep learning models undergo a significant increase in the number of parameters they possess, leading to the execution of a larger number of operations during inference. This expansion significantly contributes to higher energy consumption and prediction latency. In this work, we propose EAT, a gradient-based algorithm that aims to reduce energy consumption during model training. To this end, we leverage a differentiable approximation of the ℓ₀ norm, and use it as a sparse penalty over the training loss. Through our experimental analysis conducted on three datasets and two deep neural networks, we demonstrate that our energy-aware training algorithm EAT is able to train networks with a better trade-off between classification performance and energy efficiency.

Cited by

• Electricity Demand and Grid Impacts of AI Data Centers: Challenges and Prospects

Related

• Uncovering Energy-Efficient Practices in Deep Learning Training: Preliminary Steps Towards Green AI
• Estimating Deep Learning energy consumption based on model architecture and training environment
• Towards energy-efficient Deep Learning: An overview of energy-efficient approaches along the Deep Learning Lifecycle
• Accuracy is not the only Metric that matters: Estimating the Energy Consumption of Deep Learning Models
• ssProp: Energy-Efficient Training for Convolutional Neural Networks with Scheduled Sparse Back Propagation