InstantFT: An FPGA-Based Runtime Subsecond Fine-tuning of CNN Models

2025/06/06 by Sugiura, Keisuke, Matsutani, Hiroki
#FOS: Computer and information sciences #Hardware Architecture (cs.AR) #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2506.06505

Abstract

Training deep neural networks (DNNs) requires significantly more computation and memory than inference, making runtime adaptation of DNNs challenging on resource-limited IoT platforms. We propose InstantFT, an FPGA-based method for ultra-fast CNN fine-tuning on IoT devices, by optimizing the forward and backward computations in parameter-efficient fine-tuning (PEFT). Experiments on datasets with concept drift demonstrate that InstantFT fine-tunes a pre-trained CNN 17.4x faster than existing Low-Rank Adaptation (LoRA)-based approaches, while achieving comparable accuracy. Our FPGA-based InstantFT reduces the fine-tuning time to just 0.36s and improves energy-efficiency by 16.3x, enabling on-the-fly adaptation of CNNs to non-stationary data distributions.

Related

• EF-Train: Enable Efficient On-device CNN Training on FPGA Through Data Reshaping for Online Adaptation or Personalization
• Full-stack Optimization for Accelerating CNNs with FPGA Validation
• Accelerating CNN inference on FPGAs: A Survey
• A Resource-Driven Approach for Implementing CNNs on FPGAs Using Adaptive IPs
• Systolic-CNN: An OpenCL-defined Scalable Run-time-flexible FPGA Accelerator Architecture for Accelerating Convolutional Neural Network Inference in Cloud/Edge Computing