2024/03/06 by Elia Cereda, Cereda, Elia, Manuele Rusci +5 · 1 citation
Computer Science · Engineering · #Advanced Memory and Neural Computing #Artificial Intelligence (cs.AI) #CCD and CMOS Imaging Sensors #FOS: Computer and information sciences #Neural Networks and Applications #Robotics (cs.RO)
paper · pdf · doi:10.48550/arxiv.2403.04071
openalex publication_date 2024/03/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Sub-\SI50\gram nano-drones are gaining momentum in both academia and industry. Their most compelling applications rely on onboard deep learning models for perception despite severe hardware constraints (\ie sub-\SI100\milli\watt processor). When deployed in unknown environments not represented in the training data, these models often underperform due to domain shift. To cope with this fundamental problem, we propose, for the first time, on-device learning aboard nano-drones, where the first part of the in-field mission is dedicated to self-supervised fine-tuning of a pre-trained convolutional neural network (CNN). Leveraging a real-world vision-based regression task, we thoroughly explore performance-cost trade-offs of the fine-tuning phase along three axes: i) dataset size (more data increases the regression performance but requires more memory and longer computation); ii) methodologies (\eg fine-tuning all model parameters vs. only a subset); and iii) self-supervision strategy. Our approach demonstrates an improvement in mean absolute error up to 30% compared to the pre-trained baseline, requiring only \SI22\second fine-tuning on an ultra-low-power GWT GAP9 System-on-Chip. Addressing the domain shift problem via on-device learning aboard nano-drones not only marks a novel result for hardware-limited robots but lays the ground for more general advancements for the entire robotics community.