Fast model inference and training on-board of Satellites

2023/07/17 by Vít Růžička, Růžička, Vít, Gonzalo Mateo‐García +11 · 4 citations
Computer Science · Engineering · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #CCD and CMOS Imaging Sensors #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.2307.08700

openalex publication_date 2023/07/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Artificial intelligence onboard satellites has the potential to reduce data transmission requirements, enable real-time decision-making and collaboration within constellations. This study deploys a lightweight foundational model called RaVAEn on D-Orbit's ION SCV004 satellite. RaVAEn is a variational auto-encoder (VAE) that generates compressed latent vectors from small image tiles, enabling several downstream tasks. In this work we demonstrate the reliable use of RaVAEn onboard a satellite, achieving an encoding time of 0.110s for tiles of a 4.8x4.8 km² area. In addition, we showcase fast few-shot training onboard a satellite using the latent representation of data. We compare the deployment of the model on the on-board CPU and on the available Myriad vision processing unit (VPU) accelerator. To our knowledge, this work shows for the first time the deployment of a multi-task model on-board a CubeSat and the on-board training of a machine learning model.

Cited by

• First On-Orbit Demonstration of a Geospatial Foundation Model
• Bringing Federated Learning to Space
• Efficient SAR Vessel Detection for FPGA-Based On-Satellite Sensing
• Optimizing Methane Detection On Board Satellites: Speed, Accuracy, and Low-Power Solutions for Resource-Constrained Hardware

Related

• Stitching Satellites to the Edge: Pervasive and Efficient Federated LEO Satellite Learning
• Decentralised Semi-supervised Onboard Learning for Scene Classification in Low-Earth Orbit
• Satellite Edge Artificial Intelligence with Large Models: Architectures and Technologies
• The Final Frontier: Deep Learning in Space
• Artificial Intelligence implementation of onboard flexible payload and adaptive beamforming using commercial off-the-shelf devices