AION-1: Omnimodal Foundation Model for Astronomical Sciences

2025/10/20 by Liam Parker, François Lanusse, Parker, Liam +51 · 2 citations
Physics and Astronomy · #Astronomy and Astrophysical Research #Cosmology and Nongalactic Astrophysics (astro-ph.CO) #FOS: Physical sciences #Galaxies: Formation, Evolution, Phenomena #Gamma-ray bursts and supernovae #Instrumentation and Methods for Astrophysics (astro-ph.IM)

paper · pdf · doi:10.48550/arxiv.2510.17960

openalex publication_date 2025/10/20 · openalex created_date 2025/10/24 · openalex updated_date 2026/07/28

Abstract

While foundation models have shown promise across a variety of fields, astronomy still lacks a unified framework for joint modeling across its highly diverse data modalities. In this paper, we present AION-1, a family of large-scale multimodal foundation models for astronomy. AION-1 integrates heterogeneous imaging, spectroscopic, and scalar data using a two-stage architecture: modality-specific tokenization followed by transformer-based masked modeling of cross-modal token sequences. The model is pretrained on five large-scale surveys: Legacy Survey, Hyper Suprime-Cam (HSC), Sloan Digital Sky Survey (SDSS), Dark Energy Spectroscopic Instrument (DESI), and Gaia. These span more than 200 million observations of stars, galaxies, and quasars. With a single frozen encoder, AION-1 achieves strong results on a broad suite of downstream tasks, including galaxy and stellar property estimation, galaxy morphology classification, similarity-based retrieval, galaxy image segmentation, and spectral super-resolution. We release AION-1 model variants ranging from 300 M to 3.1 B parameters. Beyond astronomy, AION-1 provides a scalable blueprint for multimodal scientific foundation models that can seamlessly integrate noisy, instrument-specific observations. All code, tokenizers, pretrained weights, and a lightweight evaluation suite are released under an open-source license.

Citations

• The Multimodal Universe: Enabling Large-Scale Machine Learning with 100TB of Astronomical Scientific Data
• How far can bias go? -- Tracing bias from pretraining data to alignment
• AstroM³: A self-supervised multimodal model for astronomy
• SpectraFM: Tuning into Stellar Foundation Models
• The Llama 3 Herd of Models
• 4M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalities
• AstroPT: Scaling Large Observation Models for Astronomy
• Learning Diffusion Priors from Observations by Expectation Maximization
• Scaling Laws for Galaxy Images
• PAPERCLIP: Associating Astronomical Observations and Natural Language with Multi-Modal Models
• Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
• Determining Stellar Elemental Abundances from DESI Spectra with the Data-Driven Payne
• Survey of Gravitationally lensed Objects in HSC Imaging (SuGOHI) - X. Strong Lens Finding in The HSC-SSP using Convolutional Neural Networks
• Deep Learning Segmentation of Spiral Arms and Bars
• Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation
• Multiple Physics Pretraining for Physical Surrogate Models
• Finite Scalar Quantization: VQ-VAE Made Simple
• ImageBind: One Embedding Space To Bind Them All
• Visual Instruction Tuning
• DINOv2: Learning Robust Visual Features without Supervision
• Micrograph segmentations for DDEVD
• Segment Anything
• GPT-4 Technical Report
• ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders
• MAGVIT: Masked Generative Video Transformer
• Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
• Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution
• Self-supervised similarity search for large scientific datasets
• Pix2seq: A Language Modeling Framework for Object Detection
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• Overview of the DESI Legacy Imaging Surveys
• Neural Discrete Representation Learning
• The Hyper Suprime-Cam SSP Survey: Overview and survey design
• The DESI Experiment Part I: Science,Targeting, and Survey Design
• The 16th Data Release of the Sloan Digital Sky Surveys: First Release from the APOGEE-2 Southern Survey and Full Release of eBOSS Spectra

Cited by

• Semantic search for 100M+ galaxy images using AI-generated captions
• MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities

Related

• Multi-modal Foundation Model for Cosmological Simulation Data
• AstroPT: Scaling Large Observation Models for Astronomy
• The Multimodal Universe: Enabling Large-Scale Machine Learning with 100TB of Astronomical Scientific Data
• Multi-Modal Masked Autoencoders for Learning Image-Spectrum Associations for Galaxy Evolution and Cosmology
• Modelling the Galaxy in the era of Gaia