Astro-MoE: Mixture of Experts for Multiband Astronomical Time Series

2025/07/16 by Martina Cádiz-Leyton, G. Cabrera-Vives, Cádiz-Leyton, Martina +7
Computer Science · Engineering · #Astronomical Observations and Instrumentation #FOS: Physical sciences #Instrumentation and Methods for Astrophysics (astro-ph.IM) #Time Series Analysis and Forecasting

paper · pdf · doi:10.48550/arxiv.2507.12611

openalex publication_date 2025/07/16 · openalex created_date 2025/10/18 · openalex updated_date 2026/07/28

Abstract

Multiband astronomical time series exhibit heterogeneous variability patterns, sampling cadences, and signal characteristics across bands. Standard transformers apply shared parameters to all bands, potentially limiting their ability to model this rich structure. In this work, we introduce Astro-MoE, a foundational transformer architecture that enables dynamic processing via a Mixture of Experts module. We validate our model on both simulated (ELAsTiCC-1) and real-world datasets (Pan-STARRS1).

Citations

• A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications
• Multiband Embeddings of Light Curves
• Don't Pay Attention to the Noise: Learning Self-supervised Representations of Light Curves with a Denoising Time Series Transformer
• Inferring Black Hole Properties from Astronomical Multivariate Time Series with Bayesian Attentive Neural Processes
• Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
• GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
• Attention Is All You Need
• Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer

Related

• Multivariate Time-series Transformer Embeddings for Light Curves
• Paying Attention to Astronomical Transients: Introducing the Time-series Transformer for Photometric Classification
• AstroPT: Scaling Large Observation Models for Astronomy
• Transformer-Based Astronomical Time Series Model with Uncertainty Estimation for Detecting Misclassified Instances
• LightCurve MoE: A Dynamic Sparse Routing Mixture-of-Experts Architecture for Efficient Stellar Light Curve Classification