2024/03/11 by Jun-Hong Shen, Shen, Junhong, Tanya Marwah +3 · 7 citations
Computer Science · Engineering · #BIM and Construction Integration #Cloud Computing and Resource Management #Distributed and Parallel Computing Systems #FOS: Computer and information sciences #Machine Learning (cs.LG)
paper · pdf · doi:10.48550/arxiv.2403.07187
openalex publication_date 2024/03/11 · openalex created_date 2024/03/14 · openalex updated_date 2026/07/28
We present Unified PDE Solvers (UPS), a data- and compute-efficient approach to developing unified neural operators for diverse families of spatiotemporal PDEs from various domains, dimensions, and resolutions. UPS embeds different PDEs into a shared representation space and processes them using a FNO-transformer architecture. Rather than training the network from scratch, which is data-demanding and computationally expensive, we warm-start the transformer from pretrained LLMs and perform explicit alignment to reduce the modality gap while improving data and compute efficiency. The cross-modal UPS achieves state-of-the-art results on a wide range of 1D and 2D PDE families from PDEBench, outperforming existing unified models using 4 times less data and 26 times less compute. Meanwhile, it is capable of few-shot transfer to unseen PDE families and coefficients.