2024/02/25 by Tianyu Chen, Chen, Tianyu, Haoyi Zhou +12 · 4 citations
Business, Management and Accounting · Computer Science · #Artificial Intelligence (cs.AI) #Big Data and Business Intelligence #FOS: Computer and information sciences #Machine Learning (cs.LG) #Online Learning and Analytics
paper · pdf · doi:10.48550/arxiv.2402.16014
openalex publication_date 2024/02/25 · openalex created_date 2024/02/28 · openalex updated_date 2026/07/28
Foundation models have revolutionized language modeling, while whether this success is replicated in scientific computing remains unexplored. We present OmniArch, the first prototype aiming at solving multi-scale and multi-physics scientific computing problems with physical alignment. We addressed all three challenges with one unified architecture. Its pre-training stage contains a Fourier Encoder-decoder fading out the disharmony across separated dimensions and a Transformer backbone integrating quantities through temporal dynamics, and the novel PDE-Aligner performs physics-informed fine-tuning under flexible conditions. As far as we know, we first conduct 1D-2D-3D united pre-training on the PDEBench, and it sets not only new performance benchmarks for 1D, 2D, and 3D PDEs but also demonstrates exceptional adaptability to new physics via in-context and zero-shot learning approaches, which supports realistic engineering applications and foresight physics discovery.