Neural Functions for Learning Periodic Signal

2025/06/11 by Woojin Cho, Minju Jo, Cho, Woojin +5
Computer Science · Physics and Astronomy · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Neural Networks and Reservoir Computing #Time Series Analysis and Forecasting

paper · pdf · doi:10.48550/arxiv.2506.09526

openalex publication_date 2025/06/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

As function approximators, deep neural networks have served as an effective tool to represent various signal types. Recent approaches utilize multi-layer perceptrons (MLPs) to learn a nonlinear mapping from a coordinate to its corresponding signal, facilitating the learning of continuous neural representations from discrete data points. Despite notable successes in learning diverse signal types, coordinate-based MLPs often face issues of overfitting and limited generalizability beyond the training region, resulting in subpar extrapolation performance. This study addresses scenarios where the underlying true signals exhibit periodic properties, either spatially or temporally. We propose a novel network architecture, which extracts periodic patterns from measurements and leverages this information to represent the signal, thereby enhancing generalization and improving extrapolation performance. We demonstrate the efficacy of the proposed method through comprehensive experiments, including the learning of the periodic solutions for differential equations, and time series imputation (interpolation) and forecasting (extrapolation) on real-world datasets.

Citations

• Koopman-informed recurrent neural networks
• Almost-Linear RNNs Yield Highly Interpretable Symbolic Codes in Dynamical Systems Reconstruction
• Hypernetwork-based Meta-Learning for Low-Rank Physics-Informed Neural Networks
• Neural Functional Transformers
• Deep Learning on Implicit Neural Representations of Shapes
• WIRE: Wavelet Implicit Neural Representations
• Learning Continuous Implicit Representation for Near-Periodic Patterns
• Are Transformers Effective for Time Series Forecasting?
• DEPTS: Deep Expansion Learning for Periodic Time Series Forecasting
• FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting
• Time-Series Anomaly Detection with Implicit Neural Representation
• From data to functa: Your data point is a function and you can treat it like one
• Meta-Learning Sparse Implicit Neural Representations
• On the difficulty of learning chaotic dynamics with RNNs
• Seeing Implicit Neural Representations as Fourier Series
• Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting
• Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding
• Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
• Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
• Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
• Implicit Neural Representations with Periodic Activation Functions
• Neural Networks Fail to Learn Periodic Functions and How to Fix It
• Multi-Time Attention Networks for Irregularly Sampled Time Series
• Neural Controlled Differential Equations for Irregular Time Series
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Time2Vec: Learning a Vector Representation of Time
• Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
• Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network
• Conditional Neural Processes
• Multilevel Wavelet Decomposition Network for Interpretable Time Series Analysis
• On the Spectral Bias of Neural Networks
• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• Neural Ordinary Differential Equations
• Attention Is All You Need
• A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction
• Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks
• Long Short-Term Memory
• Recurrent neural networks and robust time series prediction
• Deterministic Nonperiodic Flow
• Learning Deep Time-index Models for Time Series Forecasting

Cited by

• Fourier-Modulated Implicit Neural Representation for Multispectral Satellite Image Compression

Related

• Extrapolation of Periodic Functions Using Binary Encoding of Continuous Numerical Values
• Neural Networks Fail to Learn Periodic Functions and How to Fix It
• Periodic Extrapolative Generalisation in Neural Networks
• FAN: Fourier Analysis Networks
• Stationarity Exploration for Multivariate Time Series Forecasting