Deep Convolution for Irregularly Sampled Temporal Point Clouds

2021/05/01 by Erich Merrill, Stefan Lee, Merrill, Erich +8
Computer Science · Engineering · Mathematics · #3D Shape Modeling and Analysis #Architecture #Artificial intelligence #Artificial neural network #Computer science #Convolution (computer science) #Data Management and Algorithms #Data mining #FOS: Computer and information sciences #Flexibility (engineering) #Geography #Human Pose and Action Recognition #Machine Learning (cs.LG) #Mathematics #Point (geometry) #Point cloud #Process (computing) #Space (punctuation) #Variety (cybernetics) #cs.LG

paper · pdf · doi:10.48550/arxiv.2105.00137

published in arXiv (Cornell University) (Cornell University) · 12 pages, submitted to ICLR 2021

arxiv created 2021/05/01 · openalex publication_date 2021/05/01 · arxiv updated 2021/05/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

We consider the problem of modeling the dynamics of continuous spatial-temporal processes represented by irregular samples through both space and time. Such processes occur in sensor networks, citizen science, multi-robot systems, and many others. We propose a new deep model that is able to directly learn and predict over this irregularly sampled data, without voxelization, by leveraging a recent convolutional architecture for static point clouds. The model also easily incorporates the notion of multiple entities in the process. In particular, the model can flexibly answer prediction queries about arbitrary space-time points for different entities regardless of the distribution of the training or test-time data. We present experiments on real-world weather station data and battles between large armies in StarCraft II. The results demonstrate the model's flexibility in answering a variety of query types and demonstrate improved performance and efficiency compared to state-of-the-art baselines.

Citations

• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Interaction Networks for Learning about Objects, Relations and Physics
• Submanifold Sparse Convolutional Networks
• StarCraft II: A New Challenge for Reinforcement Learning
• Learning Temporal Point Processes via Reinforcement Learning
• Interpolation-Prediction Networks for Irregularly Sampled Time Series
• Self-Attentive Hawkes Processes
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• Dynamic Graph CNN for Learning on Point Clouds
• Transformer Hawkes Process

Related

• PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences
• Tranquil Clouds: Neural Networks for Learning Temporally Coherent Features in Point Clouds
• Neural Point Process for Learning Spatiotemporal Event Dynamics
• Deep Random Features for Scalable Interpolation of Spatiotemporal Data
• CaSPR: Learning Canonical Spatiotemporal Point Cloud Representations