PointRNN: Point Recurrent Neural Network for Moving Point Cloud Processing

2019/10/18 by Hehe Fan, Yi Yang, Fan, Hehe +1 · 6 citations
Computer Science · Earth and Planetary Sciences · Engineering · #3D Shape Modeling and Analysis #3D Surveying and Cultural Heritage #Computer Graphics and Visualization Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #cs.CV

paper · pdf · doi:10.48550/arxiv.1910.08287

technical report

openalex publication_date 2019/10/18 · arxiv created 2019/11/24 · arxiv updated 2019/11/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we introduce a Point Recurrent Neural Network (PointRNN) for moving point cloud processing. At each time step, PointRNN takes point coordinates \boldsymbolP ∈ ℝ^{n × 3} and point features \boldsymbolX ∈ ℝ^{n × d} as input (n and d denote the number of points and the number of feature channels, respectively). The state of PointRNN is composed of point coordinates \boldsymbolP and point states \boldsymbolS ∈ ℝ^{n × d'} (d' denotes the number of state channels). Similarly, the output of PointRNN is composed of \boldsymbolP and new point features \boldsymbolY ∈ ℝ^{n × d''} (d'' denotes the number of new feature channels). Since point clouds are orderless, point features and states from two time steps can not be directly operated. Therefore, a point-based spatiotemporally-local correlation is adopted to aggregate point features and states according to point coordinates. We further propose two variants of PointRNN, i.e., Point Gated Recurrent Unit (PointGRU) and Point Long Short-Term Memory (PointLSTM). We apply PointRNN, PointGRU and PointLSTM to moving point cloud prediction, which aims to predict the future trajectories of points in a set given their history movements. Experimental results show that PointRNN, PointGRU and PointLSTM are able to produce correct predictions on both synthetic and real-world datasets, demonstrating their ability to model point cloud sequences. The code has been released at \urlhttps://github.com/hehefan/PointRNN.

Citations

• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Long Short-Term Memory
• Unsupervised Learning of Video Representations using LSTMs
• Sequence to Sequence Learning with Neural Networks
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• Argoverse: 3D Tracking and Forecasting with Rich Maps

Cited by

• Translution: Unifying Self-attention and Convolution for Adaptive and Relative Modeling
• Learning Scene Dynamics from Point Cloud Sequences
• Human Segmentation with Dynamic LiDAR Data
• GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action Control
• Temporal LiDAR Frame Prediction for Autonomous Driving
• Uni4D: A Unified Self-Supervised Learning Framework for Point Cloud Videos

Related

• Spatio-temporal Graph-RNN for Point Cloud Prediction
• Point Cloud Processing via Recurrent Set Encoding
• CloudLSTM: A Recurrent Neural Model for Spatiotemporal Point-cloud Stream Forecasting
• Point Cloud Mamba: Point Cloud Learning via State Space Model
• Tranquil Clouds: Neural Networks for Learning Temporally Coherent Features in Point Clouds