Spherical CNNs

2018/01/30 by Taco S. Cohen, Mario Geiger, Jonas Koehler +1 · 2 voices · 30 citations
#cs.LG #stat.ML

paper · pdf

Abstract

Convolutional Neural Networks (CNNs) have become the method of choice for learning problems involving 2D planar images. However, a number of problems of recent interest have created a demand for models that can analyze spherical images. Examples include omnidirectional vision for drones, robots, and autonomous cars, molecular regression problems, and global weather and climate modelling. A naive application of convolutional networks to a planar projection of the spherical signal is destined to fail, because the space-varying distortions introduced by such a projection will make translational weight sharing ineffective. In this paper we introduce the building blocks for constructing spherical CNNs. We propose a definition for the spherical cross-correlation that is both expressive and rotation-equivariant. The spherical correlation satisfies a generalized Fourier theorem, which allows us to compute it efficiently using a generalized (non-commutative) Fast Fourier Transform (FFT) algorithm. We demonstrate the computational efficiency, numerical accuracy, and effectiveness of spherical CNNs applied to 3D model recognition and atomization energy regression.

Cited by

• E3DGS: Unified Geometric-Photometric Equivariance for 3D Gaussian Splatting via Color-as-Geometry Embedding
• Factorized Neural Operators Decompose Dynamic and Persistent Responses
• Physical Self-Supervised Learning: IMU Sensing without Manual Labels
• A Single Architecture for Representing Invariance Under Any Space Group
• Generalized Spherical Neural Operators: Green's Function Formulation
• Equivariant symmetry-aware head pose estimation for fetal MRI
• Fast, Robust, Permutation-and-Sign Invariant SO(3) Pattern Alignment
• Categorical Equivariant Deep Learning: Category-Equivariant Neural Networks and Universal Approximation Theorems
• Unified Spherical Frontend: Learning Rotation-Equivariant Representations of Spherical Images from Any Camera
• Shaping manifolds in equivariant recurrent neural networks
• Learning with Category-Equivariant Architectures for Human Activity Recognition
• Fourier Neural Operators for Two-Phase, 2D Mold-Filling Problems Related to Metal Casting
• Learning from Frustration: Torsor CNNs on Graphs
• AQCat25: Unlocking spin-aware, high-fidelity machine learning potentials for heterogeneous catalysis
• DFNN: A Deep Fréchet Neural Network Framework for Learning Metric-Space-Valued Responses
• Bridging Symmetry and Robustness: On the Role of Equivariance in Enhancing Adversarial Robustness
• Universally Invariant Learning in Equivariant GNNs
• CAT: Curvature-Adaptive Transformers for Geometry-Aware Learning
• Spatial CAPTCHA: Generatively Benchmarking Spatial Reasoning for Human-Machine Differentiation
• Learning Generalizable Shape Completion with SIM(3) Equivariance
• Single-Core Superscalar Optimization of Clifford Neural Layers
• TusoAI: Agentic Optimization for Scientific Methods
• Local-Canonicalization Equivariant Graph Neural Networks for Sample-Efficient and Generalizable Swarm Robot Control
• Facet: highly efficient E(3)-equivariant networks for interatomic potentials
• Leveraging Equivariances and Symmetries in the Control Barrier Function Synthesis
• Surfel-based 3D Registration with Equivariant SE(3) Features
• GazeProphet: Software-Only Gaze Prediction for VR Foveated Rendering
• Adaptive Riemannian Graph Neural Networks
• Correspondence-Free Fast and Robust Spherical Point Pattern Registration
• MH-GIN: Multi-scale Heterogeneous Graph-based Imputation Network for AIS Data (Extended Version)

Discussions

• Spherical CNNs (2018) [hn, 22 points, 3 comments]
• Spherical CNNs (2018) https://arxiv.org/abs/1801.10130 [bsky, 0 points, 0 comments]

Related

• Scaling Spherical CNNs
• Efficient Generalized Spherical CNNs
• Spherical Convolutional Neural Networks: Stability to Perturbations in SO(3)
• Spin-Weighted Spherical CNNs
• Scattering Networks on the Sphere for Scalable and Rotationally Equivariant Spherical CNNs