Learning with 3D rotations, a hitchhiker's guide to SO(3)

2024/04/17 by A. René Geist, Geist, A. René, Jonas Frey +7 · 1 voice · 10 citations
Computer Science · Earth and Planetary Sciences · #Mathematics, Computing, and Information Processing #Distributed and Parallel Computing Systems #Geological Modeling and Analysis

paper · pdf · doi:10.48550/arxiv.2404.11735

Abstract

Many settings in machine learning require the selection of a rotation representation. However, choosing a suitable representation from the many available options is challenging. This paper acts as a survey and guide through rotation representations. We walk through their properties that harm or benefit deep learning with gradient-based optimization. By consolidating insights from rotation-based learning, we provide a comprehensive overview of learning functions with rotation representations. We provide guidance on selecting representations based on whether rotations are in the model's input or output and whether the data primarily comprises small angles.

Cited by

• Mind to Hand: Purposeful Robotic Control via Embodied Reasoning
• VLD: Visual Language Goal Distance for Reinforcement Learning Navigation
• Equivariant symmetry-aware head pose estimation for fetal MRI
• SPEAR-1: Scaling Beyond Robot Demonstrations via 3D Understanding
• A Primer on SO(3) Action Representations in Deep Reinforcement Learning
• Towards Foundation Models for Cryo-ET Subtomogram Analysis
• Learning and Optimization with 3D Orientations
• T-MASK: Temporal Masking for Probing Foundation Models across Camera Views in Driver Monitoring
• Learning Agile Gate Traversal via Analytical Optimal Policy Gradient
• Forecasting Continuous Non-Conservative Dynamical Systems in SO(3)

Discussions

• This week's #PaperILike is "Learning with 3D rotations, a hitchhiker's guide to SO(3)" (Geist et al., ICML 2024). This is quickly becoming one of the papers I most often share. Useful, systematic, an [bsky, 19 points, 1 comments]

Related

• Deep Learning for 2D and 3D Rotatable Data: An Overview of Methods
• Learning Unorthogonalized Matrices for Rotation Estimation
• Rotation-Invariant Random Features Provide a Strong Baseline for Machine Learning on 3D Point Clouds
• RotaTouille: Rotation Equivariant Deep Learning for Contours
• Rotation Invariant Convolutions for 3D Point Clouds Deep Learning