Attacking Motion Estimation with Adversarial Snow

2022/10/20 by Jenny Schmalfuß, Lukas Mehl, Schmalfuss, Jenny +3 · 3 citations
Computer Science · #Advanced Image Processing Techniques #Advanced Vision and Imaging #Adversarial Robustness in Machine Learning #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2210.11242

openalex publication_date 2022/10/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Current adversarial attacks for motion estimation (optical flow) optimize small per-pixel perturbations, which are unlikely to appear in the real world. In contrast, we exploit a real-world weather phenomenon for a novel attack with adversarially optimized snow. At the core of our attack is a differentiable renderer that consistently integrates photorealistic snowflakes with realistic motion into the 3D scene. Through optimization we obtain adversarial snow that significantly impacts the optical flow while being indistinguishable from ordinary snow. Surprisingly, the impact of our novel attack is largest on methods that previously showed a high robustness to small L_p perturbations.

Cited by

• Are Synthetic Corruptions A Reliable Proxy For Real-World Corruptions?
• RobustSpring: Benchmarking Robustness to Image Corruptions for Optical Flow, Scene Flow and Stereo
• DispBench: Benchmarking Disparity Estimation to Synthetic Corruptions

Related

• Distracting Downpour: Adversarial Weather Attacks for Motion Estimation
• UNDREAM: Bridging Differentiable Rendering and Photorealistic Simulation for End-to-end Adversarial Attacks
• AdvRain: Adversarial Raindrops to Attack Camera-based Smart Vision Systems
• fakeWeather: Adversarial Attacks for Deep Neural Networks Emulating Weather Conditions on the Camera Lens of Autonomous Systems
• Let it Snow! Animating 3D Gaussian Scenes with Dynamic Weather Effects via Physics-Guided Score Distillation