Certified Robustness under Heterogeneous Perturbations via Hybrid Randomized Smoothing

2026/05/13 by Blaise Delattre, Hengyu Wu, Paul Caillon +2 · 1 voice
Computer Science · #Adversarial Robustness in Machine Learning #Certificate #Gaussian #Generative Adversarial Networks and Image Synthesis #Image Enhancement Techniques #Rendering (computer graphics) #Robustness (evolution) #Smoothing #cs.LG

paper · pdf · open access · doi:10.48550/arxiv.2605.12876

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2026/05/13 · arxiv published 2026/05/13 · arxiv updated 2026/05/13 · openalex created_date 2026/05/15 · openalex updated_date 2026/07/28

Abstract

Randomized smoothing provides strong, model-agnostic robustness certificates, but existing guarantees are limited to single modalities, treating continuous and discrete inputs in isolation. This limitation becomes critical in multimodal models, where decisions depend on cross-modal semantics and adversaries can jointly perturb heterogeneous inputs, rendering unimodal certificates insufficient. We introduce a unified randomized smoothing framework for mixed discrete--continuous inputs based on an analytically tractable Neyman--Pearson formulation of the joint worst-case problem. By analyzing the joint likelihood ordering induced by factorized discrete and continuous noise, our approach yields a closed-form, one-dimensional certificate that strictly generalizes both Gaussian (image-only) and discrete (text-only) randomized smoothing. We validate the framework on multimodal safety filtering, providing, to our knowledge, the first model-agnostic Neyman--Pearson certificate for joint discrete-token and continuous-image perturbations in interaction-dependent text--image safety filtering.

Citations

• Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
• Randomized Smoothing Meets Vision-Language Models
• Defending Against Prompt Injection With a Few DefensiveTokens
• Bridging the Theoretical Gap in Randomized Smoothing
• Membership Inference Attacks on Large-Scale Models: A Survey
• Defeating Prompt Injections by Design
• Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents
• Towards the Worst-case Robustness of Large Language Models
• Achieving Domain-Independent Certified Robustness via Knowledge Continuity
• LlavaGuard: An Open VLM-based Framework for Safeguarding Vision Datasets and Models
• Sports-Traj: A Unified Trajectory Generation Model for Multi-Agent Movement in Sports
• MMCert: Provable Defense against Adversarial Attacks to Multi-modal Models
• COMMIT: Certifying Robustness of Multi-Sensor Fusion Systems against Semantic Attacks
• StruQ: Defending Against Prompt Injection with Structured Queries
• Your Diffusion Model is Secretly a Certifiably Robust Classifier
• MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models
• A Survey on Multimodal Large Language Models for Autonomous Driving
• Adversarial Attacks to Multi-Modal Models
• Fast Certification of Vision-Language Models Using Incremental Randomized Smoothing
• VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Models
• Low-Resource Languages Jailbreak GPT-4
• The Lipschitz-Variance-Margin Tradeoff for Enhanced Randomized Smoothing
• Certifying LLM Safety against Adversarial Prompting
• Code of "Sirens' Whisper: Inaudible Near-Ultrasonic Jailbreaks of Speech-Driven LLMs"
• Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models
• Are aligned neural networks adversarially aligned?
• Concrete Score Matching: Generalized Score Matching for Discrete Data
• (Certified!!) Adversarial Robustness for Free!
• Towards Adversarial Attack on Vision-Language Pre-training Models
• Masked Autoencoders Are Scalable Vision Learners
• Certified Robustness to Text Adversarial Attacks by Randomized [MASK]
• Improved, Deterministic Smoothing for L₁ Certified Robustness
• SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutions
• The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes
• Robustness Certificates for Sparse Adversarial Attacks by Randomized Ablation
• Certified Robustness to Adversarial Word Substitutions
• Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
• Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
• Tight Certificates of Adversarial Robustness for Randomly Smoothed Classifiers
• Certified Adversarial Robustness via Randomized Smoothing
• Certified Robustness to Adversarial Examples with Differential Privacy
• Towards Deep Learning Models Resistant to Adversarial Attacks

Discussions

• 📖 New LAMSADE preprint: Certified Robustness under Heterogeneous Perturbations via Hybrid Randomized Smoothing. (arXiv) https://doi.org/10.48550/arXiv.2605.12876 [bsky, 0 points, 0 comments]

Related

• Extensions and limitations of randomized smoothing for robustness guarantees
• Certified Adversarial Robustness via Randomized Smoothing
• UniCR: Universally Approximated Certified Robustness via Randomized Smoothing
• Towards Evading the Limits of Randomized Smoothing: A Theoretical Analysis
• Localized Randomized Smoothing for Collective Robustness Certification