Deep Neural Networks are Easily Fooled: High Confidence Predictions for\n Unrecognizable Images

2014/12/05 by Anh‐Tu Nguyen, Anh Nguyen, Nguyen, Anh +4 · 3 voices · 48 citations
Computer Science · Engineering · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Industrial Vision Systems and Defect Detection #Visual Attention and Saliency Detection #cs.AI #cs.CV #cs.NE

paper · pdf · doi:10.48550/arxiv.1412.1897

openalex publication_date 2014/12/05 · openalex created_date 2025/10/24 · openalex updated_date 2026/07/28

Abstract

Deep neural networks (DNNs) have recently been achieving state-of-the-art\nperformance on a variety of pattern-recognition tasks, most notably visual\nclassification problems. Given that DNNs are now able to classify objects in\nimages with near-human-level performance, questions naturally arise as to what\ndifferences remain between computer and human vision. A recent study revealed\nthat changing an image (e.g. of a lion) in a way imperceptible to humans can\ncause a DNN to label the image as something else entirely (e.g. mislabeling a\nlion a library). Here we show a related result: it is easy to produce images\nthat are completely unrecognizable to humans, but that state-of-the-art DNNs\nbelieve to be recognizable objects with 99.99% confidence (e.g. labeling with\ncertainty that white noise static is a lion). Specifically, we take\nconvolutional neural networks trained to perform well on either the ImageNet or\nMNIST datasets and then find images with evolutionary algorithms or gradient\nascent that DNNs label with high confidence as belonging to each dataset class.\nIt is possible to produce images totally unrecognizable to human eyes that DNNs\nbelieve with near certainty are familiar objects, which we call "fooling\nimages" (more generally, fooling examples). Our results shed light on\ninteresting differences between human vision and current DNNs, and raise\nquestions about the generality of DNN computer vision.\n

Citations

• Gradient-based learning applied to document recognition
• ImageNet Large Scale Visual Recognition Challenge
• ImageNet classification with deep convolutional neural networks
• Caffe: Convolutional Architecture for Fast Feature Embedding

Cited by

• The 3D Mirage: Probing and Taming 3D Hallucinations
• Detecting Neural Network Failures through Spectral Analysis of Internal Activations
• Probabilistic Extension of Neuro-Symbolic AGI Robots based on Belnap's Typed Intensional FOL
• Neuro-Symbolic Strong-AI Robots with Closed Knowledge Assumption: Learning and Deductions
• Generative Bayesian Filtering for State Estimation
• Towards end-to-end automation of AI research
• In Search of the Ingredients of Open-Endedness: Replicating Picbreeder with Large Vision-Language Models
• A New Kind of Adversarial Example: Measuring the Human-Model Gap, and Its Relationship to OOD Detection
• Adversarial Robustness of Vision in Open Foundation Models
• Predictive Sample Assignment for Semantically Coherent Out-of-Distribution Detection
• SPOOF: Simple Pixel Operations for Out-of-Distribution Fooling
• The Double-Edged Nature of the Rashomon Set for Trustworthy Machine Learning
• Exploiting Inter-Sample Information for Long-tailed Out-of-Distribution Detection
• DeepDefense: Layer-Wise Gradient-Feature Alignment for Building Robust Neural Networks
• SHRUG-FM: Reliability-Aware Foundation Models for Earth Observation
• GAFD-CC: Global-Aware Feature Decoupling with Confidence Calibration for OOD Detection
• More Than A Shortcut: A Hyperbolic Approach To Early-Exit Networks
• On the Provable Importance of Gradients for Language-Assisted Image Clustering
• Integrating Machine Learning into Belief-Desire-Intention Agents: Current Advances and Open Challenges
• GOOD: Training-Free Guided Diffusion Sampling for Out-of-Distribution Detection
• Benchmarking Out-of-Distribution Detection for Plankton Recognition: A Systematic Evaluation of Advanced Methods in Marine Ecological Monitoring
• A Hard-Label Black-Box Evasion Attack against ML-based Malicious Traffic Detection Systems
• DEXTER: Diffusion-Guided EXplanations with TExtual Reasoning for Vision Models
• A Multi-dimensional Semantic Surprise Framework Based on Low-Entropy Semantic Manifolds for Fine-Grained Out-of-Distribution Detection
• SCOPED: Score-Curvature Out-of-distribution Proximity Evaluator for Diffusion
• Out-of-Distribution Detection in LiDAR Semantic Segmentation Using Epistemic Uncertainty from Hierarchical GMMs
• Medix: Out-of-Distribution Detection from Unlabeled Wild Data via Robust Gradient Statistics
• Influence-Guided Concolic Testing of Transformer Robustness
• Smoothing-Based Conformal Prediction for Balancing Efficiency and Interpretability
• ICONIC-444: A 3.1-Million-Image Dataset for OOD Detection Research
• Graph Enhanced Trajectory Anomaly Detection
• Long-Tailed Out-of-Distribution Detection with Refined Separate Class Learning
• SOLAR: Switchable Output Layer for Accuracy and Robustness in Once-for-All Training
• Enhancing Multi-Agent Debate System Performance via Confidence Expression
• ANTS: Adaptive Negative Textual Space Shaping for OOD Detection via Test-Time MLLM Understanding and Reasoning
• TopoMap: A Feature-based Semantic Discriminator of the Topographical Regions in the Test Input Space
• Probability Density from Latent Diffusion Models for Out-of-Distribution Detection
• Wormhole Dynamics in Deep Neural Networks
• Model Interpretability and Rationale Extraction by Input Mask Optimization
• Out-of-Distribution Detection using Counterfactual Distance
• Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring
• Sample-aware RandAugment: Search-free Automatic Data Augmentation for Effective Image Recognition
• Enhancing Reliability of Medical Image Diagnosis through Top-rank Learning with Rejection Module
• ETA: Energy-based Test-time Adaptation for Depth Completion
• Collision-based Watermark for Detecting Backdoor Manipulation in Federated Learning
• AutoSIGHT: Automatic Eye Tracking-based System for Immediate Grading of Human experTise
• Beyond topography: Topographic regularization improves robustness and reshapes representations in convolutional neural networks
• Deep learning [wikipedia]

Discussions

• Deep Neural Networks Are Easily Fooled [hn, 183 points, 60 comments]
• Deep Neural Networks Are Easily Fooled [pdf] [hn, 1 points, 0 comments]
• This new paper on deep learning doesn't bode well for self-driving cars: http://arxiv.org/pdf/1412.1897v1.pdf [bsky, 0 points, 0 comments]

Related

• Comparing deep neural networks against humans: object recognition when the signal gets weaker
• Are DNNs fooled by extremely unrecognizable images?
• Minimal Images in Deep Neural Networks: Fragile Object Recognition in Natural Images
• Harmonizing the object recognition strategies of deep neural networks with humans
• Generalisation in humans and deep neural networks