Deep Learning: A Critical Appraisal
2018/01/02 by Gary Marcus, Marcus, Gary · 13 voices · 388 citations
Computer Science · Mathematics · Psychology · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Artificial intelligence #Computer science #Deep learning #Domain Adaptation and Few-Shot Learning #Enthusiasm #Field (mathematics) #Psychology #Social psychology #Term (time) #acm:97R40 #cs.AI #cs.LG #msc:97R40 #stat.ML
paper · pdf · doi:10.48550/arxiv.1801.00631
published in arXiv (Cornell University) (Cornell University) · 1 figure
arxiv created 2018/01/02 · openalex publication_date 2018/01/02 · arxiv updated 2018/01/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Abstract
Although deep learning has historical roots going back decades, neither the term "deep learning" nor the approach was popular just over five years ago, when the field was reignited by papers such as Krizhevsky, Sutskever and Hinton's now classic (2012) deep network model of Imagenet. What has the field discovered in the five subsequent years? Against a background of considerable progress in areas such as speech recognition, image recognition, and game playing, and considerable enthusiasm in the popular press, I present ten concerns for deep learning, and suggest that deep learning must be supplemented by other techniques if we are to reach artificial general intelligence.
Citations
Cited by
- Not Minds, but Signs: Reframing LLMs through Semiotics
- Mechanism-Based Intelligence (MBI): Differentiable Incentives for Rational Coordination and Guaranteed Alignment in Multi-Agent Systems
- Few-Shot Learning of a Graph-Based Neural Network Model Without Backpropagation
- AI/ML based Joint Source and Channel Coding for HARQ-ACK Payload
- Foundations of Artificial Intelligence Frameworks: Notion and Limits of AGI
- Consciousness in Artificial Intelligence? A Framework for Classifying Objections and Constraints
- Towards Efficient and Secure Delivery of Data for Deep Learning with Privacy-Preserving
- Spectral imaginings and sympoietic creativity: AI hallucinations and the ethics of posthuman creativity
- Education Paradigm Shift To Maintain Human Competitive Advantage Over AI
- Levels of Analysis for Machine Learning
- Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning
- Shallow Unorganized Neural Networks using Smart Neuron Model for Visual Perception
- Neuro-Symbolic Spatial Reasoning in Segmentation
- Dual approach for object tracking based on optical flow and swarm intelligence
- Hybrid Models for Natural Language Reasoning: The Case of Syllogistic Logic
- Towards Neurocognitive-Inspired Intelligence: From AI's Structural Mimicry to Human-Like Functional Cognition
- HTMformer: Hybrid Time and Multivariate Transformer for Time Series Forecasting
- Physics Knowledge in Frontier Models: A Diagnostic Study of Failure Modes
- Intercontinental prediction of soybean phenology via hybrid ensemble of knowledge-based and data-driven models
- Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies
- Shortcut learning in deep neural networks
- Out of Distribution Generalization in Machine Learning
- Learning model-based strategies in simple environments with hierarchical q-networks
- Deep opacity and AI: A threat to XAI and to privacy protection mechanisms
- Cross-Domain Few-Shot Learning by Representation Fusion
- A decision theoretic approach to model evaluation in computational drug discovery
- Memorisation and forgetting in a learning Hopfield neural network: bifurcation mechanisms, attractors and basins
- Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs
- Limits of trust in medical AI
- A model of cortical cognitive function using hierarchical interactions of gating matrices in internal agents coding relational representations
- Don't Explain without Verifying Veracity: An Evaluation of Explainable AI with Video Activity Recognition
- Giving Up Control: Neurons as Reinforcement Learning Agents
- JUMPER: Learning When to Make Classification Decisions in Reading
- The neural architecture of language: Integrative modeling converges on predictive processing
- Zero-shot task adaptation by homoiconic meta-mapping
- Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI
- The Serial Scaling Hypothesis
- On the Ethics of Building AI in a Responsible Manner
- Towards sample-efficient episodic control with DAC-ML
- Solutions to problems with deep learning
- Learning a metacognition for object perception
- Neural Abstract Reasoner
- Computational principles of intelligence: learning and reasoning with neural networks
- Bridging Neural Networks and Dynamic Time Warping for Adaptive Time Series Classification
- Lexicon Learning for Few-Shot Neural Sequence Modeling
- Unsupervised Deep Learning by Injecting Low-Rank and Sparse Priors
- Differentiable Logic Machines
- Treatment, evidence, imitation, and chat
- Towards Neural Theorem Proving at Scale
- On (Emergent) Systematic Generalisation and Compositionality in Visual Referential Games with Straight-Through Gumbel-Softmax Estimator
- Learning Translation Invariance in CNNs
- Deep Stable Learning for Out-Of-Distribution Generalization
- Evolution, Future of AI, and Singularity
- Evolutionary Developmental Biology Can Serve as the Conceptual Foundation for a New Design Paradigm in Artificial Intelligence
- A Hierarchy of Limitations in Machine Learning
- Learning Compositional Rules via Neural Program Synthesis
- Synergetic Learning Systems: Concept, Architecture, and Algorithms
- Opening the black box of deep learning
- Fit to Measure: Reasoning about Sizes for Robust Object Recognition
- Human-like generalization in a machine through predicate learning
- Symbolic Behaviour in Artificial Intelligence
- Predicate learning in neural systems: Discovering latent generative structures
- Semantic Communication meets System 2 ML: How Abstraction, Compositionality and Emergent Languages Shape Intelligence
- Compositional generalization in a deep seq2seq model by separating syntax and semantics
- Deep Adaptive Semantic Logic (DASL): Compiling Declarative Knowledge into Deep Neural Networks
- Language and Thought: The View from LLMs
- Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning
- On the Parallels Between Evolutionary Theory and the State of AI
- Combining Abstract Argumentation and Machine Learning for Efficiently Analyzing Low-Level Process Event Streams
- Continuous Thought Machines
- "Weak AI" is Likely to Never Become "Strong AI", So What is its Greatest Value for us?
- The curious case of developmental BERTology: On sparsity, transfer learning, generalization and the brain
- The Neuroscience of Transformers
- A Boxology of Design Patterns forHybrid Learningand Reasoning Systems
- Reconciling deep learning with symbolic artificial intelligence: representing objects and relations
- Cyberbullying Detection -- Technical Report 2/2018, Department of Computer Science AGH, University of Science and Technology
- AI Survival Stories: a Taxonomic Analysis of AI Existential Risk
- A Road Map to Strong Intelligence
- Human Activity Recognition Using Inertial, Physiological and Environmental Sensors: A Comprehensive Survey
- ML-PersRef: A Machine Learning-based Personalized Multimodal Fusion Approach for Referencing Outside Objects From a Moving Vehicle
- Foundational Requirements for Artificial General Intelligence: A Falsifiable Framework Based on Signal Prediction
- Siamese recurrent networks learn first-order logic reasoning and exhibit zero-shot compositional generalization
- Rethinking supervised learning: insights from biological learning and from calling it by its name
- AI anthropomorphism [wikipedia]
Discussions
- Deep learning: a critical appraisal [hn, 147 points, 61 comments]
- Deep Learning: A Critical Appraisal (2018) [hn, 77 points, 41 comments]
- Deep Learning: A Critical Appraisal [hn, 5 points, 0 comments]
- Deep learning: a critical appraisal [pdf] [hn, 4 points, 0 comments]
- Deep Learning: A Critical Appraisal [pdf] [hn, 4 points, 0 comments]
- Deep Learning: A Critical Appraisal [hn, 3 points, 0 comments]
- Deep Learning: A Critical Appraisal by Gary Marcus [hn, 3 points, 0 comments]
- Deep Learning: A Critical Appraisal [hn, 3 points, 0 comments]
- Deep Learning, a critical appraisal [hn, 2 points, 0 comments]
- In his 2018 critical appraisal of deep learning, @garymarcus.bsky.social made a similar point that he has been making since: while performance can be mustered in "some limited" circumstances, it's "qu [bsky, 2 points, 1 comments]
- Deep Learning: A Critical Appraisal [pdf] [hn, 1 points, 0 comments]
- A good critical look at deep learning https://arxiv.org/pdf/1801.00631.pdf [bsky, 0 points, 0 comments]
- Weekend reading. Going back to a paper by Gary Marcus from 2018 Marcus, Gary. “Deep Learning: A Critical Appraisal,” January 2, 2018. doi.org/10.48550/arX.... #academicSky #PhilAI [bsky, 0 points, 0 comments]
Related